AlignFields 0.9

An Avisynth 2.5x plugin for deinterlacing video clips

By Rainer Wittmann gorw@gmx.de

The binary and the source code   are subject to the GNU General Public License .  March 13, 2004

Overview

AlignFields is a multi-mode deinterlacer. Those who want quick results and don't like to read documentation should use AlignFields without any arguments. AlignFields is intended for interlaced clips. On one hand it should preserve a maximum amount of details and on the other hand the output should be highly compressible (see the comparison table below). It is not intended for telecined clips. However, the damage on other kinds of clips is limited. Thus AlignFields() without any argument is for the careless user, who doesn't know exactly what kind of clip he wants to process. Even Higher compression can be achieved with AlignFields(mode=3). However, this mode should definitely only be used for interlaced clips and the user must specify the correct field order (the default is top field first). With modes 0,1,3 AlignFields allows also to double the frame rate, giving very smooth motion or high quality 50% slow motion like Donald Graft's DGBob. AlignFields supports all of Avisynth's color spaces, but it was designed for YUV color spaces, in particular for YV12. It requires a cpu with integer SSE capability. All questions, comments, etc. should be posted to the AlignFields thread  of the doom9 forum. No email please.

Installation

The binary package of AlignFields contains two versions of the plugin. AlignField.dll (dynamically linked, hence small) and AlignFieldS.dll (staticly linked, hence big). Try first the small dll and copy it to Avisynth's plugin directory. If it doesn't work, this is probably due to a missing msvcr70.dll library. Either you install this library in C:\windows\system32 or you delete AlignField.dll and replace it by AlignFieldS.dll. Please do not put both dlls into the plugin directory. If you fill the plugin directory with all kinds of superfluous dlls, then you only slow down the start of any application which uses Avisynth.

No Motion No Change

Interlace creates ugly combs, which result in very poor compression, only if there is motion. Thus the natural idea for deinterlacing is to change a frame only in those parts, which contain motion. Thus in areas of a frame without any motion there should not be any change. I call this the "no motion no change" principle. In all its four modes AlignFields adheres to this fundamental principle. Surprisingly I know of no others deinterlacers which truely obey this principle. Donald Graft's Fielddeinterlace(full= false) implementsa global "no motion no change" principle, though. However, if there is motion in a sufficiently large area, then all areas - whether they contain motion or not - are deinterlaced by Fielddeinterlace(full= false). AlignFields, on the other hand only deinterlaces those areas, which contain motion. The no motion no change principle, prevents a deinterlacer from unnecessary blurring to achieve a higher compression ratio. The following Avisynth script can be used to check the validitiy of the no motion no change principle of a deinterlacer.

ImageReader("MyPicture.jpg", 0,1000,25,true) input=ConvertToYV12() deinterlaced = MyDeinterlacer(input) difference(deinterlaced, input)

Here MyDeinterlacer has to be replaced by the Deinterlacer of your choice. AlignFields must be installed, because it contains also the very simple filter difference which compares the frames of two clips and sends the results to debugview, which must be installed as well. MyPicture.jpg should be a picture with rich details (its width and height must be a multiple of 2 such that ConvertToYV12 works properly).

After having discussed, when and where to deinterlace, we now discuss the question, how to deinterlace. There are basicly two strategies: sharp deinterlacers and blur deinterlacers. A sharp deinterlacer usually preserved one field and interpolates the other in motion areas. For this interpolation only pixels from the preserved field are used. Consequently, a sharp deinterlacer removes combs completely. On the other hand, a blur deinterlacer, averages a motion pxiel with neighbouring pixels such that both fields have equal weight within these averages. Consequently, combs are not erased, they are only converted into shadows or ghosts. Now, why should one prefer a blur over a sharp deinterlacer, if it only converts combs into ghosts? The answer: sharp motion does not bode well for codecs and the human eye likewise. For the human eye sharp motion at 25 fps will be quite jerky, if the speed is sufficiently high. It can watch a video much more relaxed, if there is no fast, sharp motion. Anyway, the human eye can only concentrate on details in areas without motion. These are the reasons, why most deinterlacers are blur deinterlacers. Tomsmocomp is an outstanding exception. For instance, Tomsmocomb(1,5,0) completely preserves one field and nevertheless achieves fairly good compression ratios. However, it does additional blurring to achieve these compression rates, whence violates the no motion no change principle. Tomsmocomp(1,5,1) does a lot more blurring and no more preserves any field. The compression ratios are even better, but it massively violates the no motion no change principle (check it with the above script). Alignfields(mode=0) and AlignFields(mode=1) are sharp deinterlacers as well. They preserve one field (if topfield=true, they preserve the top field otherwise the bottom field) and as all AlignFields deinterlacers they adhere to the no motion no change principle. There compression ratios are slightly worse than Tomsmocomb(1,5,0) and clearly inferior to Tomsmocomp(1,5,1). AlignFields(mode=0) produces the sharpest pictures and is particularily useful for snapshots with VirtualDub. AlignFields(mode=0) is fairly slow and AlignFields(mode=1) is fairly fast. Both produce nearly identical compression ratios. AlignFields(mode=2) (the default mode) and AlignFields(mode= 3) are blur deinterlacers, with very high compression ratios. They get beaten by other deinterlacers only, if a clip generated from a single picture by ImageReader is taken. While AlignFields never changes such a clip, it is blurred by other deinterlacers, which usually results in a better compression ratio. AlignFields(mode=3) uses a fairly sophisticated temporal-spatial averaging and achieves higher compression rates than AlignFields(mode=2), which uses only spatial averaging. Because of the temporal component it has to be used with care. Firstly, you must get the field order right. It can be specified with the variable tff. If tff=true (the default), then the input clip is expected to be top field first. If  tff=false then the input clip is expected to be bottom field first. Clips from digital satellite tv are always top field first in Europe. Clips from dv cameras are always bottom field first. Thus there is no need to check the field order in these cases, However, I wonder what happens if a dv camera clip is broadcasted over digital satellite tv. Anyway, we will see later, how to check the field order visually using AlignFields. It is less clear whether a clip is progressive or interlaced and AlignFields(mode=3) should never be applied to progressive clips. Again, we will see later how to decide this question visually with AlignFields. In all modes AlignFields uses the same motion detection. The motion detection is controlled by the two variables ythreshold (default value=40) and uvthreshold (default value=30). ythreshold is for the luma and uvthreshold is for the two chroma planes. The lower the values of these variables, the more deinterlacing. If they are too high, then combs are no more completely erased. If one of these variables is zero, the no motion no change principle is violated. AlignFields(mode=0) and AlignFields(mode= 1) are still producing reasonably good output if  ythreshold=0 and uvthreshold=0. This shows the quality of their interpolation method. For a black&white clip the option lumaonly=true saves time, because only the luma plane is processed. The chroma planes are left unchanged. Do not use this option for color clips. AlignFields(mode=3) is very sensitve to sharp scene switches. To avoid artifacts at scene switches, you have to enable scene switch detection with the variable scene. scene=3 is appropriate and should not be changed. If scene < 3 too many scene switches are detected and the qualitiy deteriorates. Clips with special effects like fast blinking large areas irritate the motion detection and AlignFields(mode=3) should not be used. In general AlignFields(mode= 3) should onlybe used for homogeneoushigh quality video not older than 10 years. For mixed (i.e. it contains also progressive or telecined parts) and old material always the default AlignFields(mode=2) should be used. For mode!=3 scene switch detection only costs time and even negatively impacts output quality. Thus scene= 0 is the default and you should always specify scene=3 if you use AlignFields(mode=3), otherwise sharp scene switches are not handled correctly.  

Doubling the Number of Frames

Donald Graft was the first, who implemented this feature into his DGBob filter. I use it primarily as a diagnostic tool to decide visually, whether a clip is interlaced, telcined, progressive or badly progressive (see the subsequent section). Also the field order can be decided this way. If this option is enabled, then for even ouput frame n the first field (if tff=true then this is the top field) of the input frame n/2 is taken as an anchor for deinterlacing and for odd frames n  the second field if tff=true then this is the bottom field) of the input frame n/2 is taken as an anchor for deinterlacing. In this way not only the number of frames is doubled but also the motion information of an interlaced video is exploited much better. Consequently the motion of the output video is much smoother. On the other hand such a video is bigger and if the frame rate is doubled, the cpu has to work much harder. AlignFields has two boolean variables slomo and fps2 (of course, for both variables false is the default value). If slomo=true, then the number of frames is doubled but the frame rate is left unchanged. Consequently, the output is a 50% high quality slow motion. If fps2=true and slomo=false, then not only the number of frames but also the frame rate is doubled. The output then has normal speed but the motion is significantly smoother. The drawback, of course, is that the number of frames is doubled and whence the compressed video will be much bigger. Because the output video alternates between topfield anchor and bottom anchor, subsequent frames do not fit together optimally and the frame rate doubling flicker appears. This flicker has a significant negative impact on compression. However, for Alignfield(mode=3, fps2=true) frame rate doubling flicker is much smaller than with Alignfield(mode=0, fps2=true) or Alignfield(mode=1, fps2=true). Thus if one wants smooth or slow motion videos one should always take mode=3. With the default mode=2 frame doubling is not possible.

AlignFields as a Diagnostic Tool

So far we have seen AlignFields as therapeutic tool against certain kind of combs, but to choose the right weapon in the war against combs we need also a diagnostic tool to determine the cause of the combs, which is crucial for the choice of the right filter. AlignFields(mode=0, fps2=true) can be used as such a tool. To this end we load the script

Mpeg2Source("input.d2v") AlignFields(mode=0, fps2=true)

into Virtualdub and move to a scene with a significant amount of motion. Then the clip has twice as many frames as the original clip. Each frame corresponds to a field of the original clip. The even frames correspond to the first fields of the original clip (i.e. the topfields if tff=true) and the odd frames correspond to the second fields of the original clip (i.e. the bottom fields if tff=true). Now we use vdub's arrow key -> to check whether there is motion between even and odd frames (we call this even/odd motion) and whether there is motion between odd and even frames (we call this (odd/even motion). If there is even/odd motion and odd/even motion, i.e. we have the motion pattern 111111 (0=no motion, 1=motion)  for several frames in a row, then the clip is an interlaced clip. If the motion looks jerky, i.e. two steps forward and one step backward, then the field order is incorrect and the tff variable should be changed.  If there is only odd/even motion but no even/odd motion, i.e. the motion pattern 01010101 then the clip is a correctly mastered progressive clip without combs. If there is only even/odd motion, i.e. the motion pattern 10101010 then the clip is an incorrectly mastered progressive clip (unfortunately about 5% of all progressive broadcasts in Germany are of this type). The combs of such clips are twice as large as the combs of interlaced clips with the same amount of motion. AlignFields should not be applied to such clips. Trbarry's Uncomb filter could be applied in such a situation but it doesn't work well in practice. It is better to use Avisynth's builtin filters SeparateFields and Weave to fix such a clip. Rarely it happens, that parts of the clip have only good odd/even motion while others have only bad even/odd motion. Such a poor quality progressive clip should be either thrown away or deinterlaced, but not with mode=3. If one observes the following motion pattern: 01001010010100101001 then the clip is a telecined progressive clip (3:2 pull down). For such kind of clips Donald Graft's Telecide but not AlignFields should be used. Rarely there are also different motion patterns caused by other kinds of telecining. The most evil technique to ruin a video is field blending. It is also very nicely displayed by the above script. Because this phenomenon is rare in Europe and because I don't have currently any good medicine against this kind of damage, I throw them all away.

When a Blur Deinterlacer is Bad

Sometimes at the end of a movie there are some very fast moving writings. If these are deinterlaced with a blur deinterlacer they are often hardly readable. It even happens that the end of an otherwise progressive movie is overlayed with a fast moving interlaced writing. If nothing is done against it, your clip ends with a bad impression. However, with Avisynth such a problem can easily be solved with a script like the following:

input=Mpeg2Source("input.d2v") blur = AlignFields(input,mode=3,scene=3) sharp = TomsMoComp(input, 1,5,1) start_fast_writing=65333 trim(blur, 0, start_fast_writing - 1) + trim(sharp, start_fast_writing,0)

As the name indicates the value of start_fast_writing is just the frame, where the fast moving writings begin. Thus for the critical part we simply use a sharp deinterlacer. Then the writings are probably a bit jerky but sharp. We could avoid jerkiness if we would double the frame rate for the last part of this movie, but unfortunately the frame rate must be constant throughout the clip.

The most extreme example of a clip, which performs very poorly with a blur deinterlacer, is the stock market ticker tape which is displayed on the bottom of the screen by business tv stations like CNBC. If the ticker tape is processed with a blur deinterlacer, the numbers are hardly readable while they are sharp with a sharp deinterlacer. In fact, I use the ticker tape to decide, whether a deinterlacer is a blur or a sharp deinterlacer.

Optimal Usage

All deinterlacers have quality problems for some boundary pixels. AlignFields is particularily bad in this respect. Thus "crop after deinterlacing" is a golden rule. I always crop a few pixels on each side. Crop(8,4,-8,-4) is typical for most of my encodings. In this way I never have to care about deinterlacing boundary pixels. "Never resize before deinterlacing" is an even more important rule. It is always a very bad idea to resize clips, which are not truely progressive. If you really want to do it, separate the fields with SeparateFields(), then resize both fields and weave them together again with Weave().

Debugging

If AlignFields is used with debug=true, then AlignFields sends output like the following to debugview:

[1912] [791] AlignFields: y = 31224, u = 10738, v = 1386 [1912] [792] AlignFields: y = 32530, u = 13016, v = 1748 [1912] [793] AlignFields: y = 25660, u = 9666, v = 1416 [1912] [794] AlignFields: y = 27188, u = 9572, v = 1470 [1912] [795] AlignFields: y = 30890, u = 13564, v = 1734 [1912] [796] AlignFields: y = 35108, u = 14488, v = 2000 [1912] [797] AlignFields: y = 35357, u = 12520, v = 1958 [1912] [798] AlignFields: y = 37580, u = 14392, v = 1946 [1912] [799] AlignFields: y = 32636, u = 12484, v = 1590 [1912] [800] AlignFields: y = 29788, u = 10362, v = 1544 [1912] [801] AlignFields: y = 34548, u = 15368, v = 1554 [1912] [802] AlignFields: y = 34114, u = 15484, v = 1786 [1912] [803] AlignFields: y = 28474, u = 10418, v = 1154 [1912] [804] AlignFields: y = 27779, u = 9682, v = 1534

y = 27779 means that 27779 pixels of the y plane have been changed by AlignFields. Similarily, u = 9682 means that 9682 pixels of the u plane have been changed by AlignFields. The first number [1912] is the id of the process, which runs the script and the second number [804] is just the frame number.

For visual debugging, AlignFields has to be used with show=true. Then those pixels, which would get deinterlaced, are colored black. The coloring is different if only the chroma or part of the chroma of a pixel is deinterlaced.

All Variables at a Glance

Name	Type	Default Value	Remarks
mode	Integer	2	only 0,1,2,3
tff	Boolean	true
topfield	Boolean	true
fps2	Boolean	false
slomo	Boolean	false
ythreshold	Integer	40	>=0
uvthreshold	Integer	30	>=0
doscillation	Integer	260	>=0
scene	Integer	0	=3 if mode=3
lumaonly	Boolean	false
debug	Boolean	false
show	Boolean	false
cache	Boolean	true

Comparison of Various Deinterlacers

We compare properties and benchmarks of various fine deinterlacers. The benchmark clip has 6322 frames, no audio, 720x576 size and is high quality interlaced with motion above average. After deinterlacing a few pixels were cropped on each side. The clip was always compressed with Divx 5.1.1, bframes enabled, and quantizer 6 (1 pass quality vbr mode). When XviD 1.0 is out I will also provide similar benchmarks for XviD.

Deinterlacer	no motion no change	1 field preserved	type	output size	speed
AlignFields(mode=0)	yes	yes	sharp	41.290.042	46 fps
AlignFields(mode=1)	yes	yes	sharp	41.360.543	132 fps
AlignFields(mode=2)	yes	no	blur	33.654.126	97 fps
AlignFields(mode=3,scene=3)	yes	no	blur	31.786.665	67 fps
FieldDeinterlace()	no	no	blur	39.094.505	40 fps
KernelDeint(1)	no	yes	sharp(?)	41.873.749	58 fps
TomsMoComp(1,5,0)	no	yes	sharp	41.039.499	85 fps
TomsMoComp(1,5,1)	no	no	sharp	36.992.867	90 fps