Image denoising using multiple motion compensated images.

Here is a method for reducing noise in a images by using multiple shots taken in sequence and blending them together. The key to this method is each shot is motion compensated to match a reference shot, so modest amount of subject or camera movement have little affect the outcome. That is, this method does not require a tripod, or a perfectly still subject.

Results:

The five images used for blending

The five images used for blending

Comparison of noise reduction techniques

Clockwise, from top left:

    The original image, shot 2 stops under exposed at ISO 1600, and then converted from RAW;
    A version of the original image process by traditional noise reduction method, using Noiseware;
    The image blended using my method;
    and finally, my method plus Noiseware.

Artefacts near edge of image

Artefacts near edge of image

Artefacts

All is not perfect however. Things get a bit blocky around the edge of the image (and in some other places where there is strong parallax). This is because too few images have the subject in frame, and so the algorithm cannot work properly.

How it works:

You take several images of your subject, technically the more the better. Pick one as your final image, or target. The software then compares the other images to the target and moves blocks of pixels around to best match the target image, producing a new image. The other images, which now match the target image very closely and then blended with the target to produce the final image.

It works best if there is a lot of overlap in each image, and if objects are visible in most of the images. It works reasonably well with parallax, but if there are things that are only visible in your target and not in the other images you will get artefacts.

I Used:

AviSynth – A video frameserver
MVTools filter for AviSynth
VirtualDub – A simple video editing program

AviSynth and MVTools are the brain of the operation. VirtualDub does next to nothing!

Avisynth takes a script as input and processes. Here is the script I used.
——————

# Create single frame video clips from input images. Cropping so that height is divisible by 2 and height is divisible by 4
pic1=ImageReader("F:\Denoise\DSC_4684.NEF.jpg",start=1,end=1).crop(0,0,2012,3038).converttoyuy2()
pic2=ImageReader("F:\Denoise\DSC_4685.NEF.jpg",start=1,end=1).crop(0,0,2012,3038).converttoyuy2()
target=ImageReader("F:\Denoise\DSC_4687.NEF.jpg",start=1,end=1).crop(0,0,2012,3038).converttoyuy2()
pic3=ImageReader("F:\Denoise\DSC_4688.NEF.jpg",start=1,end=1).crop(0,0,2012,3038).converttoyuy2()
pic4=ImageReader("F:\Denoise\DSC_4689.NEF.jpg",start=1,end=1).crop(0,0,2012,3038).converttoyuy2()

#create 2 frame clips of images and target
t1=pic1+target
t2=pic2+target
t3=pic3+target
t4=pic4+target

#compare pic1 to target and generate new motion compensated image
super1 = MSuper(t1,pel=4)
analyse1=MAnalyse(super1,blksize=8,search=3,searchparam=50,isb = false,dct=0,chroma=true,overlap=4)
clean1=MCompensate(t1,super1,analyse1).trim(1,1)

#compare pic2 to target and generate new motion compensated image
super2 = MSuper(t2,pel=4)
analyse2=MAnalyse(super2,blksize=8,search=3,searchparam=50,isb = false,dct=0,chroma=true,overlap=4)
clean2=MCompensate(t2,super2,analyse2).trim(1,1)

#compare pic1 to target and generate new motion compensated image
super3 = MSuper(t3,pel=4)
analyse3=MAnalyse(super3,blksize=8,search=3,searchparam=50,isb = false,dct=0,chroma=true,overlap=4)
clean3=MCompensate(t3,super3,analyse3).trim(1,1)

#compare pic1 to target and generate new motion compensated image
super4 = MSuper(t4,pel=4)
analyse4=MAnalyse(super4,blksize=8,search=3,searchparam=50,isb = false,dct=0,chroma=true,overlap=4)
clean4=MCompensate(t4,super4,analyse4).trim(1,1)

#blend compensated clips with each other
clean12=Overlay(clean1,clean2, opacity=0.5)
clean34=Overlay(clean3,clean4, opacity=0.5)
clean1234=Overlay(clean12,clean34, opacity=0.5)

#Blend with target
out=Overlay(clean1234,target, opacity=0.2).converttorgb()

#Write image to disk
ImageWriter(out,file="F:/Denoise/clean", type="bmp")

———————-

Open the script in VirtualDub and out pops a clean image – F:/Denoise/Clean00000.bmp

It is possible to modify the script so that each motion compensated image is saved, and can then be imported as a separate layer into your image editor. This would allow you to minimise artefacts by using masks.

It took 6 minutes to process this 6megapixel image on my 2 core 2GHz Intel processor. Unfortunately, AviSynth does not run in separate threads unless you use a special plug-in, so I was only using half of my CPU power. I did use settings fro MVTools that require a lot of processing. Each 8×8 block of pixels is compared to a lot of nearby blocks with sub-pixel precision, and blocks are overlapped as well. All this effort reduces artefacts, but may be unnecessary in some cases.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s