Stabilo video stabilizer
From Noah.org
stabilo -- video stabilization filter
This script processes successive frames of a video stream and finds the offset between each frame. It then adjust each frame of the output to minimize the offset. The result is smoother video.
This is a working alpha project at the moment.
This requires a patch to PIL Imaging-1.1.6 to add an RMS function, difference_rms(), to the ImageChops module. This function will calculate the RMS difference between two images. I could have implemented this in pure Python, but writing the function in C yielded a 10X speed increase. This video filter processing is slow to begin with, so any whole factors of speed increase is a great help.
samples
This unprocessed video was shot from a paraglider. This shows a bad, shaky video -- I was a thousand feet up, holding the camera with one hand and steering the glider away from the rocks with the other hand.
before
This processed video shows how the algorithm smooths the video. In this example, offset compensation is unrestricted so you can see how the video will wrap around as the algorithm tries to chase a moving video.
after unrestricted
the source
The code is fairly simple. Currently the algorithm does not check if rotation of the image will improve smoothness. Rotations are very slow and don't help much. I have code stubbed out to add this feature as a final enhancement once a good translation offset is found.
The algorithm is fairly stupid -- it's a O(N^2) search. I have a few ideas for improving this. One way would be by sampling different offsets and ignoring regions that hurt smoothness and then returning to regions that improved smoothness.
Click to download: stabilo.py
#!/usr/bin/env python """stabilo SYNOPSIS stabilo [-v] [-?] DESCRIPTION This calculates offsets between a sequence of images to minimum the difference between each image. For processing files should in a subdirectory called "input" with filenames in the following format: 00000001.jpg 00000002.jpg 00000003.jpg OPTIONS -? : display this help -v : verbose output EXAMPLES The following mplayer examples may come in handy when working with videos and Stabilo. == Convert video file to a sequence of images == This will convert a video file to a sequence of JPEG images: mplayer -nosound -vo jpeg input.avi == Play a sequence of images == This will play all jpeg images in the directory as a video. No need to first convert them to a video file. This will work with most image types. Playing a JPEG sequence is shown here: mplayer "mf://output/*.jpg" -mf fps=24 == Convert a sequence of images to video file == Same idea as sequence playback, but use mencoder to make a video file: mencoder "mf://*.jpg" -mf fps=24 -o output.avi -ovc lavc -lavcopts vcodec=mpeg4 AUTHOR Noah Spurrier <noah@noah.org> LICENSE 2007 This script is in the public domain, free from copyrights or restrictions. VERSION $Id: stabilo.py 62 2007-08-16 02:07:06Z noah $ """ import sys, os, traceback import re import getopt import time from PIL import ImageChops from PIL import Image VERBOSE = False def exit_with_usage (): print globals()['__doc__'] os._exit(1) def parse_args (options='', long_options=[]): try: optlist, args = getopt.getopt(sys.argv[1:], options+'?', long_options+['help','?']) except Exception, e: print str(e) exit_with_usage() options = dict(optlist) if [elem for elem in options if elem in ['-?','--?','--help']]: exit_with_usage() return (options, args) def stabilo_scan (im1, im2, xrange=10, yrange=10, arange=1, xmax=50, ymax=50, amax= 10, last=(0,0,0)): global VERBOSE a = 0 deltas = {} for y in range(-yrange,yrange+1): for x in range(-xrange,xrange+1): dup_im1 = ImageChops.duplicate(im1) dup_im2 = ImageChops.duplicate(im2) dup_im2 = ImageChops.offset(dup_im2, x, y) # dup_im2 = dup_im2.rotate(a, Image.BICUBIC) size = dup_im1.size dup_im2.crop((40,40,size[0]-40, size[1]-40)) dup_im1.crop((40,40,size[0]-40, size[1]-40)) d = ImageChops.difference_rms (dup_im1, dup_im2) deltas[d] = (last[0]+x,last[1]+y,a) keys = deltas.keys() keys.sort() best = keys[0] if VERBOSE: print "Minimum RMS:", str(best) last = deltas[best] # = ImageChops.offset(im2, last_offset[0]+offset[0],last_offset[1]+offset[1]) # for a in range (-arange,arange+1,1): # dup_im1 = ImageChops.duplicate(im1) # dup_im2 = ImageChops.duplicate(im2) # dup_im2 = ImageChops.offset(dup_im2, last[0]+x, last[1]+y) # dup_im2 = dup_im2.rotate (last[2]+a, Image.BICUBIC) # size = dup_im1.size # dup_im2.crop((40,40,size[0]-40, size[1]-40)) # dup_im1.crop((40,40,size[0]-40, size[1]-40)) # d = ImageChops.difference_s (dup_im1, dup_im2) # deltas[d] = (x,y,a) return last def main (): global VERBOSE (options, args) = parse_args('v') # if args<=0: # exit_with_usage() if '-v' in options: VERBOSE = True else: VERBOSE = False try: os.mkdir("output") except: pass filename1 = "input/%08d.jpg" % 1 last_offset = (0,0,0) for n in range (2, 99): filename2 = "%08d.jpg" % (n) im1 = Image.open(filename1) im2 = Image.open("input/" + filename2) offset = stabilo_scan(im1, im2, 4,4,0, last=last_offset) last_offset = offset if VERBOSE: print n, offset, "output/"+filename2 im_out = ImageChops.offset(im2, offset[0],offset[1]) #im_out = im_out.rotate (last_offset[2]+offset[2], Image.BICUBIC) im_out.save ("output/" + filename2, "JPEG", quality=85) filename1 = "output/" + filename2 filename1 = "input/%08d.jpg" % n # if VERBOSE: print ImageChops.difference_rms (im1, im2) # if VERBOSE: print ImageChops.difference_rms (im2, im3) # #ImageChops.difference (im1,im2).show() # #if VERBOSE: print ImageChops.difference_rms (im1, ImageChops.offset(im2, -1, 0)) # stabil_scan(im1,im2) if __name__ == '__main__': try: start_time = time.time() if VERBOSE: print time.asctime() main() if VERBOSE: print time.asctime() if VERBOSE: print "TOTAL TIME IN MINUTES:", if VERBOSE: print (time.time() - start_time) / 60.0 sys.exit(0) except SystemExit, e: raise e except Exception, e: print 'ERROR, UNEXPECTED EXCEPTION' print str(e) traceback.print_exc() os._exit(1)