Stabilo video stabilizer
It appears that this has all been written. Which is fine. It was a fun exercise, but I am no mathematician.
I appears that what I am trying to do (crudely and slowly) is perform two operations: Image Registration and Motion Vector Calculation.
- Image registration. Image registration gives the transformation operations necessary to maximize the correlation between image A and image B. The operations considered are linear transformations (rotation, scaling, shear) and translation (or shift). In my case, I consider only rotation and X and Y translation. I believe that my mathematician friends call this an "Affine transformation". Note that an image from a camera lens can also transform an image on other ways that I do not consider here. For example, I do not consider "scale". This would happen if you are walking towards a subject (or zooming) while filming. For extremely fast rates of zooming it may not be possible to register the images.
- Motion vector calculation: The X, Y shift and rotation angle give a motion vector. This describes how to convert image A into image B so that the new transformed image has a maximal correlation. This basically undoes any motion of the camera between image A and image B.
-- Noah 2010-10-21
stabilo -- video stabilization filter
This script processes successive frames of a video stream and finds the offset between each frame. It then adjust each frame of the output to minimize the offset. The result is smoother video.
This is a working alpha project at the moment.
This requires a patch to PIL Imaging-1.1.6 to add an RMS function, difference_rms(), to the ImageChops module. This function will calculate the RMS difference between two images. I could have implemented this in pure Python, but writing the function in C yielded a 10X speed increase.
This unprocessed video was shot from a paraglider. This shows a bad, shaky video -- I was a thousand feet up, holding the camera with one hand and steering the glider away from the rocks with the other hand.
This processed video shows how the algorithm smooths the video. In this example, offset compensation is unrestricted so you can see how the video will wrap around as the algorithm tries to chase a moving video.
The code is fairly simple. Currently the algorithm does not check if rotation of the image will improve smoothness. Rotations are very slow and don't help much. I have code stubbed out to add this feature as a final enhancement once a good translation offset is found.
The algorithm is fairly stupid -- it's a O(N^2) search. I have a few ideas for improving this. One way would be by sampling different offsets and ignoring regions that hurt smoothness and then returning to regions that improved smoothness... Apparently, using the FFT to convert to the frequency domain makes registration a much simpler problem; however, the FFT operation is relatively slow and would have to be done on every frame. Perhaps registering just a small sample space in the middle of the image would work well enough. Once registration is found for a small subsection you then apply the same transform to the entire image.
Click to download: stabilo.py <include svncat src="file:///home/svn/src/python/stabilo/stabilo.py" highlight="python" />