Intensity Based Image Mosaicing

We know that two images in a panoramic image sequence, or two images belonging to a planar scene, are related by an affine transormation. In particular, the image coordinates of any point in the scene, in two different images are related by an equation of the form:

The problem now becomes that of determining the transformation parameters between every two adjacent images, in order to merge the set of images into a single complete image.

The idea would then be to choose the parameters such that the sum of squared difference between all pixels between the two images is minimized. That is :

The problem is that of non-linear minimization which can be solved by an iterative algorithm. The algorithm that was used in the present work, namely, the Levenberg Marquardt algorithm, involved computation of the approximate Hessian matrix at each stage of the iteration. This in turn, involved the computation of partial derivatives of the error with respect to the parameters being solved for. The following shows the derivation:

As is a constant,

Using the above formula, the partial derivatives were calculated. They are:

The Hessian matrix (which is symmetric), and the gradient at each stage has entries

The equations solved at each stage belong to the linear system:

where is a tuning parameter that is adjusted according to the change in the sum of squared differences at each stage. The following describes the algorithm in more detail:

1. Choose a small start value for . (Current work used )
2. For each pixel in the reference image , do:
   1. Compute the transformed coordinates using the current estimate of the parameters
   2. Compute the error in intensity for each pixel location
   3. Compute the intensity gradients in the unregistered image, namely, and at
   4. Compute the partial derivatives
   5. Add the pixel's contribution to the Hessian and gradient matrices A and b.
3. Solve the system of equations and obtain the increment:

   

4. Refine the current guess to obtain the new guess:

   

5. If the sum of squared differences has decreased, reduce by a factor (typically 10). Else, increase a factor.
6. If the value of has reduced to a small amount (typically ) stop. Else, go to 2.

The above algorithm was quite robust, with falling to the minimum in about 20 iterations.