Computer Vision, Fall 2016

Project #2: Panorama Mosaic Stitching

Click here for source codes!

Date Submitted: 5 Oct 2016

446216 (Ying Wang)

Project Description

In this project, you will implement a system to combine a series of photographs into a 360 degree panorama (see panorama above). You will first detect discriminating features in the images and find the best matching features in the other images, using your code from Project 1 (or SIFT). For this project, you will then automatically align the photographs (determine their overlap and relative positions) and then blend the resulting photos into a single seamless panorama.
-- Courtesy of Course Website

Overall Procedure

Basicly, the workflow of generating panoramic photos is as follows:

  • Take a bunch of photos.
  • Warp them to spherical coordinate system.
  • Align them properly.
  • Stitch, crop and blend them to generate the final panorama.
  • Details of each step

    1. Warping to spherical coordinate system

    The focal length of the camera is 380 pixel. This is estimated by the formula: focal length in pixels = (image width in pixels) * (focal length in mm) / (CCD width in mm). According to the image we took, the width in pixels is 720. The focal length in mm is 18. The CCD width in mm is 22.71. Thus, the focal length is around 720/22.71 * 18 = 571.

    warping

    2. Image align

    I use SIFT + RANSAC to imolement image alignment. The parameter I used are default 200 times for RANSAC and threshold 0.8 for SIFT method.

    3. Blend Images

    1. Alpha Blending

    I assign a weight function to the source image, which weight w(x) varies linearly from 1 to 0 at the edge.

    I notice that when applying alpha blending, choosing appropriate window size (blendWidth) is very important. ’Good’ window can generate smooth but not ghosted panorama. If we want to avoid discontinuities, we can set the window size equivalent to the largest prominent feature. To avoid ghosting, we can set the window less than 2*size of smallest prominent feature. While this method is easy to implement, it can also cause blurring of high frequency detail if there are small registration errors. Details will be discussed as folllows.

    Good Results

    1. Campus


    High resolution

    2. Crazed Interloper

    This is a pano which the same person frequently runs into the camera.


    High resolution

    3. Inside a building

    We can see noticable seams in this panorama because of the large intensity variance.


    High resolution

    Bad Results

    4. WashU [Bad Alignment]

    This panorama is actually a failure because the x translation in pairlist is only half of the actual x translation, which is only around 125. My idea is that there are several images whose texture is simple and similar to each other. Thus the alignmenet didn't work well among this set of images.


    High resolution


    High resolution

    5. Hillside in Forrest Park [Bad Focal Length]

    This pano is a failure because I used a wrong focal length parameter when warping images. Thus the result has many ghosting artifacts.


    High resolution

    Conclusions

    Let's take a look at the result and analysis what worked and what didn't.


    High resolution

    The Good: (green part)

  • 1. Drift works well. This panorama turns out reasonly well in that it had very little vertical drift. We can see the panorama is aligned horizontally.
  • 2. Alpha blending works well. We can barely notice the seam between pairing images.
  • 3. Radial distortion correction works well. Notice the left edge is arc-shape instead of a straight line.
  • The Bad: (red part)

  • 1. There are ghosting artifacts in those three regions. I think it is because I use a relatively large blend width to get a better result. As I decrease the blend width, the panorama became worse with more and more visible boundaries and sometimes even strange artifacts. Compare the following results with different blend widths: 50, 100, 200, 300, 400.
  • blendWidth = 50
    50 High resolution
    blendWidth = 100
    50 High resolution
    blendWidth = 200
    50 High resolution
    blendWidth = 300
    50 High resolution
    blendWidth = 400
    50 High resolution

    Bell & Whistles(Extra Points)

    1. Support radial distortion correction inside WarpSpherical.cpp.

    Without radial distortion correction
    With radial distortion correction

    Apparently, image with radial distortion correction has less artifacts near the boundaries. Thus we can get a sharper and clearer panorama using this method.

    2. Try a sequence in which the same person appears multiple times