NEW AUTOSTITCH PANORAMA SOFTWARE
PhD student Matthew Brown and supervising professor David Lowe of the University of British Columbia have developed software that automatically recognizes and stitches panoramas.
by Michelle Bienias
University of British Columbia computer science PhD candidate Matthew Brown has developed software that automatically stitches multiple photos to create seamless panoramas of up to 360-degrees. Brown’s AutoStitch Panorama software automatically recognizes and matches similar images. This is a major step forward in object recognition and computer vision, says Brown’s supervisor Professor David Lowe, a leading researcher in the field.
The matching process of unordered image sets is fully automated and quick, taking only three minutes to match and register the images and render the panorama, as compared to current software that requires the user take the photos in a fixed sequence and then manually identify and align them, a time-consuming process that requires some technical skill of the user. Brown, 25 and a native of Manchester, England, is hoping to improve on that time in the future.
Brown and Lowe built on Lowe's previous research to create the Autostitch software, which uses a probabilistic model to detect and verify similarities to match the images and then automatically stitch them into the panorama. The software requires no special camera.
(Sample images produced by AutoStitch Panorama software can be viewed here.)
Brown is about to present a paper on the research, entitled 'Recognizing Panoramas', for the first time at the 10th International Conference on Computer Vision in Nice, France, October 13-16.
One restriction the two researchers hope to overcome soon is to enable the software to match images of one scene taken from a multitude of locations.
For now, they're hoping an outside company will license the software and develop it further for commercial use. Virtual tourism websites and online walkthroughs of interiors to sell real estate are just two practical applications where Panorama software is already in use.
After Brown returns from the Nice conference, he will be heading to Microsoft and a four-month internship with Rick Szeliski, a pioneer in this area of computer science research.
We caught up with Brown via email to ask him a few questions:
 Currently, your software produces cylindrical panoramas; are you working on enabling cubic panoramas? What are the problems, restrictions and issues with automatically producing cubics?
 When our software has run we have the rotation and focal length of each of the cameras in the sequence. We can then create any surface around the shooting position (sphere, cylinder, cube, etc. etc.) and render the rays to that. So far, we have just implemented this for the spherical case, but it would be straightforward to add the ability to render to a cylinder or cube.
You are close to enabling the software to produce object-oriented panoramas I understand; how do the technical issues in developing objects differ from the typical cylindrical pano?
We are interested in the problem of creating 3D models from unordered datasets in the same way that we have with panoramic images. The idea then is that the user takes multiple shots of a number of scenes, and for each scene the camera center is moving (unlike for a panorama where it is fixed), then the computer would automatically recognize the matching images, and generate 3D models. There is software to do this already e.g. Boujou software from 2d3, Realviz MatchMover etc., but it operates on video sequences. Our interest is to incorporate object recognition into this problem so that the software can automatically pick out separate 3D objects / scenes from a database of images.
If you reverse a panorama by moving the camera around a sphere and taking all rays "through" a point instead of keeping the camera fixed and taking all rays "from" a point, you get a cyclograph.
However, you "can't" generate all views of the object from a cyclograph in the same way that you "can" generate all views of a scene from a panoramic image. So cyclographs aren't so interesting in my opinion.
However, given a set of images of a scene taken from a moving camera, it is generally possible to build a 3D model of the scene that allows you to render any view of it. This problem is still the subject of a lot of research.
What companies do you think are best qualified to license and develop your software for commercial use?
The software is available as a UBC License 03-136 AutoStitch. Potential licensees should contact Randy Smith randy.smith@uilo.ubc.ca. We're open-minded about the type of company that might want to license the software. We could see it being used in any of three ways:
1. Standalone application.
2. Bundled with camera software - the software would automatically recognize and stitch matching images when the user downloads his/her digital images to the computer.
3. Incorporated into the OS - eventually, users will be able to plug cameras directly into the computer, and the OS will recognize the camera and do all sorts of processing on the image sets. As well as stitching panoramas it could rotate images to have the correct orientation, and establishes matches to other images in the users collection e.g. people’s faces etc.
Your website states that you solve the camera parameters, including focal length, using bundle adjustment. Can you explain this process?
Bundle adjustment is a technique from the computer vision research field that minimizes an error function based on all of the images in the dataset. The rotation and focal length of each camera are parameters in this optimization problem. Intuitively it is very much like the way people would align their photos. You move them around until the features in the images e.g. lines, corners etc. match up nicely!
Is it correct to interpret that your software can recognize and create multiple separate panoramas from a flash card or film (i.e. I shoot 4 panos comprised of 32 shots each, and load them into the software, and it automatically recognizes which shots correspond with each pano.)?
This is correct. The software recognizes multiple panoramas in an image set and stitches them fully automatically.
How is it that the software is insensitive to the ordering, orientation, scale, illumination and noise of the images?
The reason that the input images can have any scale, orientation, illumination etc. is that the features we use are invariant to these changes. These features come from about 20 years of object recognition research in the computer vision field.
How long have you and Lowe worked on this project?
I've been working on my PhD for 2 years with David Lowe, but we started the Recognizing Panoramas project in September 2002.
How much work, time-wise, do you think it will take to create a commercial product once a licensee is found?
As for how long it will take to develop a commercial product, there is not a lot of work to do. The code needs to be ported from MATLAB probably to C/C++ and a GUI built.
Email Matthew Brown: mbrown@cs.ubc.ca
For licensing options please contact Randy Smith at UBC Industry Liason Office email: randy.smith@uilo.ubc.ca quoting 03-136 AutoStitch
|  | |
| The purpose of this banner is to raise funds for a new VR community project VRMag will launch in a few months.
| |
