Photosynth

by Jin, 09-25-08 // 1 comments

I remember the first time I saw the virtual tour of the Titanic featured on Mediadome.com(R.I.P.) in 1997. It was the very first VT on the web, powered by Livepicture software. I was caught in one of those “wow” moments. I thought, this “web” thing will take off, to become something more interactive than just text and still images. Countless web technologies came and went since. VRML anyone? These days, I’m almost afraid to embrace a new technology. I do it defensively. It’s like every fall season when the new TV shows debut, I’m afraid to watch any of them because who knows if they’ll still be around after the third episode.

I feel compelled to write today, because it’s been a while since I got so excited about a piece of web technology. Its name is Photosynth.

Photosynth is a software application from Microsoft Live Labs and the University of Washington that analyzes digital photographs to build a three-dimensional point cloud of a photographed object. Pattern recognition components compare portions of images to create points, which are then compared to convert the image into a model. Users are able to view and generate their own models using a software tool available for download at the Photosynth website.

I first saw the demo video of Photosynth over a year ago, and I’ve been keeping an eye on it ever since. So instead of me attempting to describe what Photosynth does in words, I think it’s best you watch the demo video, presented by the architect behind this project, Blaise Aguera y Arcas

After much anticipation, Photosynth was released to the public a month ago. The launch was quite rough. The site quickly went to a static mode due to high traffic. 7727 synths were created on the first day, using 286,689 images. Even when the site did come back up, it often crashed unexpectedly.

The site is quite stable now, and I’ve gotten the chance to create a few synths myself.

As you saw in the demo video, Photosynth doesn’t “stitch” photos together to form a big panoramic photo like a typical pseudo 3D virtual tour program does. Instead, it intelligently arranges photos in a true 3D format, while maintaining their original integrity.

In order to create a synth, you need to download the free software from the Photosynth site. It contains the client app used to create and upload the synth, and a browser plugin. Currently the program only supports Windows XP/Vista machines. A Mac version is in the works. The browser plugin works for both IE and Firefox. You also need a Windows Live ID to sign in to the Photosynth account.

Photosynth Client

The client app is pretty straight forward. It looks a lot like Flickr‘s client uploader. You simply drag your photos into the box, give your synth a name and appropriate tags, then click on “Synth.” This action creates the synth, and uploads it to the server. When the process is done, you will receive a synth percentage. The percentage simply tells what percentage of the photos you uploaded were actually used in the synthing process. The more overlapping areas in your photos, the higher the percentage, and therefore a better 3d experience.

The best way to take “synthable” photos is to start shooting 360 photos from the center of the scene, each photo should have a decent overlapping area. Then do the opposite, go around the scene, shoot towards the center. This way, it covers the most area from different perspectives. Photosynth’s own blog explains this very well, with illustrations.

A couple of days ago, I took my new camera to the fish market in southwest Washington DC. I shot about 120 photos, and selected 82 photos for the synth process. Photosynth accounts come with a generous 20gig space. I ended up getting a 28% synth rate. I didn’t do a very good job on shooting overlaps. Each photo was uploaded in their original size, 4288×2848px, 300dpi. It’s pretty amazing what kind of details you can see. High quality photos aren’t needed, however, it really shows the viewer’s power. Check out my fish market synth for other angles.

Screenshot 1.

zoomed in on the red circle I drew in the screen shot above

My sample doesn’t do Photosynth justice. Here are some impressive synths: wedding, boxer, and National Geographics’s owns synths.

What the future holds for Photosynth remains to be seen. While I’m very impressed by the technology, I question its longevity. The way Photosynth is setup right now, IMHO it prevents itself from being popular among everyday people. Social networking capability makes or breaks web technology these days. Photosynth’s site is setup to be a social networking site on the surface, with blogs, comments etc. However, I don’t see it building up a fan base by itself. Most synths on the site are created by people for testing purpose, but that’s about it. People’s photos are contained in synths on their own accounts. There’s no interaction, no sharing between each user. Comments don’t count.

A typical person doesn’t go around shooting hundreds of overlapping photos around the same scene. This is why a community pool of photos from thousands of people will make Photosynth practical and useful. Very much like Flickr’s. Imagine if Photosynth had the vast photo resource Flickr did, you’d be able to go to a pool called “Niagara Falls,” view a synth made up by tens of thousands photos taken at all angles, during different times of the day, etc. That’d be a truly rich experience.

I hope Photosynth will not fizzle like other fads. It’s a revolutionary step towards finer photo manipulation and user interaction experience. I feel for it to succeed, the team will need to think harder about how to market it. Fine engineering alone cannot succeed today, without a strong user community to promote it.

If you’ve experienced with Photosynth before, I’d like to hear your thoughts.

1 comments

also feel free to contact me on twitter or via email

[...] is an awesomely impressive application for dealing with images and text on the web – a review by Jin has the Photsynth TED Conference presentation that explains what’s going on. It’s [...]