Refocusing digital photos after the fact
I dropped my subscription to the ACM Graphics SIG some time back, so this is the first I’ve heard of this project, which is very cool. Take your photos now, and decide what to focus on later.
From Wired News, via A Venture Forth:
A computer science Ph.D. student at Stanford University has outfitted a 16-megapixel camera with a bevy of micro lenses that allows users to take photos and later refocus them on a computer using software he wrote.
The system works by capturing information about the direction of the incoming light, as well as the intensity. This is then used to compute the image that would have been formed if the sensor was in a slightly different plane, effectively changing the focal length. The paper published by Ren Ng and team observes:
As an aside from the biological perspective, it is interesting to note that our optical design can be thought of as taking a human
eye (camera) and replacing its retina with an insect eye (microlens / photosensor array). No animal has been discovered that possesses such a hybrid eye [Land and Nilsson 2001], but this paper (and the work of Adelson and Wang) shows that such a design posseses unique and compelling capabilities when coupled with sufficient processing power (a computer).
The system works best with more data, 16 megapixels appears to work pretty well. They indicate that 8 megapixels would still work but with a narrower computed focus range. As shown in the schematic, the effective output resolution is limited by the microarray lens, not the sensor resolution, but it needs the high resolution sensor data to determine the direction of incoming light. The prototype is built in a medium format (Contex) body to make it easier to build the sensor assembly.
It doesn’t look like this is going to turn up in consumer devices any time soon, but I’m sure there are some interesting applications that can afford the cost and physical bulk of the system already.
Links:
- Light Field Photography with a Hand-Held Plenoptic Camera (project page at Stanford)
- Light Field Photography with a Hand-held Plenoptic Camera (Stanford Tech Report CTSR 2005-02, PDF)
- Gallery of movies demonstrating digital refocusing (Windows Media)
- The Plenoptic Camera (1992 project at MIT)
November 23rd, 2005 at 9:11 am
There is a resolution tradeoff that will keep this out of mainstream cameras for a while. The resolution of the resulting image will be the resolution of one imaged microlens. Given consumers’ obsession with resolution, to the point of paying for more even when it does not noticably improve image quality, I don’t see this being marketable until we have a huge surplus of pixels.
(Ren Ng gave a talk on this at HP Labs this summer).
November 23rd, 2005 at 9:31 am
I did notice that the output resolution is determined by the microlens array, not the underlying sensor. I didn’t spend enough time to get a sense of the tradeoffs between the cost, size, optical performance, and sensor noise, though. In particular, I have no idea how much the lens array currently costs, or how quickly they might improve.
Another thought I had was whether the scenes that had enough lighting to support the microarray system could be shot at a small enough aperture to cover a greater depth of field instead. This wouldn’t help for artistic composition, but would be a consideration for security and surveillance applications.