The End of Hardware Book - Augmented Reality Displays

Augmented Reality is more than Virtual Reality

Upscaling

Perfect 3D recording requires a camera array to gather at least a sufficient collection of perspectives for 3D modeling or holographic encoding.
Pixel rasters from different cameras are never perfectly aligned, due to different perspectives and lots of possible calibration errors. Essentially the alignment in a one pixel range can be considered random. Recorded patterns can be realigned by software, but in any case about 3/4 of the camera signals involved have to be dropped because they don't fit into a common raster with at least about 1/2 pixel accuracy in x and y direction.

This difficulty can be turned into an advantage: We use low resolution cameras with good optics and an active pixel size smaller than the raster. The overlaid pixel arrays then look like this:

We see that a sufficient number of cameras can entirely fill a higher resolution raster, e.g. 2x or even 4x higher resolution than the single camera. Of course, not all high resolution pixels will always fit to that of at least one camera sensor.
We may require a fit of at least 1/2 (i.e. ±1/4) high resolution pixel size in x and y direction.

Probabilities not to hit a target, simply multiply as often as we try (this is a special case of the more general binomial distribution*). Applying this to various configurations, we get the following probability table:

number of cameras	Resolution multiplication 2 3 4
number of cameras	Resolution multiplication 2 3 4
4x4	0,356	0,637	0,777
5x5	0,199	0,494	0,675
6x6	0,098	0,363	0,567
7x7	0,042	0,251	0,462
8x8	0,016	0,165	0,365
9x9	0,005	0,102	0,279
10x10	0,002	0,060	0,207
11x11	0,000	0,033	0,149
12x12	0,000	0,017	0,104
13x13	0,000	0,009	0,070
14x14	0,000	0,004	0,046
15x15	0,000	0,002	0,029
16x16	0,000	0,001	0,018

Probability that a high resolution pixel is not aligned to at least one
camera pixel by at least ±1/4 pixel width in x and y direction

6x6 cameras for 2x resolution will already yield an 98% chance for pixel coverage. Even fewer cameras will suffice if we use a temporal smoothing, i.e. averaging between subsequent frames. So we could increase both resolution and reduce noise. This upscaling variety has meanwhile been implemented for single cameras, where only the temporal variations are available: substantial research has been done in this area at TU Berlin, in context with 3D reconstruction, and even a first end user application, from another source, has meanwhile appeared on the web.

The above calculations will principally apply to both, the number of cameras, or the number of sequential frames used.

For perfect 3D, we will need some more than the minimum number of cameras, naturally enabling much higher resolutions, 4x for example, more than sufficient for full scale cinema. The fact that we can use low resolution cameras would make it affordable to use even hundreds of them. Giving each of them a dedicated processor chip and wiring them in an array, would solve the computing requirements. In the end, we would get a signal hardly more complicated than that from a conventional pair of stereo cameras.

What has to be addressed: if we should use an object based 3D modeling as known from computer games, or a virtual millimeter wave hologram, or some other format not yet known.

____________________

home more notes order

Copyright © 2006-2011 Rolf R. Hainich; all materials on this website are copyrighted.
Disclaimer: All proprietary names and product names mentioned are trademarks or registered trademarks of their respective owners. We do not imply that any of the technologies or ideas described or mentioned herein are free of patent or other rights of ourselves or others. We do also not take any responsibility or guarantee for the correctness or legal status of any information in this book or this website or any documents or links mentioned herein and do not encourage or recommend any use of it. You may use the information presented herein at your own risk and responsibility only. To the best of our knowledge and belief no trademark or copyright infringement exists in these materials. In the fiction part of the book, the sketches, and anything printed in special typefaces, names, companies, cities, and countries are used fictitiously for the purpose of illustrating examples, and any resemblance to actual persons, living or dead, organizations, business establishments, events, or locales is entirely coincidental. If you have any questions or objections, please contact us immediately. "We" in all above terms comprises the publisher as well as the author. If you intend to use any of the ideas mentioned in the book or this website, please do your own research and patent research and contact the author.