The End of Hardware Book - Augmented Reality Displays

Augmented Reality is more than Virtual Reality

Foreword by Oliver Bimber

I was captivated by the last edition of "The End of Hardware" on a round-trip flight to Los Angeles. This book is not only an in-depth introduction to the concept of head-attached displays for augmented reality (AR), but also a great source of inspiration for many professionals - at least it is for me.

The augmented reality community has carried out extensive research and development for many years to make the vision of AR come true. So far, head-attached displays have not found their way to the mass-market. One reason for this might be, the technological foundation still has too many limitations to become an efficient and widely applied user interface.

In "The End of Hardware", Rolf R. Hainich puts together concepts and ideas that he has collected for years from a large variety of fields to sketch out his vision of eye-worn displays. He explains, how advanced optics and electronics as well as electromechanics can be used to build personal displays that have the potential to overcome the limitations of today's devices.

Being a technical guy, I particularly enjoyed reading his technical design chapter which, in this new edition of the book, has been greatly extended with many details on holography, light fields and MEMS. Despite a technological focus, the book is written in a popular-scientific style - and therefore allows easy access to the material - even for non-experts.

If I were to characterize this book in a single word, that word would be "inspiring". I can only hope that one day, someone will pick up and realize these ideas. For although, this might not be the "End of Hardware" - it could well be the beginning of many new and exciting interfaces to the digital world in which we all live in.

Weimar, 23rd of March 2009
Oliver Bimber

Oliver Bimber is head of the Institute of Computer Graphics at Johannes Kepler University Linz. From 2003-2010 he served as Junior Professor of Augmented Reality at the Media System Science Department of Bauhaus-University Weimar.

3rd Edition - author's comment

When the first edition of this book appeared, in April 2006, a certain revival of Augmented Reality (AR) could already be noticed, yet almost entirely limited to research projects in software and to industrial applications. The idea of AR for private use had always been widely ignored except for a few pioneers, manly exploring it as an ingredient to wearable computing. In Wikipedia for example, simply none of the myriads of possible everyday AR applications had ever appeared before.
The book I therefore originally wrote with two main objectives in mind: bringing consumer AR back into common discussion, and promoting the development of dedicated hardware. The hardware issues are however exceeding the capability of almost any academic research facility, and without resolving them, most people have difficulties figuring out all the many possibilities with this technology. The typical AR research paper deals with rendering issues - hence software - and off-the-shelf hardware. Sometimes, quite similar issues are being taken up just with new hardware every some years.
The optics necessary, as it turned out at a closer look, can be approached in a way understandable by anybody with just a basic school education, even including often mystified topics like holography. So indeed anybody from other research disciplines will surely be able to evaluate the possibilities, and this is necessary to move anything here.
Lots of things happened since the first edition appeared.
Besides several university lectures, in the meantime I've been invited for a keynote at the ISMAR 2008 conference in Cambridge [123], where I took the opportunity to present the new metaphor of focus steered 'ghost' objects to a greater audience, along with many other materials that I had published in lectures and at the book's website since the last book edition. You will now find all of them in this 3rd edition as well.

Said conference also contained the first full fledged optical analysis of single mirror near eye display optics [103], and there it was proven that it would work, even without any electronic geometry or focus correction already. Another especially remarkable contribution was a paper about gaze steering [104].
Someone approached me about 'contact lens displays'. Pretty far fetched, but researching it a bit I discovered that some misunderstandings in the news media required the real possibilities to be addressed. So I added a few pages about it.
In the last edition of this book, I identified the eye operated cell phone as the likely killer application for general use AR and in May 2007, I proposed a version with double use optics at the book's website. Shortly after, a renown manufacturer of laser scanners [15] displayed an imagination of a spectacle phone on their website, seeking cooperations with cell phone manufacturers. Instead, micro projectors are now the next gadget being developed for mobiles. Projected screens however cannot replace near-eye displays, and the very same company also acquired a major public grant for the development of laser based display glasses. Another remarkable project is ISTAR [118]. So the topic is taking up momentum, and hope may be justified that these developments will be resulting in real products not too far away.
Meanwhile, old fashioned cell phones have been used for augmented reality experiments quite often, applications similar to some outlined here, but replacing the glasses with the phone display.

This is not the real thing of course, people walking around
pointing their phones like crazed vampire hunters...

Several interesting hardware developments have occurred that I wanted to include in the book. One is advances in piezo-electric micro motors, a promising solution for the dynamic adaptation we will most probably need for any kind of AR display, adapting it to the never ending dynamic movements of our eyes and allowing for focus adjustment and other goodies to come.

Not only hardware news are fueling this edition. I introduce the new 'ghost' objects already mentioned above, consisting of many layers that can be activated one at a time, according to the willfully changed distance accommodation of the beholder's eyes.

In the book's first edition, I proposed holographic encoding instead of the object oriented MPEG extensions usually tried for representing three-dimensional images. Just shortly after, it turned out that others were also following this idea [83], including practical experiments and conversions between holographic and light field representations. In this edition, I included new ideas for a light field display as well as multi camera acquisition and encoding, refurbished the holography section, and added a thorough introduction to holographic optical elements (excerpt).

Many more new or extended ideas and insights, about virtual worlds for example, improved and new explanations, made this book many pages stronger than the last edition (see page 11 for an overview). The fiction part received many and new items as well (example). I think it's space well spent.

Rolf R. Hainich

1st edition: The End of Hardware - A Novel Approach to Augmented Reality.
ISBN 1-4196-3036-9. 250 pages. March, 2006
2nd edition: The End of Hardware - A Novel Approach to Augmented Reality.
ISBN 1-4196-5218-4 300 pages. October, 2006

3rd edition: The End of Hardware - Augmented Reality and Beyond.
ISBN 1-4392-3602-X. 416 pages. May, 2009

notes content

Copyright © 2006-2011 Rolf R. Hainich; all materials on this website are copyrighted.
Disclaimer: All proprietary names and product names mentioned are trademarks or registered trademarks of their respective owners. We do not imply that any of the technologies or ideas described or mentioned herein are free of patent or other rights of ourselves or others. We do also not take any responsibility or guarantee for the correctness or legal status of any information in this book or this website or any documents or links mentioned herein and do not encourage or recommend any use of it. You may use the information presented herein at your own risk and responsibility only. To the best of our knowledge and belief no trademark or copyright infringement exists in these materials. In the fiction part of the book, the sketches, and anything printed in special typefaces, names, companies, cities, and countries are used fictitiously for the purpose of illustrating examples, and any resemblance to actual persons, living or dead, organizations, business establishments, events, or locales is entirely coincidental. If you have any questions or objections, please contact us immediately. "We" in all above terms comprises the publisher as well as the author. If you intend to use any of the ideas mentioned in the book or this website, please do your own research and patent research and contact the author.