A team of researchers says they have successfully created an immersive 3D augmented reality headset using ordinary glasses instead of the bulky apparatus employed by typical VR and AR headsets.
While still in the prototype phase, the researchers believe that their augmented reality 3D glasses could impact anything from gaming and social media environments to training and education applications.
“Our headset appears to the outside world just like an everyday pair of glasses, but what the wearer sees through the lenses is an enriched world overlaid with vibrant, full-color 3D computed imagery,” said Stanford’s Gordon Wetzstein, an associate professor of electrical engineering.
“There is no other augmented reality system out there now with comparable compact form factor, or that matches our 3D image quality,” added Gun-Yeal Lee, a postdoctoral researcher in the Stanford Computational Imaging lab and co-first author of the published paper outlining the team’s efforts.
Bulky Headsets Have Limited Application of Augmented Reality
The Stanford researchers faced a number of technical hurdles in combining the benefits of current VR headsets with the versatility and ease of ordinary glasses. The first was reducing or replacing the complex optical systems used in bulkier headsets. That’s because these headsets don’t allow the viewer to see the landscape directly.
Instead, tiny cameras capture what the wearer is “looking” at and then project a viewable, 3D image on a tiny screen embedded within the device. For augmented reality purposes, any data used to aid the viewer is overlaid on this digital projection rather than appearing on top of the actual image.
“The user sees a digitized approximation of the real world with computed imagery overlaid,” explained Lee. “It’s sort of augmented virtual reality, not true augmented reality.”
Unfortunately, these types of headsets employ magnifying lenses between the user’s eye and the screen. To work properly, this layout requires a set distance between the eye, the lens, and the screen, resulting in the oversized, bulky headsets currently on the market. According to user experience, this setup can also detract from the overall immersive “realness” of the augmented reality experience, defeating the entire point of the system.
“Beyond bulkiness, these limitations can also lead to unsatisfactory perceptual realism and, often, visual discomfort,” explained Suyeon Choi, a doctoral student in the Stanford Computational Imaging lab and co-author of the paper.
80-Year-Old Technique Meets Cutting-Edge Science to Break Technological Barriers
Determined to overcome these obstacles, the Stanford researchers turned to a Nobel Prize-winning technology first developed in the 1940s: holography. According to the team, this technique has seen limited use in VR and #D systems due to issues with the depth perception of visual cues, “leading to an underwhelming, sometimes nausea-inducing, visual experience.”
Still, the researchers believed that holography was the key to eliminating the bulky lens-and-screen technique and creating a true, augmented reality headset that allows the viewer to see the real image while simultaneously receiving overlaid information about that image in real time. The key, they determined, was improving the 80-year-old holography techniques with the power of 21st-century AI computing to dramatically improve the depth cues of the projected holographic images.
“With holography, you also get the full 3D volume in front of each eye increasing the life-like 3D image quality,” said Brian Chao, a doctoral student in the Stanford Computational Imaging lab and also co-author of the paper.
The team also employed modern breakthroughs in nanophotonics and waveguide technologies, which increased the complexity and overall realness of the projected images. According to the press release announcing the breakthrough, “A waveguide is constructed by etching nanometer-scale patterns onto the lens surface. Small holographic displays mounted at each temple project the computed imagery through the etched patterns which bounce the light within the lens before it is delivered directly to the viewer’s eye.”
As a result, someone wearing the team’s new ordinary-looking glass lenses can see both the “real world” and the 3D computer-enhanced augmented reality images displayed on top of the world.
Potential Applications Include Gaming, Military Simulations, and On-The-Job Training
Having completed their initial prototype, the team says they can now envision a number of potential applications for their system. Some obvious uses include enhanced gaming or other recreational uses that have thus far eluded current bulkier headsets. However, the team says they believe their system could dramatically improve the work done by highly-trained experts in medicine or engineering.
“One could imagine a surgeon wearing such glasses to plan a delicate or complex surgery or airplane mechanic using them to learn to work on the latest jet engine,” said Manu Gopakumar, a doctoral student in the Stanford Computational Imaging lab and co-first author of the paper.
Although more work is expected before the augmented reality headset made from ordinary glasses is commercially available, the Stanford team says their ability to combine modern AI techniques and nanophotonic breakthroughs with 1940s holography is the step that folks in their industry have been waiting for.
“Holographic displays have long been considered the ultimate 3D technique, but it’s never quite achieved that big commercial breakthrough,” Wetzstein said. “Maybe now they have the killer app they’ve been waiting for all these years.”
Christopher Plain is a Science Fiction and Fantasy novelist and Head Science Writer at The Debrief. Follow and connect with him on X, learn about his books at plainfiction.com, or email him directly at christopher@thedebrief.org.