Jonghyun Kim

Jun 14, 2024 | developer.nvidia.com | Jonghyun Kim

Today’s immersive extended reality (XR) devices require heavy optics and displays to be secured with head straps, a necessity that adds bulk and presents a social barrier. In the envisioned future, head straps will not be required. Over the past few years, NVIDIA Research has been researching how to develop smaller and lighter XR glasses in collaboration with the Computational Imaging Group at Stanford under Professor Gordon Wetzstein.

May 8, 2024 | nature.com | Manu Gopakumar |Suyeon Choi |Brian Chao |Yifan Peng |Jonghyun Kim |Gordon Wetzstein | +1 more

Emerging spatial computing systems seamlessly superimpose digital information on the physical environment observed by a user, enabling transformative experiences across various domains, such as entertainment, education, communication and training1–3. However, the widespread adoption of augmented-reality (AR) displays has been limited due to the bulky projection optics of their light engines and their inability to accurately portray three-dimensional (3D) depth cues for virtual content, among other factors4,5. Here we introduce a holographic AR system that overcomes these challenges using a unique combination of inverse-designed full-colour metasurface gratings, a compact dispersion-compensating waveguide geometry and artificial-intelligence-driven holography algorithms. These elements are co-designed to eliminate the need for bulky collimation optics between the spatial light modulator and the waveguide and to present vibrant, full-colour, 3D AR content in a compact device form factor. To deliver unprecedented visual quality with our prototype, we develop an innovative image formation model that combines a physically accurate waveguide model with learned components that are automatically calibrated using camera feedback. Our unique co-design of a nanophotonic metasurface waveguide and artificial-intelligence-driven holographic algorithms represents a significant advancement in creating visually compelling 3D AR experiences in a compact wearable device. We develop a method for providing high-quality, holographic, three-dimensional augmented-reality images in a small form factor suitable for incorporation in eyeglass-scale wearables, using high-refraction-index glass waveguides with nanoscale metasurfaces, and incorporating artificial intelligence.