Seeing inside virtual reality (VR) headsets for the first time can be quite a shock. Whether the view is a towering mountainside or up in the clouds, the technology behind creating VR vision is quite complex. Termed “foveated rendering,” the process results in boosted resolution for a VR user, where the view mimics sight in real life. As more VR headsets are available to consumers, the evolution of forested rendering may shift to make a virtual experience seem more lifelike.
Background: Developing Foveated Rendering
Before using foveated rendering, early VR headsets had views that mimicked arcade screens, with little depth perception or resolution. Because users were already enamored with the VR technology as a whole, for a while, no one gave the graphics much thought. In 2014, everything changed when foveated rendering was introduced by the VR company FOVE, based out of Tokyo. FOVE’s technology used infrared to track eye movements and pupil dilation, better understanding how users see in a headset. This led to other companies looking into tracking technology to better understand their target demographic.
Since then, foveated rendering has evolved, as the newest VR headsets use dozens of sensors to track eye movement, blinking, and pupil dilation. Some companies are even developing VR motion suits that will sync with VR vision to allow for a smoother experience. This can help make the virtual experience more convincing and enjoyable for users.
Analysis: The Science of VR Vision
Foveated Rendering not only assists a player’s vision but also reduces the cost of pixel shading for developers. This is because foveated rendering works by anchoring in on a “fixation point,” blurring out the rest of the image in the periphery. The “fixation point” is the highest resolution of the image, and is supposed to correspond with the retina of the eye, also called the fovea. As the fovea moves, the eye-tracking technology can shift the VR vision, allowing viewers to change views smoothly. For developers, having a “fixation point” can also make the graphics and design process easier, as they have a visual path that they know the user will follow within the experience.
One of the biggest companies implementing foveated rendering is Sony’s PlayStation. They have used the process to help boost sales for their VR2 controller. Other companies are following suit, seeing the success that foveated rendering has on a user’s VR experience.
Outlook: Foveated Rendering can Help with Cybersickness
Because foveated rendering is supposed to the body’s natural vision process, it can help combat cybersickness. Cybersickness, or virtual reality sickness, happens when a user gets disoriented when wearing a VR headset. Symptoms of cybersickness include nausea, dizziness, and even vomiting. Cybersickness has been correlated to poor graphics and transitions between views, something that foveated rendering can fix. With new VR headsets like Facebook’s and Apple’s to hit the market soon, it will be no surprise if foveated rendering advances to develop more life-like VR experiences.
Kenna Castleberry is a staff writer at the Debrief and the Science Communicator at JILA (a partnership between the University of Colorado Boulder and NIST). She focuses on deep tech, the metaverse, and quantum technology. You can find more of her work at her website: https://kennacastleberry.com/.