When it comes to 5G NR, it is quite common to see smart glasses as one of the primary devices that need the power of 5G. But how? Watching all those pre-rendered, futuristic, conceptual clips will not help users understand actual usage any better. This article looks at the different perspectives that 5G NR needs to solidify smart glass use cases.
Two major consumption modes for smart glasses are virtual reality (VR) and augmented reality (AR). Both provide end users with computer-generated graphics. The significant difference between the two is that VR requires a graphic rendering that covers the vision completely. The human interaction with the scene will also occur inside the virtual world. People used to call these VR devices head-mounted displays (HMDs) because they are so bulky that they mount on a user's head, but newer versions with lighter-weight are also emerging in the market. These new versions are the same size as typical smart glasses. AR, on the other hand, renders computer graphics over the actual objects that end users can still see in the real world through the translucent or transparent lens, and the user interaction will be mainly occur with the natural world objects.
5G NR for VR
Figure 1: 1080p VR streaming to the user’s vision.For a taste of VR, users can buy a Google Cardboard or equivalent at a low cost, then put it in his or her phone, and start VR videos on YouTube. It gives the basics of VR, but it does not take advantage of 5G for another level of experience. The YouTube VR resolution might not be high enough, although it is already in 1080 p. The fact is that it features horizontally 1920 pixels spreading all around the user. With approximately 120° of human vision, people can have only 640 pixels for the full width of the eyesight, which feels like watching a DVD in 2020. As such, 4K resolution will improve the experience, but it is still far from being immersive, not to mention stereoscopic view will further cut the resolution half horizontally or vertically. However, this delivery model has a definite advantage in responsiveness because it has all the pixels needed in each frame. Also, it is more cost-effective and requires less planning in production.
An advanced delivery model will be focusing on the end user’s field of view. High-resolution video or graphics is delivered only to the end-users field of view. A much lower resolution version of the same frame is given for the rest of the scene outside of the user’s vision. This model maintains the high definition (HD) of graphics to the user’s eye, however inducing a new concern about responsiveness. When the users move their head, the HD field of view needs to catch up with the viewing angle change. The motion to photon latency should be in 20 ms or less to avoid VR sickness, so this is where 5G NR comes into play. With the low latency of 5G, HD videos can always catch up with the head movement, thus continuously delivering HD views to the end-user.
Figure 2: High definition video to user’s vision only in VR.
5G NR for AR
So, how about AR? Many AR use cases are mostly on mobile phones or tablets. There are also some AR smart glasses on the market, but they are not ready for the mass market yet. There will likely be more AR smart glasses equipped with light detection and ranging (lidar) technology in the market starting in 2021, enhancing the precision of the rendered objects for better “existence” in the real space. However, limitations like heat, cost, processing power, battery life and physical size are still there.
The ultimate AR experience is that the virtual graphics are too real and blend into reality flawlessly. But it is too far from the current generation of technology unless the cloud renders the whole thing remotely. When high-end graphics are streamed instead of generated locally, it solves all problems mentioned above. Think Stadia or GeForce Now, but with lidar and display on smart glasses. Such a level of detail requires high mobile bandwidth, 50 Mbps or above, and round trip latency below 20 ms. 5G NR is the first generation of the mobile networks that can bring an entirely new experience.
More possibilities from smart glasses with 5G
VR use cases might be more on the entertainment side, for instance gaming/esports and watching videos. There are also additional possibilities like learning, telepresence or “virtual desktops.” This does not refer to just any virtual desktops (e.g., Citrix), but also to web browsers or video editors floating inside the virtual space of VR vision. There will be no more concept of “display panel,” or, depending on customer preference, as many as they wish, surrounding the user in the virtual world. Likewise, 8K videos can be edited on the cab because they are processed on the cloud, and the display is delivered through 5G NR.
AR wise, one major use case will be providing supplementary information to daily life. Whether it is checking in-store specials or reading instructions to assemble a new piece of furniture, all the advice from machines will become more intuitive through the smart glasses when the required information overlays in the right place at the right time.
Conclusion
5G NR is here and coming to more cities around the world in 2021. As such, the wait for the first 5G smart glasses has begun. Some say Apple will release their first smart glasses product in a couple of years and a tight software/hardware integration that can change the game. Experts are also waiting for a lower cost 5G and real 5G content because having the current generation of content at the speed of 5G will not make it a new experience.
Smart glasses for the commercial world will be discussed in a future article.
