What is VR, AR & XR? A guide to virtual, augmented and extended realities

VR, short for virtual reality, is the term heard most often in popular culture, but VR describes just part of a range of experiences known as the virtuality continuum. Picture the continuum as a line spanning the fully physical world on one end, and the fully virtual on the other. VR exists near the fully virtual end, while AR can be defined by its position along the spectrum. Here is a definition of each:

• Virtual Reality (VR) is an experience where users are fully immersed in a virtual world, unable to see the surrounding physical space.
• Augmented Reality (AR) is the experience nearest to the physical end of the virtuality continuum. In AR, users remain fully aware of the physical space but with digital objects or effects overlaying their view.
• Extended Reality (XR) is the umbrella term that encompasses all of these technologies. Therefore, XR may refer to an experience anywhere along the continuum or multiple experiences collectively.

All forms of XR have the similar purpose of producing interactive digital content, but they differ in their levels of virtuality and modes of interaction. Therefore, each form relies on distinct techniques to influence the user's perception and provide control in various ways. To understand the unique uses of VR and AR, you must know about their underlying technologies and terms.

How does VR work?

The virtual experience is completely immersive. To that end, VR uses a variety of devices and techniques to block out the real world and make the visuals, audio and controls feel as natural as possible. Esto incluye:

• Headset: A head-mounted device (HMD), more commonly known as a headset, is the crucial hardware in any wearable VR system. A headset covers the users' eyes with lenses viewing internal displays, while built-in speakers or headphones deliver the sound. Modern headsets typically come in one of three types. Mobile headsets work by converting a smartphone into an HMD. Tethered/connected headsets run off a computer with a wired connection. Standalone headsets are all-in-one, requiring no connection to another device.
• Stereoscopic imaging: Also called stereoscopy, this is the technique for creating 3D visuals through binocular vision. A VR headset contains two lenses, one for each eye, which present with slight lateral displacement from one another. Just like in real life, our eyes combine the offset images to create the perception of depth.
• Field of view (FoV): A major challenge in VR development has been producing a lifelike field of view in HMDs. Currently, only the very best headsets can approach the natural human FoV of 200 - 220 degrees, but a lower FoV of 100 - 180° is still enough to feel immersive.
• Frame rate (FPS) and refresh rate: These aspects of video quality are particularly important for making VR believable to our brains. Anything less than 90 FPS and a comparable refresh rate will likely induce feelings of motion sickness, so VR systems require powerful graphics processing to exceed the demand.
• Latency: Another important factor for reducing cybersickness, or the feeling of motion sickness during VR, is latency. This is the measurement of response time between motion tracking and the corresponding movement rendered on-screen. Low latency (quick response) is crucial for a smooth VR experience, but it requires very sensitive tracking, high processing power and a fast internet connection.
• Degrees of freedom (DoF): VR headsets enable either three or six degrees of freedom. 3DoF can track rotational movement (looking around from a fixed location) but 6DoF is required for tracking directional movement through 3D space.
• Motion tracking sensors: VR headsets rely on an array of sensors to track the movement of the head, controllers and even the eyes in some cases. Capabilities vary based on the device, but the best modern headsets can accomplish 6DoF with practically zero lag in the response rate.
• Spatial audio: The earliest HMDs used stereo speakers to differentiate sounds in the left and right ears. This technology has advanced significantly, and now VR headsets use immersive audio — also known as 3D audio or spatial audio — which tricks the brain into perceiving sound from any direction, including above or below. Spatial audio not only has immersive audio for both ears, but it can emulate sound that has depth – essentially showing the distance a sound is coming from.
• VR controllers: Most VR headsets come with handheld or hand-worn controllers, which can vary in design. The simplest ones use familiar console-style controls with buttons and joysticks. More advanced versions use motion tracking, touch sensors and haptic feedback. The current trend in development is to improve natural hand tracking and haptics for even more responsive and immersive controls.
• Haptic feedback: Haptic feedback is the feeling of touch conveyed by controllers or other devices to the human user. A simple example is the vibration of a smartphone, but VR is driving the next generation of development in haptic headsets, gloves and even full-body suits.

How does AR work?

AR, on the other hand, is instead concerned with combining the virtual and physical experience. The technologies used in these forms of XR are different from those in VR, although there is some crossover. Aquí te mostramos algunas:

• Computer vision: This core technology of AR uses the science of extracting digital information from visual images. The computer must be able to detect interest points, use them to map the 3D structure of a scene and track relevant points through movement in the space.
• Modes of tracking: Delivery of AR/MR content typically relies on one or more of three distinct tracking methods. Marker-based tracking uses a known visual cue like a QR code or brand logo to trigger an action. Markerless tracking uses more advanced feature detection to recognize dimensions and movement, such as on a human face. Location-based tracking uses spatial coordinates, typically GPS, to map a scene.
• Motion sensors: To move naturally through the 3D space, AR/MR devices must be able to track motion with precision. This may be accomplished by optical sensors, accelerometers, gyroscopes, GPS or any combination of the above technologies.
• Smartphones: Modern smartphones contain all the capabilities listed above, along with adequate processing power, making them suitable for AR interaction through the camera and screen. Many familiar apps already use AR, such as Snapchat and Pokémon GO.
• AR glasses: AR glasses or goggles are other devices that can be used for AR. By projecting a miniature display in front of one or both eyes, the lenses can overlay information on the user’s real-world view. Several AR glasses are available to consumers today, but they are not considered AR just by having this see-through display. Apple and other companies do have AR glasses in the works, however, so the technology could soon become mainstream.

Although the technology behind extended reality is already unfolding and reaching new lengths, the idea of immersive technology is nothing new. People were experimenting long before we even had computers and today's achievements in XR tech can be traced back more than 100 years.

The foundations of modern VR-related technology began with the invention of 3D imaging by Charles Wheatstone in 1838. Wheatstone was the first scientist to describe binocular vision and he created a device called the stereoscope combining two photographs into a 3D illusion. Stereoscopic imaging remains a key component of VR and many subsequent advancements have led to the virtual experiences we have today.

• 1957: The Sensorama was invented by Martin Heilig. This was an arcade-style booth that used stereoscopic screens, stereo speakers, fans, scent generators and a vibrating chair to immerse the viewer in any of six specially produced short films. Though the term virtual reality was not yet coined, Heilig’s Sensorama is today often credited as the first true VR/XR experience.
• 1960: Heilig also invented the first HMD, which he called the Telesphere Mask. It featured a wide-view stereoscopic display and stereo sound.
• 1975: Computer artist Myron Krueger created the world’s first interactive VR experience, then known as artificial reality. Using cameras, computers and projectors, Myron’s project allowed users’ body movements to control an onscreen figure and interact with the figures of other users who stood in separate rooms.
• 1985: Technology for HMDs improved throughout the 1970s and inventors came up with finger-tracking gloves in the early 1980s. Beginning in 1985, however, the company VPL Research made improvements and became the first to sell VR goggles and gloves. VPL’s founders were also the first to popularize the name “virtual reality.”
• The 1990s: This decade saw rapid advancement of virtual reality technology. Notable developments included a system by NASA for driving the Mars rover and video games by SEGA and Virtuality Group, as well as PTSD therapies for Vietnam veterans by university researchers.
• The 2000s: This decade saw a comparative lull in enthusiasm for VR, as the tech world focused on mobile and smartphone technology. One milestone, however, was the launch of Google Street View in 2007, which used 360-degree cameras and later 3D imagery to show immersive depictions of real-world locations.
• 2010: A young inventor named Palmer Luckey created a prototype headset that would soon become the Oculus Rift. This device made major improvements over HMDs available at the time and its success effectively revived popular interest in VR technology.
• 2014: Facebook bought Luckey’s Oculus VR company for more than $2 billion.
• 2014: Google releases Cardboard, the first of many VR headsets the company releases. Unlike other sleek versions of VR, this headset looks like it’s made of actual cardboard, hence the name.
• 2015: Snapchat launched Lenses, commonly called face filters, which served as an introduction to AR technology for millions of users.
• 2015: Samsung releases Gear VR, the first ever VR set created by the company. For this edition, Samsung partnered with Oculus to make it available to consumers everywhere.
• 2016: Google releases Daydream, a more enhanced and sleeker version of their first VR headset, Cardboard.
• 2016: Microsoft released its Windows Mixed Reality platform and associated headset, the HoloLens.
• 2016: Sony released the PlayStation VR headset and associated VR games for the PS4.
• 2016: HTC and Valve partner up to create their virtual headset, Vive.
• 2018: Following a wave of other companies releasing their own VR products, Facebook released the Oculus Go and later the Oculus Quest, two standalone headsets that led a design push away from tethered devices.
• 2020: Facebook introduced controller-free hand tracking to the Quest and released the updated Oculus Quest 2.
• 2021: Facebook officially rebranded to Meta and the company announced plans to create a virtual world called the Metaverse, which would rely on VR for immersive experiences.

These developments over nearly two centuries have led to the XR of today, which is evolving faster now than ever before. Though video games and mobile apps have been major drivers of the technology so far, VR-related technology is quickly expanding in many other industries as well.

Most people know VR when it comes to gaming or video streaming; these are the main uses for which popular headsets like the Meta Quest and Sony PS VR are marketed. Similarly, AR is best known for mobile apps like Snapchat and Pokemon Go. However, other uses for these technologies are widespread and continually growing.

XR tech for work, education, healthcare and art are all part of an overall a trend toward interconnectivity and automation in the modern economy. This momentum was undoubtedly accelerated by pandemic lockdowns and remote work, but as you can tell from the history of VR, this movement has been in the making for decades. To better understand the current state of VR-related technology and where it's headed in the future, it's helpful to know about some of the many use cases that exist today.

Examples of XR in education

The technology is quickly being adopted for education, with many AR/VR apps already available for use in the classroom. Examples include a VR game that lets you build cells of the body and watch how they function and AR apps that place historical figures and events in the room with students in 3D. Today, most apps for education use AR via smartphones and tablets, but in the future, more classrooms may have VR devices like smart goggles and headsets.

These virtual experiences promise a learning environment that’s more interactive and engaging for today’s digital native generation. In addition, XR-based lessons and virtual field trips can offer a few distinct advantages over traditional education methods:

• Lower cost than traveling or buying specialized equipment for hands-on learning;
• Improved safety and risk management;
• Encouragement of active learning and multiple learning styles;
• Immersive, fun, and memorable lessons.

Examples of XR for health and safety

Some of XR’s most compelling uses may be in the field of medicine, and in fact, virtuality in healthcare is nothing new. VR treatments for PTSD began in the 1990s and today more doctors are utilizing VR for trauma-focused therapy, allowing veterans to recall and cope with their experiences in a safe environment. To help even further, Verizon partnered with the Veterans Health Administration (VHA) in the initiative “Project Convergence” to provide Medivis’ 3D technology and other VR technology to offset some of the expenses and struggles veterans face when seeking healthcare.

Another rapidly growing use of the technology is in surgical operations. AR and VR have proved useful for training with simulated surgeries, but some hospitals are even performing augmented surgery on live patients, using AR overlays as a kind of x-ray vision for doctors. There are also collaborative surgeries using AR/MR, where surgeons can stand together in a room as avatars and view holographic displays over the patient.

These are just a few examples of XR tech in healthcare; many other cases exist today that may only be a preview of what will soon be possible, such as:

• XR can help patients communicate their symptoms and help doctors describe treatments with interactive visual aids.
• Continued development of XR systems, along with enhanced performance through 5G networks, will enable improved training and collaboration among medical teams around the world.
• AR apps could aid in safety and emergency response by guiding users to locations of defibrillators, fire extinguishers, emergency exits, blue light phones or public safe spaces.
• First responders can train with VR simulations to prepare them for dangerous real-life scenarios.

Examples of XR in business

Many industries are embracing XR technology for collaboration among remote teams. The ability to visualize people, products or equipment virtually can be a game-changer for collaborative design and workflow. To meet this demand, companies like Microsoft are developing advanced AR headsets and enterprise software, while Meta is advertising VR workrooms in the metaverse.

The retail industry has also been a major driver of these technologies, particularly AR. The global market for AR in retail was estimated at nearly $2.5 billion in 2021 and is expected to grow to more than $12 billion by 2027. For both online and brick-and-mortar stores, AR has the potential to improve the customer experience while reducing expenses. Here are some examples of how:

• “Trying on” outfits and accessories virtually, whether at home or with in-store displays;
• Taking measurements for fitting through online stores;
• Visualizing how products will look in place, with options to try different styles or sizes before buying;
• Picking and matching colors for interior design or home renovations;
• Aiding real-world navigation through cities and shopping spaces with AR cloud information;
• Expanded opportunities for advertising and content marketing in the virtual space;
• Increasing brand engagement through social sharing and customer loyalty through informational or entertainment value.

Examples of XR in art and entertainment

In addition to video games and social media, VR and AR enjoy widespread popularity for entertainment, art, activism and more. Digital artwork, virtual travel and metaverse creations promise endless possibilities. Por ejemplo:

• One of the most popular uses of VR headsets is watching shows and movies, with platforms like Youtube and Bigscreen offering dedicated VR content. Most headsets also offer their own apps that let you watch Netflix or other 2D streaming in a virtual space like a living room or movie theater.
• XR technologies offer unlimited potential for artists and designers and the growing market for NFTs provides an exciting opportunity for creators to profit from their work. Apps like Tilt Brush and Masterpiece Studio enable you to create your artwork in immersive 3D.
• The app, AR Pro Interactive, is available in both Google and Apple application stores and provides an interactive look at how professional sports players perform so well. It also guides you through your own performance in certain cases.
• Virtual tourism is destined to expand greatly as more people get access to XR tech. With any VR device, you can already take virtual trips to U.S. national parks or cities around the world. With the introduction of the metaverse, there may soon be an explosion of additional destinations to explore in VR.
• AR is being used extensively to enhance real-world travel experiences as well. Many AR mobile apps can help with navigation and translation while on the go. In addition, artists can use AR to place virtual installations in real-world locations.

Artists are also embracing the technology to spark action on climate change and social justice issues. Activist creators use VR artwork and AR apps to help visualize nature in urban environments or to demonstrate changes to the planet that may happen in the future.

XR products already enjoy a broad and booming market that exceeded $20 billion globally in 2021 and is projected to reach $453.5 billion by 2030, according to one study by Report Ocean. No longer just an experimental field, VR and AR have proven their value for increasing connectivity, productivity and safety in several industries. Now, with hybrid capabilities also being realized, XR tech is firmly a part of the so-called Fourth Industrial Revolution, which describes the dramatic societal changes expected with the exponential development of advanced technologies.

Another part of this revolution is the expansion of 5G, which is delivering faster and more reliable internet to people around the world. With the simultaneous improvement of mobile hardware, AR can be used more readily with the Internet of Things, or the network of smart devices that is growing every day.

Then there is the metaverse, which has been promised by the company Meta but will include versions created by other organizations as well. These virtual worlds are being designed for VR, with the expectation that more and more people will embrace the technology in the coming years.

VR and AR will all become more widespread as the technologies improve. Devices will get smaller, lighter, more stylish and more affordable as the software gets more capable. These trends, along with 5G, the metaverse and the general pace of increasing connectivity in the modern world will make XR technology more familiar in our everyday lives - for social interaction, for work and for just having fun.