What is the AR cloud and what is it used for?

In the simplest terms, the AR cloud is a digital copy of the real world. Developers can build augmented reality experiences using this framework instead of starting from scratch.

The AR cloud will make it possible to localize experiences. For example, if you are walking down a street, you will be able to use your smartphone camera to view your surroundings. An AR app, such as Google Lens, will use digital overlays to help you search for specific sites, see details about surrounding businesses and get crowdsourced information, such as restaurant reviews.

The AR cloud is growing out of previous technologies, including geopositioning, image recognition, and machine learning. It is also being aided by faster internet and mobile data connections, including 5G.

The AR cloud is a unique system, even compared to other cloud-based technologies.

Cloud technologies help to create virtualized platforms for work, games, and information. Like other cloud-based systems, the AR cloud will let users access information without having to first download it to a computer or device.

However, there are some significant differences between the AR frameworks and other cloud-based technologies.

• The AR cloud brings virtual information to the physical world. Other cloud-based technologies focus solely on creating virtual platforms for work, study, research, or entertainment. For example, a traditional cloud-based platform could provide a central place where users can upload reviews of local coffee shops. Readers would have to go to the cloud-based site to see the reviews. However, an app on the AR cloud would recognize a specific coffee shop and overlay reviews without requiring you to visit the site.
• The AR cloud works with specific hardware. All cloud-based technologies are accessible from desktops, laptops, and mobile devices.
• The AR cloud will also operate with specialized wearable devices, such as smart glasses or headsets. That said, much of early AR development is for mobile devices, so users will likely also be able to rely on more traditional hardware.

Despite these differences, the accessibility and scalability of cloud-based systems are the same for both AR and more-traditional cloud platforms.

Early examples of AR include live photo filters on Snapchat and Instagram and augmented reality games, like Pokemon GO. These games follow the same concept as more-complex cloud-based AR: they place an abstraction layer over actual real-world images.

The AR cloud will allow this kind of digital overlay on a massive scale by offering a cloud-based medium for AR content. The cloud enables AR applications to take advantage of much higher computational power than you get with a single device running an AR app. A smartphone or head-mounted display limits an AR app's processing power to the device's graphics processing unit (GPU) and central processing unit (CPU). Comparatively, AR apps in the cloud can leverage an entire cluster of CPUs and GPUs. Not only does this improve performance and the quality of graphics, it means you can do more with AR.

For example, a restaurant in a specific location could post its menu on the cloud, allowing AR users to view it automatically when they are in a specific location or when the camera and image recognition software on their devices recognize the restaurant building.

Also, rather than a single filter, animated character or single piece of data, the AR cloud could allow massive amounts of information that users could call up based on their settings and manual or voice controls.

AR cloud computing is still in its infancy. However, 5G connectivity, edge computing, and advances in geolocation, image recognition, and processing speeds have made future advances more realistic. Here is a closer look at how these technologies will help AR cloud development.

• 5G networks will allow the rapid transfer of data to handle geolocation, image recognition processing, and sending information to augmented reality layers. The speed will allow access to details in real-time with virtually no latency (also known as lag), which is necessary for AR systems to be useful. Slow-loading information, sluggish image recognition processes, and lagging geolocation all mean the AR experience won’t correspond with the real-world setting.
• Edge computing is another essential element of AR. It involves utilizing resources close to the user to decrease latency. For example, an edge system may use existing Wi-Fi networks or nearby servers to speed up the interaction between devices and cloud-based AR systems.
• Remote rendering of 3D objects or entire apps is perhaps the most important capability of the AR cloud. With remote rendering (also known as pixel streaming or split rendering) solutions such as Interactive Streaming for Augmented Reality (ISAR), an app on the user’s device sends sensor data, such as room tracking and gesture input, to a server in the cloud. There, the AR app renders graphics-intensive 3D content and sends it back to the user’s device with very low latency, allowing the user to interact with complex 3D objects in real time.
• Geolocation is very important for AR cloud development. GPS is accurate for navigation, but it could be off by up to 20 feet. That is not precise enough for hyper-localized interactions. With exact 3D models, AR cloud-connected devices can pinpoint locations more accurately than GPS.
• Image recognition tools are also necessary. With enough data, an AR program may be able to recognize an object and then search for further details about it. For example, perhaps an AR system could recognize a book on a table. Once it defines the object, it can look for more data to find the title of the book, allowing the user to call up reviews, summaries and other information if they wish.

These technologies can help the AR cloud bring more benefits to users.

• The AR cloud will save time because you will not have to manually search for information with an app or browser.
• The AR cloud can also come with privacy controls and better security than stand-alone apps.
• Businesses and organizations will be able to reach customers in new ways and provide information that can help improve their experiences.
• The precise geolocation and information overlays can make navigation much easier, even if you are in an unfamiliar place.
• The AR cloud will make the medium more accessible to everyone. You won't need specialized knowledge and funding to create an augmented reality app.

What’s more, there are many useful applications for cloud-based AR.

The applications of AR cloud technology could go well beyond entertainment and advertising. 

• Retail. AR programs can help you find information, such as price, reviews, and measurements, for products in a retail store. Some companies, such as IKEA, are already developing this type of AR tool.
• Navigation. In addition to helping tourists find their way around a new city, AR cloud tools can also overlay direction and location information for rescue workers, the military, law enforcement, or first responders who need information quickly in high-stress situations.
• Job training. AR can help people learn new processes when they start a career. The digital overlay can provide directions for unfamiliar tasks or warn of possible missteps.
• Security. AR cloud platforms will likely have security and safety requirements that all developers will have to meet. This can enhance account security and ensure privacy. The Internet of Things (IoT) can also support integrating AR systems with existing cameras and other remote monitoring devices connected to the IoT.
• Accessible safety information. Companies will undoubtedly use the AR cloud for marketing, but they can also use it to provide safety information. The government or regulators can likewise post information about consumer safety, recalls, or dangers of specific products or places on the AR cloud.
• Education. Teachers, schools, and classrooms may be able to leverage the AR cloud to make assignments and curricula more engaging and accessible to students. Textbooks and lessons could be augmented, and cloud storage used to preserve notes or other custom elements for future reference.

The AR cloud is in the early stages of development, so many of these benefits are not yet a reality.

New technology could increase the speed at which the AR cloud develops. Forecasts suggest that 5G technology could lead to increased use by 2025. By 2030, the AR market could be worth $76 billion, with numerous mainstream and specialized applications in finance, healthcare, travel, and public safety.