EasyAR's Headset and Glasses Support
The EasyAR SDK offers powerful cross - platform AR capabilities, and its design concept is also applicable to emerging spatial computing devices - headsets. This article will introduce how EasyAR supports headset devices and how developers can utilize these features to build immersive experiences.
Terminology Explanation
In this document, "headset" or "Headset" specifically refers to a type of computing device that can be worn on the head and supports immersive or see - through interaction. These devices can present virtual content in front of the user's eyes, enabling augmented reality (AR) or mixed reality (MR) experiences. This includes:
- Optical See - Through, OST Headset: Allows users to directly view the real world through semi - transparent lenses.
- Video See - Through, VST Headset: Captures the real world via a camera and displays it as a video stream.
Conventionally, headsets are also often referred to as AR glasses, MR glasses, or VR helmets. The specific names may vary depending on the manufacturer and device type, but in this article, we use "headset" to refer to this type of device uniformly.
Basic Working Principle of Headsets
To better understand the principle of EasyAR's support for headsets, we first need to understand the basic working process of headset devices:
- Environmental Sensing: Through built - in multiple cameras, depth sensors (such as iToF), and inertial measurement units (IMUs), it can sense the geometric structure, lighting conditions, and object surfaces of the surrounding environment in real - time.
- Spatial Computing: Based on sensor data, the SLAM system tracks the user's 6DoF head pose (position + orientation) in real - time.
- Content Rendering and Display: Renders 3D content (such as models and special effects) according to the device's pose and projects the rendering results onto the display screen. In VR mode, only pure virtual images are displayed; in AR/MR mode, virtual images are combined with the real environment (VST camera image or OST see - through background).
- Interaction System: Receives user instructions and responds through controllers, gesture recognition, voice, or eye tracking.
Principle of EasyAR's Support for Headsets
EasyAR does not replace the headset's native spatial tracking or rendering pipeline but works in collaboration with it as a "spatial computing enhancer". As a professional AR algorithm engine, it provides spatial perception and computing capabilities for various AR scenarios and collaborates efficiently with the device's original system.
| Scope of Responsibility | Role Division |
|---|---|
| 6DOF head tracking, display rendering, basic interaction, etc. | Headset's native SDK/runtime |
| Advanced perception capabilities such as image/object recognition and tracking, large - scale spatial positioning | EasyAR SDK |
The EasyAR SDK provides core AR functions for world perception, such as image/object recognition, sparse reconstruction, dense reconstruction, and large - scale spatial positioning. It is responsible for "understanding" the world and telling the headset's application where to place the virtual content.
The EasyAR SDK is integrated into the headset's application development framework (usually Unity or Unreal) as a plugin or library. It receives the original data stream from the device system, processes and calculates it, and then outputs a pose matrix relative to the device's spatial coordinate system. Finally, the headset engine's rendering pipeline draws the virtual objects in the correct position.
Support Status and Implementation Methods
EasyAR provides comprehensive support for mainstream headset development platforms, mainly implemented in the following two ways:
- Through Unity/Unreal Engine: This is the most mainstream and recommended way. Headset manufacturers usually provide dedicated Unity/Unreal plugins or XR SDKs. EasyAR can be seamlessly integrated into the manufacturers' SDKs.
- Through Native Platform: For scenarios that require extreme performance or specific native development, you can use EasyAR's native interfaces in C++, Java, or Objective - C. This usually requires developers to handle the interface docking with the device's underlying data by themselves.
EasyAR has been tested and verified on multiple mainstream headset platforms through Unity. The currently confirmed supported devices are as follows:
| Headset Device Model | System/SDK Version Requirement |
|---|---|
| Apple Vision Pro | visionOS 2 or later |
| PICO 4 Ultra1 | PICO Unity Integration SDK 3.4.0 or later |
| PICO Space Pro1 | PICO Unity Integration SDK 3.4.0 or later |
| Rokid Max Pro （Using Rokid AR Studio or other official supported usage methods of Rokid） |
Rokid Unity OpenXR Plugin 3.0.3 or later |
| XREAL Air2 Ultra | XREAL SDK 3.1 or later |
| Xrany X1 | Xrany Yuanni SDK |
Note
Rokid Max Pro can support Rokid UXR 3 through the Rokid Unity OpenXR Plugin, but it is recommended to use the XR Interaction Toolkit, especially for cross - device usage.
Important
Both Apple Vision Pro and XREAL require corresponding enterprise authorization to use. If you have any questions, please contact the business department.
- Due to the interface authorization restrictions of Apple Vision Pro, only devices with Apple enterprise API licenses are supported.
- Due to the interface authorization restrictions of XREAL, only devices with enterprise authorization are supported.
For headset devices of other manufacturers not mentioned above, EasyAR provides an extended access method such as custom cameras. You can refer to Create an EasyAR Headset Extension Package for access, and you can complete the docking by yourself.
This usually involves the following steps:
- Obtain device development permissions: Apply for a developer account and SDK documentation for the target headset.
- Obtain sensor data streams: Get necessary data such as camera images (video frames) and camera parameters from the device SDK.
- Call EasyAR API: Use EasyAR's underlying API to send the obtained sensor data to the EasyAR
FrameSourcefor processing. - Obtain and apply the calculation results: Get the calculation results (camera pose) from the EasyAR engine and apply them to your 3D rendering engine.
We provide detailed development guides and sample code to help you complete this process. If you encounter problems during the docking process, you are welcome to seek technical support in our developer community.
Available Core Functions
On headset devices, you can fully utilize EasyAR's full - feature matrix to build rich spatial applications:
- Planar Image Tracking: Recognize and track preset pictures, and overlay dynamic videos or 3D models on the pictures.
- 3D Object Tracking: Recognize and track preset 3D models (such as toys and product boxes), and enable virtual content to interact with them.
- Sparse Spatial Map: Scan the surrounding environment to generate a 3D visual map and provide visual positioning and tracking functions. The generated map can be saved or shared in real - time among multiple devices.
- Dense Spatial Map: Scan and generate a dense point cloud map and mesh model (Mesh) of the surrounding environment to realize the physical occlusion relationship between virtual and real objects, greatly enhancing the sense of immersion.
- Cloud Image Recognition: Connect to the EasyAR cloud database to realize the recognition and management of a large number of pictures, suitable for scenarios such as exhibitions and education.
- Mega Large - Scale Spatial Positioning: A city - level spatial computing solution that connects to the EasyAR cloud positioning service to achieve stable, fast, and accurate positioning and tracking, greatly breaking through and expanding the scope of AR experiences.
Platform - Specific Guides
To help you quickly get started with specific platforms, we have prepared detailed multi - platform integration guides. Please click on the tabs below to view the quick - start tutorials for the corresponding platforms.
- Introduction
- Using Headset Samples
- Enabling Headset Support
- Project Configuration
- Create an EasyAR Headset Extension Package