Table of Contents

EasyAR headset support in Unity

This document introduces the overall architecture and considerations for EasyAR headset support in Unity.

Before you begin

  • Read EasyAR headset support to learn about the types of headsets supported by EasyAR and the EasyAR features that can run on headsets.

Overview of headset support

EasyAR provides flexible support for headsets in Unity, commonly in two ways:

  • Built-in support: Typically, the EasyAR Sense library directly interfaces with the device SDK and provides corresponding interfaces in Unity (e.g., Apple Vision Pro).
  • Extended support: Interfaces with the device SDK through the Unity headset extension package (e.g., Pico).
block
  columns 4
  block:groupApp:4
    block:groupAppWrapper
      space
      App1["EasyAR + Device A<br>App"]
      space
      App2["EasyAR<br>App"]
      space
      App3["EasyAR + Device B<br>App"]
    end
  end
  
  block:groupSensePluginExtension
    columns 1
    SensePluginExtension["EasyAR Sense Unity Plugin<br>Extension for Device A"]
    space
  end
  block:groupSensePlugin
    columns 1
    SensePlugin["EasyAR Sense Unity Plugin"]
    space
  end
  block:groupXRI
    columns 1
    XRI["XR Interaction Toolkit"]
    space
  end
  block:groupARF
    columns 1
    ARF["AR Foundation"]
    space
  end
  
  block:groupDeviceAUnity
    columns 1
    DeviceAUnity["Device A<br>Unity SDK"]
    space
  end
  block:groupSense
    columns 1
    Sense["EasyAR Sense"]
    block:groupSenseWrapper
      MDeviceB["Device B<br>CameraDevice"]
      Others["..."]
    end
  end
  block:groupXRSubsystem:2
    columns 1
    XRSubsystem["XR Subsystems"]
    XRSDK["Unity XR SDK"]
  end

  block:groupSystem:4
    columns 1
    System["Native Library"]
    block:groupSystemWrapper
      space
      DeviceA["Device A<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Library&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"]
      space
      space
      DeviceB["Device B<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Library&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"]
      space
    end
  end
  
  SensePluginExtension --> App1
  SensePlugin --> App1
  SensePlugin --> App2
  SensePlugin --> App3
  ARF --> App3
  XRI --> App1
  XRI --> App3
  groupSense --> SensePlugin
  groupDeviceAUnity --> SensePluginExtension
  SensePlugin --> SensePluginExtension
  DeviceA --> groupDeviceAUnity
  DeviceA --> XRSDK
  XRSubsystem --> ARF
  XRSubsystem --> XRI
  DeviceB --> MDeviceB
  DeviceB --> XRSDK
  
  style groupApp fill:none,stroke:none,stroke-width:0px
  style groupAppWrapper fill:none,stroke:none,stroke-width:0px
  style groupSensePlugin fill:none,stroke:none,stroke-width:0px
  style groupARF fill:none,stroke:none,stroke-width:0px
  style groupXRI fill:none,stroke:none,stroke-width:0px
  style DeviceAUnity fill:none,stroke:none,stroke-width:0px,color:#fff
  style Sense fill:none,stroke:none,stroke-width:0px,color:#fff
  style groupSenseWrapper fill:none,stroke:none,stroke-width:0px
  style XRSubsystem fill:none,stroke:none,stroke-width:0px,color
  style System fill:none,stroke:none,stroke-width:0px
  style groupSystemWrapper fill:none,stroke:none,stroke-width:0px
  style groupSensePluginExtension fill:none,stroke:none,stroke-width:0px  

  classDef EasyAR fill:#6e6ce6,stroke:#333,color:#fff
  class groupSense EasyAR
  class SensePlugin EasyAR
  class SensePluginExtension EasyAR

  classDef Device fill:#636,stroke:#333,color:#fff
  class groupDeviceAUnity Device
  class DeviceB Device
  class DeviceA Device

In the diagram:

  • Device A belongs to extended support.

    In practice, Device A usually has a corresponding Unity SDK for interfacing with Unity XR SDK or independently implementing headset rendering capabilities.

    The headset extension package for Device A is responsible for connecting the EasyAR Sense Unity Plugin with the Unity SDK of Device A, enabling EasyAR to run on Device A. This support package may be provided by EasyAR or the device manufacturer.

  • Device B belongs to built-in support.

    In practice, Device B may or may not have a corresponding Unity SDK, depending on the implementation by the device manufacturer. For example, Apple Vision Pro does not have a corresponding Unity SDK, while XREAL does.

Both built-in and extended support headsets support the use of custom cameras.

Important

When using trial products (such as personal edition license, trial XR license, or trial Mega service, etc.) on custom cameras or headsets, EasyAR Sense will stop responding after 100 seconds (Mega users can adjust the duration after approval by EasyAR business) upon each startup. There is no such limitation when using the paid version of EasyAR Sense and the paid EasyAR Mega service.

In Unity, the rendering, projection matrix, and transforms of virtual cameras are not controlled by EasyAR; they are typically managed by the device SDK or Unity XR SDK. Device-specific features, such as gesture recognition and eye tracking, are still provided by the device and its SDK. When using them, you usually need to use both EasyAR and the device SDK.

Next steps