Table of Contents

EasyAR headset support in Unity

This document introduces the overall architecture and considerations for EasyAR headset support in Unity.

Before you begin

  • Read EasyAR headset support to understand the types of headsets supported by EasyAR and the EasyAR features that can run on headsets.

Headset support overview

EasyAR supports headsets in Unity flexibly, commonly in two ways:

  • Built-in support: Typically directly interfaces with device SDKs in the EasyAR Sense library and provides corresponding interfaces in Unity (e.g., Apple Vision Pro)
  • Extended support: Interfaces with device SDKs through Unity headset extension packages (e.g., Pico)
block
  columns 4
  block:groupApp:4
    block:groupAppWrapper
      space
      App1["EasyAR + Device A<br>App"]
      space
      App2["EasyAR<br>App"]
      space
      App3["EasyAR + Device B<br>App"]
    end
  end
  
  block:groupSensePluginExtension
    columns 1
    SensePluginExtension["EasyAR Sense Unity Plugin<br>Extension for Device A"]
    space
  end
  block:groupSensePlugin
    columns 1
    SensePlugin["EasyAR Sense Unity Plugin"]
    space
  end
  block:groupXRI
    columns 1
    XRI["XR Interaction Toolkit"]
    space
  end
  block:groupARF
    columns 1
    ARF["AR Foundation"]
    space
  end
  
  block:groupDeviceAUnity
    columns 1
    DeviceAUnity["Device A<br>Unity SDK"]
    space
  end
  block:groupSense
    columns 1
    Sense["EasyAR Sense"]
    block:groupSenseWrapper
      MDeviceB["Device B<br>CameraDevice"]
      Others["..."]
    end
  end
  block:groupXRSubsystem:2
    columns 1
    XRSubsystem["XR Subsystems"]
    XRSDK["Unity XR SDK"]
  end

  block:groupSystem:4
    columns 1
    System["Native Library"]
    block:groupSystemWrapper
      space
      DeviceA["Device A<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Library&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"]
      space
      space
      DeviceB["Device B<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Library&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"]
      space
    end
  end
  
  SensePluginExtension --> App1
  SensePlugin --> App1
  SensePlugin --> App2
  SensePlugin --> App3
  ARF --> App3
  XRI --> App1
  XRI --> App3
  groupSense --> SensePlugin
  groupDeviceAUnity --> SensePluginExtension
  SensePlugin --> SensePluginExtension
  DeviceA --> groupDeviceAUnity
  DeviceA --> XRSDK
  XRSubsystem --> ARF
  XRSubsystem --> XRI
  DeviceB --> MDeviceB
  DeviceB --> XRSDK
  
  style groupApp fill:none,stroke:none,stroke-width:0px
  style groupAppWrapper fill:none,stroke:none,stroke-width:0px
  style groupSensePlugin fill:none,stroke:none,stroke-width:0px
  style groupARF fill:none,stroke:none,stroke-width:0px
  style groupXRI fill:none,stroke:none,stroke-width:0px
  style DeviceAUnity fill:none,stroke:none,stroke-width:0px,color:#fff
  style Sense fill:none,stroke:none,stroke-width:0px,color:#fff
  style groupSenseWrapper fill:none,stroke:none,stroke-width:0px
  style XRSubsystem fill:none,stroke:none,stroke-width:0px,color
  style System fill:none,stroke:none,stroke-width:0px
  style groupSystemWrapper fill:none,stroke:none,stroke-width:0px
  style groupSensePluginExtension fill:none,stroke:none,stroke-width:0px  

  classDef EasyAR fill:#6e6ce6,stroke:#333,color:#fff
  class groupSense EasyAR
  class SensePlugin EasyAR
  class SensePluginExtension EasyAR

  classDef Device fill:#636,stroke:#333,color:#fff
  class groupDeviceAUnity Device
  class DeviceB Device
  class DeviceA Device

In the diagram:

  • Device A belongs to extended support

    In practice, Device A usually has a corresponding Unity SDK for interfacing with Unity XR SDK or independently implementing headset rendering capabilities.

    The headset extension package for Device A is responsible for connecting the EasyAR Sense Unity Plugin with Device A's Unity SDK, enabling EasyAR to run on Device A. This support package may be provided by EasyAR or the device manufacturer.

  • Device B belongs to built-in support

    In practice, Device B may or may not have a corresponding Unity SDK, depending on the device manufacturer's implementation. For example, Apple Vision Pro has no corresponding Unity SDK, while XREAL has a corresponding Unity SDK.

Both built-in and extended support headsets support using custom cameras.

Important

When using trial products (such as personal edition license, trial XR license, or trial Mega service) on custom cameras or headsets, EasyAR Sense will stop responding 100 seconds after each startup (the duration can be adjusted via EasyAR Business upon approval for Mega users). Using the paid version of EasyAR Sense and the paid EasyAR Mega service does not have this restriction.

In Unity, rendering, projection matrices, transforms, etc. of virtual cameras are not controlled by EasyAR. They are typically controlled by the device SDK or Unity XR SDK. Device-specific features such as gesture recognition and eye tracking are still provided by the device and its SDK. During use, both EasyAR and the device SDK are typically required simultaneously.

Next steps