Table of Contents

EasyAR headset support in Unity

This document introduces the overall architecture and notes for EasyAR headset support in Unity.

Before you begin

  • Read EasyAR headset support to learn about headset types already supported by EasyAR and EasyAR features that can run on headsets.

Headset support overview

EasyAR supports headsets in Unity in a flexible way. There are two common approaches:

  • Built-in support: Usually connects directly to the device SDK in the EasyAR Sense library and provides corresponding interfaces in Unity, such as Apple Vision Pro
  • Extension support: Connects to the device SDK through a Unity headset extension package, such as Pico
block
  columns 4
  block:groupApp:4
    block:groupAppWrapper
      space
      App1["EasyAR + Device A<br>App"]
      space
      App2["EasyAR<br>App"]
      space
      App3["EasyAR + Device B<br>App"]
    end
  end
  
  block:groupSensePluginExtension
    columns 1
    SensePluginExtension["EasyAR Sense Unity Plugin<br>Extension for Device A"]
    space
  end
  block:groupSensePlugin
    columns 1
    SensePlugin["EasyAR Sense Unity Plugin"]
    space
  end
  block:groupXRI
    columns 1
    XRI["XR Interaction Toolkit"]
    space
  end
  block:groupARF
    columns 1
    ARF["AR Foundation"]
    space
  end
  
  block:groupDeviceAUnity
    columns 1
    DeviceAUnity["Device A<br>Unity SDK"]
    space
  end
  block:groupSense
    columns 1
    Sense["EasyAR Sense"]
    block:groupSenseWrapper
      MDeviceB["Device B<br>CameraDevice"]
      Others["..."]
    end
  end
  block:groupXRSubsystem:2
    columns 1
    XRSubsystem["XR Subsystems"]
    XRSDK["Unity XR SDK"]
  end

  block:groupSystem:4
    columns 1
    System["Native Library"]
    block:groupSystemWrapper
      space
      DeviceA["Device A<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Library&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"]
      space
      space
      DeviceB["Device B<br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;Library&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;"]
      space
    end
  end
  
  SensePluginExtension --> App1
  SensePlugin --> App1
  SensePlugin --> App2
  SensePlugin --> App3
  ARF --> App3
  XRI --> App1
  XRI --> App3
  groupSense --> SensePlugin
  groupDeviceAUnity --> SensePluginExtension
  SensePlugin --> SensePluginExtension
  DeviceA --> groupDeviceAUnity
  DeviceA --> XRSDK
  XRSubsystem --> ARF
  XRSubsystem --> XRI
  DeviceB --> MDeviceB
  DeviceB --> XRSDK
  
  style groupApp fill:none,stroke:none,stroke-width:0px
  style groupAppWrapper fill:none,stroke:none,stroke-width:0px
  style groupSensePlugin fill:none,stroke:none,stroke-width:0px
  style groupARF fill:none,stroke:none,stroke-width:0px
  style groupXRI fill:none,stroke:none,stroke-width:0px
  style DeviceAUnity fill:none,stroke:none,stroke-width:0px,color:#fff
  style Sense fill:none,stroke:none,stroke-width:0px,color:#fff
  style groupSenseWrapper fill:none,stroke:none,stroke-width:0px
  style XRSubsystem fill:none,stroke:none,stroke-width:0px,color
  style System fill:none,stroke:none,stroke-width:0px
  style groupSystemWrapper fill:none,stroke:none,stroke-width:0px
  style groupSensePluginExtension fill:none,stroke:none,stroke-width:0px  

  classDef EasyAR fill:#6e6ce6,stroke:#333,color:#fff
  class groupSense EasyAR
  class SensePlugin EasyAR
  class SensePluginExtension EasyAR

  classDef Device fill:#636,stroke:#333,color:#fff
  class groupDeviceAUnity Device
  class DeviceB Device
  class DeviceA Device

In the figure:

  • Device A belongs to extension support

    In practice, device A usually has a corresponding Unity SDK for connecting to Unity XR SDK or independently implementing headset rendering capabilities.

    The headset extension package of device A is responsible for connecting EasyAR Sense Unity Plugin with the Unity SDK of device A, thereby enabling EasyAR to run on device A. This support package may be provided by EasyAR or by the device vendor.

  • Device B belongs to built-in support

    In practice, device B may or may not have a corresponding Unity SDK, depending on the device vendor's implementation. For example, Apple Vision Pro does not have a corresponding Unity SDK, while XREAL has a corresponding Unity SDK.

Headsets with built-in support and extension support both support using custom camera.

Important

When using trial products on a custom camera or headset, such as a Personal license, trial XR license, or trial Mega service, EasyAR Sense stops responding 100 seconds after each startup. For Mega users, this duration can be adjusted after approval through EasyAR business. This limitation does not apply to paid versions of EasyAR Sense or paid EasyAR Mega services.

In Unity, virtual camera rendering, projection matrix, transform, and so on are not controlled by EasyAR. They are usually controlled by the device SDK or Unity XR SDK. Device-specific features, such as gesture recognition and eye tracking, are still provided by the device and device SDK. When using them, you usually need to use EasyAR and the device SDK at the same time.

Next steps