Table of Contents

Access AR feature components in the session

In a running session, you can access various feature components through the Assembly property. This article explains how to access these components and the considerations to keep in mind.

Before you begin

Configure AR components during editing or before startup

Sometimes, certain component options (such as DesiredFocusMode) must be configured before the component starts. If you don't want to manually configure and start the component after the session starts, a simple method is to configure all potentially used frame source components before assembling the session. The assembly process will retain one or more of these components and apply their configurations.

In this case, you can use methods like FindAnyObjectByType<T>() or GetComponent<T>() or any other basic Unity methods to find the components and then configure them.

Note

It is uncertain whether the AR components obtained through this method will be included in the session at runtime. Therefore, all possible scenarios must be configured.

For example, the following code demonstrates the process of modifying the focus mode for all frame source components before assembling the session:

void Awake()
{
    var allFrameSources = Session.GetComponentsInChildren<FrameSource>();
    foreach (var source in allFrameSources)
    {
        if (source is CameraDeviceFrameSource)
        {
            ((CameraDeviceFrameSource)source).DesiredFocusMode = autoFocus ? CameraDeviceFocusMode.Continousauto : CameraDeviceFocusMode.Medium;
        }
        else if (source is MotionTrackerFrameSource)
        {
            ((MotionTrackerFrameSource)source).DesiredFocusMode = autoFocus ? MotionTrackerCameraDeviceFocusMode.Continousauto : MotionTrackerCameraDeviceFocusMode.Medium;
        }
        else if (source is ARCoreFrameSource)
        {
            ((ARCoreFrameSource)source).DesiredFocusMode = autoFocus ? ARCoreCameraDeviceFocusMode.Auto : ARCoreCameraDeviceFocusMode.Fixed;
        }
        else if (source is ARKitFrameSource)
        {
            ((ARKitFrameSource)source).DesiredFocusMode = autoFocus ? ARKitCameraDeviceFocusMode.Auto : ARKitCameraDeviceFocusMode.Fixed;
        }
        else if (source is AREngineFrameSource)
        {
            ((AREngineFrameSource)source).DesiredFocusMode = autoFocus ? AREngineCameraDeviceFocusMode.Auto : AREngineCameraDeviceFocusMode.Fixed;
        }
        else if (source is ThreeDofCameraDeviceFrameSource)
        {
            ((ThreeDofCameraDeviceFrameSource)source).DesiredFocusMode = autoFocus ? ThreeDofCameraDeviceFocusMode.Auto : ThreeDofCameraDeviceFocusMode.Fixed;
        }
        else if (source is InertialCameraDeviceFrameSource)
        {
            ((InertialCameraDeviceFrameSource)source).DesiredFocusMode = autoFocus ? InertialCameraDeviceFocusMode.Auto : InertialCameraDeviceFocusMode.Fixed;
        }
        else if (source is ARFoundationFrameSource)
        {
            cameraManager.autoFocusRequested = autoFocus;
        }
    }
}

The above process can also be completed in the editor, and all components must be configured accordingly:

alt text

Among them, the configurations for the two frame sources ARCoreARFoundationFrameSource and ARKitARFoundationFrameSource are located on the Main Camera component.

Warning

AR components obtained through this method can only be used for pre-runtime configuration.
Since the assembly process filters AR components, the AR components obtained through the scene hierarchy may not be included in the session and will not function properly.

Using assembled AR components during runtime

The AR components running in a session are determined after assembly. Before assembly is complete, no AR components can be used. The assembled AR components can be accessed through the Assembly property.

Assembly is available when the session's state is >= Assembled. Specifically, the Assembly property is assigned only after the Assemble() method completes, allowing access to the session components. After the session stops or becomes corrupted, the Assembly property is cleared, and the components can no longer be accessed.

You can check the State of the session in a script to determine whether the AR components are accessible at that time:

if (Session.State >= ARSession.SessionState.Ready)
{
    // Assembly is available
}
else
{
    // Assembly is not available
}

Alternatively, you can subscribe to the StateChanged event to monitor session state changes and access the AR components at the appropriate time. Generally, to capture the Ready state, you should subscribe to the StateChanged event before the session starts. It is usually safe to subscribe in Awake():

void Awake()
{
    Session.StateChanged += (state) =>
    {
        if (Session.State == ARSession.SessionState.Ready)
        {
            // Assembly is available, and it remains accessible until the session stops or becomes corrupted
        }
        else if (Session.State < ARSession.SessionState.Ready)
        {
            // Assembly is not available, and it remains inaccessible until the session restarts
        }
        else
        {
            // Assembly is available, usually no handling is needed
        }
    };
}
Caution

If you obtain AR components through methods like FindAnyObjectByType<T>() or GetComponent<T>(), they will always be included in the session and can be used during runtime.
Storing references to these components is safe, but when using them, you must ensure the session is running and these components are correctly included in the session. Otherwise, exceptions or unexpected behavior may occur.
These components will not function before the session starts or after it stops. Even when using this approach, it is recommended to monitor the session's State and StateChanged event.

Access frame source component

You can use the ARAssembly.FrameSource property to access the frame source component. In a normally running session, there is one and only one ARAssembly.FrameSource.

When using a session, you usually need to access ARAssembly.FrameSource to determine the actual type of frame source component used at runtime, so as to access the specific properties and methods of that component.

For example, the following code shows how to use different plane detection methods based on the frame source:

void PlaceObject(Vector2 touchPosition)
{
    if (Session.Assembly.FrameSource is MotionTrackerFrameSource)
    {
        Ray ray = Session.Assembly.Camera.ScreenPointToRay(touchPosition);
        if (Physics.Raycast(ray, out var hitInfo))
        {
            TouchRoot.transform.position = hitInfo.point;
        }
    }
    else if (Session.Assembly.FrameSource is ARFoundationFrameSource)
    {
        var raycastManager = Session.Assembly.Origin.Value.GetComponent<UnityEngine.XR.ARFoundation.ARRaycastManager>();
        var hits = new List<UnityEngine.XR.ARFoundation.ARRaycastHit>();
        if (raycastManager.Raycast(touchPosition, hits, UnityEngine.XR.ARSubsystems.TrackableType.PlaneWithinPolygon))
        {
            var hitPose = hits[0].pose;
            TouchRoot.transform.position = hitPose.position;
        }
    }
}

Access frame filter components

You can use the ARAssembly.FrameFilters property to access frame filter components. In a normally running session, there may be multiple components of any type in the ARAssembly.FrameFilters list.

For example, the following code shows how to obtain a MegaTrackerFrameFilter in the session and register corresponding events:

var megaTracker = session.Assembly.FrameFilters.Where(f => f is MegaTrackerFrameFilter).FirstOrDefault() as MegaTrackerFrameFilter;
if (megaTracker)
{
    megaTracker.LocalizationRespond += (response) =>
    {
    };
}

Access camera component

You can use the ARAssembly.Camera property to access the camera component. If there are multiple cameras in the scene, this is a quick way to find the camera used for AR.

For example, the following code shows how to get the camera in the session and perform raycasting on objects in the scene:

var ray = Session.Assembly.Camera.ScreenPointToRay(screenPoint);
if (Physics.Raycast(ray, out var hitInfo))
{
    TouchRoot.transform.position = hitInfo.point;
};

Access the origin component

You can use the ARAssembly.Origin property to access the origin component.

For example, the following code shows how to get the origin in the session and display a cone representing the current camera position and orientation in the scene:

if (session.Assembly.Origin.OnSome)
{
    GameObject frustum = Instantiate(CameraFrustumPrefab, session.Assembly.Camera.transform.position, session.Assembly.Camera.transform.rotation);
    frustum.transform.SetParent(session.Assembly.Origin.Value.transform);
}

Note that you need to check whether ARAssembly.Origin exists first.

Note

ARAssembly.Origin only exists in sessions with motion tracking enabled.

Access the CameraImageRenderer component

You can use the ARAssembly.CameraImageRenderer property to access the CameraImageRenderer component.

For example, the following code can obtain the RenderTexture of the physical camera image:

RenderTexture renderTexture;

void Awake()
{
    Session.StateChanged += (state) =>
    {
        if (state == ARSession.SessionState.Ready && Session.Assembly.CameraImageRenderer.OnSome)
        {
            Session.Assembly.CameraImageRenderer.Value.RequestTargetTexture((_, texture) => renderTexture = texture);
        }
    };
}

Note that you need to first check whether ARAssembly.CameraImageRenderer exists.

Note

ARAssembly.CameraImageRenderer is only valid in sessions where the rendering is performed by EasyAR. Generally, it is not valid when using AR Foundation or headset SDKs, as the rendering of the physical camera image is handled by AR Foundation or the headset SDK.

Access the FrameRecorder component

You can use the ARAssembly.FrameRecorder property to access the FrameRecorder component.

For example, the following code can start recording, and the file storage location depends on the configuration. By default, it will be stored in the application's internal storage directory:

if (session.Assembly.FrameRecorder.OnSome)
{
    var frameRecorder = session.Assembly.FrameRecorder.Value;
    frameRecorder.enabled = true;
}

Note that you need to first check whether ARAssembly.FrameRecorder exists.

Note

ARAssembly.FrameRecorder cannot be used in some cases, such as when using FramePlayer.

Next steps

  • Learn how to get session results, which contain the output of AR components
  • Additionally, you can explore component access through the following examples: