How to load 3D content in AR scenarios at xr-frame runtime

This article elaborates on the separation mechanism between resource loading and node mounting in xr-frame, dynamically achieving flexible mounting of 3D content under Block nodes through scripting to realize AR.

Official materials

• xr-frame development guide: WeChat official XR engine documentation.
• xr-frame official examples: Includes various basic and advanced usage examples.

The official materials already provide sufficient content on how to load 3D content at runtime. This article only briefly explains some commonly used content and loading methods in AR scenarios.

Resource loading vs node mounting

In xr-frame, displaying a 3D model is divided into two stages:

1. Resource loading: refers to downloading and parsing the model file (such as .glb) from the network or local storage into memory. At this point, the model is ready but not visible in the scene.

2. Node mounting: refers to creating a node in the scene tree and associating the loaded resource with that node. The model will then officially appear in the rendering canvas.

How to dynamically load 3D content with code

1. Resource loading

   Use the resource management system of the xr-frame scene to manually load resources by calling loadAsset.

   The type in the parameters refers to the resource type, assetId refers to the loaded resource id, and src refers to the url of the resource, usually the address of the resource hosting server.

   The assetId needs to be recorded for subsequent resource mounting and release.

   try {  
       await scene.assets.loadAsset({type: 'gltf', assetId: 'panda', src: 'url/EasyARPanda.glb'});  
   } catch (err) {  
       console.error(`Failed to load assets: ${err.message}`);  
   }  
   

2. Node mounting

   Use element.addChild() to place the loaded model under the ShadowRoot.

   const root = scene.getElementById("shadow-root");  
   let panda = scene.createElement(xrFrameSystem.XRGLTF,  
       {  
           "model": "panda",  
           "anim-autoplay": ""  
       }  
   );  
   root.addChild(panda);  
   

   The ShadowRoot element is a root node specifically used by xr-frame to prevent dynamically creating and removing nodes. For details, see Shadow node.

   Using the createXRNodeFromNodeAnnotation method provided by the plugin object can create a child node of Block based on the EMA data, ensuring the 3D content is displayed in the correct spatial position.

   const nodeAnnotation = annotation as easyar.ema.v0_5.Node;  
   const xrNode: xrfs.XRNode = easyarPlugin.createXRNodeFromNodeAnnotation(nodeAnnotation, blockHolder);  
   let panda = scene.createElement(xrFrameSystem.XRGLTF,  
       {  
           "model": "panda",  
           "anim-autoplay": ""  
       }  
   );  
   xrNode.addChild(panda);  
   

How to mount content directly under a block without using annotations

Warning

The prerequisite for using this method is that you have verified that the LocalTransform values can achieve the expected rendering effect in the xr-frame coordinate system.

For other cases, please use the annotation function of the Unity editor to achieve it.

Use getBlockById(id) to get the block node object on the scene tree. If the corresponding block node does not exist, it means that the positioning for this Block has not been successful yet (the node will be automatically created when the Block is positioned for the first time). You can use holdBlock(blockInfo, blockTransformInput) to create a node for this Block, or you can determine the successful positioning of the Block in the positioning callback before mounting the content.

Tip

Select the Block node in the scene tree of the Unity editor and record the ID displayed in its Inspector panel.

You can also check the Block ID in the cloud positioning library page.

const blockID = "aaaa1234-bbbb-cccc-dddd-eeeeee123456"
if (!blockHolder.getBlockById(blockParent.id)) {
    // No existing Block node, create one
    blockHolder.holdBlock({
        id: blockID
    })
}
let blockElement = blockHolder.getBlockById(blockParent.id).el;

Mount the model node under the specified Block, and use position.setArray(), quaternion.set(), and scale.setArray() to modify the LocalTransform of the model node.

export interface LocalTransform {
    /** @description Position */
    position: xrfs.Vector3;
    /** @description Rotation */
    rotation: xrfs.Quaternion;
    /** @description Scale */
    scale: xrfs.Vector3;
}

// Assume there is a known LocalTransform under the Block
const targetTransform: LocalTransform;

blockElement.addChild(modelNode);
let modelTransform = modelNode.getComponent(xrFrameSystem.Transform);
    modelTransform.position.setArray([
        targetTransform.position.x,
        targetTransform.position.y,
        targetTransform.position.z
    ]);
    let annoRotation = new xrFrameSystem.Quaternion().setValue(
        targetTransform.rotation.x,
        targetTransform.rotation.y,
        targetTransform.rotation.z,
        targetTransform.rotation.w
    );
    modelTransform.quaternion.set(annoRotation);
    modelTransform.scale.setArray([
        targetTransform.scale.x,
        targetTransform.scale.y,
        targetTransform.scale.z
    ]);

Xr-frame supported resource types

• Texture texture and image
• CubeTexture cube texture
• VideoTexture video texture
• EnvData environment
• GLTF model
• Keyframe frame animation
• Atlas sprite sheet

For detailed loading methods of each resource, refer to the WeChat official documentation and the xr-frame official examples

Note

Supported GLTF formats and extensions refer to the xr-frame official GLTF usage instructions