Table of Contents

WeChat Mini Program Mega Plugin Sample Project Description

This article details the usage, implementation methods, and considerations of the various features demonstrated in the sample project.

Before you begin

How to display a model at the annotation location

  1. Precisely place and upload annotations in the Unity editor, recording annotation names and their IDs

    Unity annotation location

  2. Add GLTF model resources

    Add model resources in sampleAssets within miniprogram/components/sample-easyar-mega/index.ts.

    const sampleAssets = {
    your_model_name: {
        assetId: "your_model_asset_id",
        type: "gltf",
        src: "url/model.glb",
        options: {}
    }
}

  3. Load the added model resources

    Load the model in the loadAsset() function within miniprogram/components/sample-easyar-mega/index.ts.

    async loadAsset() {
    try {
        await scene.assets.loadAsset(sampleAssets.your_model_name);
    } catch (err) {
        console.error(`Failed to load assets: ${err.message}`);
    }
}

  4. Configure annotations to be replaced

    Configure target annotations in miniprogram/components/sample-data/annotation-metadata.ts. Separate multiple entries with commas.

    export const AnnotationMetaData: Record<string, any> = {
    "aaaaaaaa-bbbb-cccc-dddd-123456789012": {
        assetId: "panda",
        scale: "0.5 0.5 0.5"
    },
    "aaaaaaaa-bbbb-cccc-dddd-123456789013": {
        assetId: "your_model_asset_id",
        scale: "1 1 1"
    }
};

  5. Replace annotations to load models

    Use xr-frame’s factory method scene.createElement(xrFrameSystem.XRGLTF, options) in EMA’s callback to create model nodes.

    • Parameters:

      • xrFrameSystem.XRGLTF: Specifies the element type as GLTF model.
      • options: Initialization properties corresponding to component attributes.

    • Key attributes:

      • "model": Required. References the loaded resource ID (asset-id).
      • "anim-autoplay": Optional. Specifies auto-play animation after loading.
      • "scale": Optional. Uses assetInfo.scale or "1 1 1".
      • name: Required. Annotation name.
    Caution

    Ensure correct attribute key formatting (string vs non-string) as shown in the example.

    Mount the model under the annotation node via xrNode.addChild(child).

    Rotate loaded GLTF models 180 degrees around the Y-axis locally to ensure consistent rendering across platforms.

    if (assetInfo && assetInfo.assetId && assetInfo.assetId.trim().length > 0) {
    model = scene.createElement(
        xrFrameSystem.XRGLTF,
        {
            /** assetId from previous step */
            "model": assetInfo.assetId,
            /** Specify model animation here */
            "anim-autoplay": assetInfo.animation ? assetInfo.animation : "",
            "scale": assetInfo.scale ? assetInfo.scale : "1 1 1",
            name: emaName
        }
    );
    xrNode.addChild(model);
    /**
     * Rotate model 180° around Y-axis locally to ensure consistent orientation 
     * between xr-frame and Unity due to differing GLTF loader behaviors
     */
    let modelTransform = model.getComponent(xrFrameSystem.Transform);
    let currentRotation = modelTransform.quaternion.clone();
    let targetRotation = currentRotation.multiply(new xrFrameSystem.Quaternion().setValue(0, 1, 0, 0));
    modelTransform.quaternion.set(targetRotation);
}

  6. Run on device

    • Compare real-device results with Unity editor positions from step 1:

    • Enable the transparent video button: A transparent-video-material cube appears at world origin (0, 0, 0).

      Note

      The origin position may be arbitrary. Place occlusion models via annotations as needed (see Create and upload annotations using Unity editor).

    • Enable the occlusion button: A panda model and stacked cubes appear at world origin (0, 0, 0). The middle cube uses occlusion material, while a static panda model with occlusion material appears opposite.

      Note

      The origin position may be arbitrary. Place occlusion models via annotations as needed (see Create and upload annotations using Unity editor).

      Models and occlusion

How to play a transparent video at the annotation location

  1. Load a video asset of type video-texture.

    async loadAsset() {
    const videoTexture = {
        assetId: "fireball",
        type: "video-texture",
        // Video resource URL
        src: "url/video-resource.mp4",
        options: {
            autoPlay: true,
            loop: true,
        }
    };
    try {
        // Load the video-texture type resource
        await scene.assets.loadAsset(videoTexture);
    } catch (err) {
        console.error(`Failed to load video texture: ${err.message}`);
    }
}

  2. Modify the EMA loading callback

    Use scene.createElement(xrFrameSystem.XRMesh,options) in the EMA loading callback to create a simple geometric shape and assign the easyar-video-tsbs material, then modify the uniform to u_baseColorMap:video-{$assetId}.

    • Parameters:

      • xrFrameSystem.XRMesh: Specifies the type of element to create as a basic geometric shape.
      • options: Initial configuration items corresponding to the component's properties.

    • Key attributes in the code:

      • "geometry": "cube": Use the built-in cube geometric data of xr-frame.
      • "material": "easyar-video-tsbs": Specify a predefined material (inferred from the name, this is a special material that supports video textures).
      • "uniforms": "u_baseColorMap:video-{$assetId}":
      Caution

      Pay attention to distinguishing between string and non-string attribute keys. Fill them in exactly as shown in the example.

      This is the dynamic binding of material parameters.

      It maps the video resource (texture) named video-{$assetId} to the base color map of the material.

      Effect: This will create a cube with a surface playing a video.

    model = scene.createElement(xrFrameSystem.XRMesh, {
    geometry: "cube",
    material: "easyar-video-tsbs",
    uniforms: "u_baseColorMap:video-fireball",
});
xrNode.addChild(model);
    Note

    When using video-texture, if the console shows a warning wx.createVideoDecoder with type: 'wemedia' is deprecated, please ignore it.

    Confirmed with the WeChat official team, this warning does not affect usage.

  3. Run on a real device

How to place an occlusion model aligned with the space

  1. Precisely place the occlusion model and upload the annotation.

    Precise alignment

  2. Load the occlusion GLTF in the xr-frame mini program.

    Use scene.assets.loadAsset() to load the model resource (requires manual unloading).

    const sampleAssets = {
    occlusion1: {
        assetId: "occlusion1",
        type: "gltf",
        src: "url/occlusion1.glb",
        options: {}
    }
}
async loadAsset() {
    if (!scene) {console.error("Empty scene"); return;}
    try {
        await scene.assets.loadAsset(sampleAssets.occlusion1);
    } catch (err) {
        console.error(`Failed to load assets: ${err.message}`);
    }
}

  3. Load the model and assign occlusion material during runtime in the EMA load callback

    Use scene.createElement(xrFrameSystem.XRGLTF,options) in the EMA load callback to create a model node.

    • Parameters:

      • xrFrameSystem.XRGLTF: Specifies the element type to be created as a GLTF model.
      • options: Initialization configuration items, corresponding to the component's properties.

    • Key properties in the code:

      • "model": Required, points to the loaded resource ID (asset-id).
      • "scale": Optional assetInfo.scale or "1 1 1".
      • name: Required, annotation name.
    Caution

    Note the distinction between string and non-string property keys. Fill them exactly as shown in the example.

    Mount the model under the annotation node using xrNode.addChild(child).

    To ensure the GLTF model appears the same across different platform loaders, rotate the loaded model 180 degrees around the Y-axis in place.

    Finally, use model.getComponent(xrFrameSystem.GLTF).meshes.forEach((m: any) => {m.setData({ neverCull: true, material: occlusionMaterial });} to modify the GLTF model's material.

    Note

    The loading, registration, deregistration, and unloading of the easyar-occulusion material are controlled by the AR Session.

    Use the model at the annotation position as occlusion:

    if (...) {
    model = scene.createElement(
        xrFrameSystem.XRGLTF,
        {
            "model": assetInfo.assetId,
            "scale": assetInfo.scale ? assetInfo.scale : "1 1 1",
            name: emaName
        }
    );
    /**
    * Due to differences in GLTF loader behavior, to ensure the model's orientation in xr-frame matches Unity's rendering results exactly,
    * sometimes it's necessary to rotate the loaded model 180 degrees around the Y-axis in place.
    */
    let modelTransform = model.getComponent(xrFrameSystem.Transform);
    let currentRotation = modelTransform.quaternion.clone();
    let targetRotation = currentRotation.multiply(new xrFrameSystem.Quaternion().setValue(0, 1, 0, 0));
    modelTransform.quaternion.set(targetRotation);
    //Note: The material must be modified after adjusting the Transform.
    if (assetInfo.assetId == 'occlusion1') {
        //Get the occlusion material provided by the mega plugin.
        let occlusionMaterial = scene.assets.getAsset("material", "easyar-occlusion");
        //Modify the occlusion material.
        model.getComponent(xrFrameSystem.GLTF).meshes.forEach((m: any) => {
            m.setData({ neverCull: true, material: occlusionMaterial });
        });
    }
}

  4. Run on the device

    Compare the results with the simulated run in the Unity editor.