Use the Mega plugin to achieve occlusion
Occlusion is a key technology to enhance the immersion of AR virtual-real fusion. This article will guide you on how to achieve occlusion effects in the xr-frame environment through EasyAR cloud positioning and annotation.
Before you begin
- Be able to use Mega Studio in Unity.
- Be able to create and upload annotations using the Unity editor.
- Be able to create content aligned with the real world.
Implementation of occlusion
Offline modeling: Use the Unity editor to create 1:1 matching geometric bodies for real-world entities (such as walls, columns, and large equipment) in the Block coordinate system; or obtain optimized models by cropping and reducing the faces of the dense Block model.
Runtime alignment: During xr-frame runtime, align the Block coordinate system with the real space through cloud localization and load the corresponding geometric bodies.
Material replacement: Assign special occlusion materials to these geometric bodies.
Visual effect: When the GPU renders other virtual objects, pixels of the occluded parts are automatically culled due to failed depth testing, making the virtual objects follow the occlusion logic of real physical space.
How to place occlusion for simple geometries
Precisely position the cube annotations by referencing the dense model and panoramic image. After placement, the annotations should resemble a "wall" or "column".
Modify the annotation name (e.g.
occlusion_wall), record its ID, and upload the annotation.Utilize the built-in geometries in the xr-frame applet as occlusion annotations.
In the EMA loading callback, create simple geometries using
scene.createElement(xrFrameSystem.XRMesh,{})and assign them theeasyar-occlusionmaterial.Note
The loading, registration, deregistration, and unloading of the
easyar-occlusionmaterial are controlled by the AR Session.
handleEmaResult(ema: easyar.ema.v0_5.Ema) {
let blockHolder: easyar.BlockHolder = session.blockHolder;
ema.blocks.forEach(emaBlock => {
const blockInfo: easyar.BlockInfo = {
id: emaBlock.id
};
// If the Block node does not exist, create a Block node
blockHolder.holdBlock(blockInfo, easyarPlugin.toXRFrame(emaBlock.transform));
});
ema.annotations.forEach(annotation => {
if (annotation.type != mega.EmaV05AnnotationType.Node) {
return;
}
const nodeAnnotation = annotation as easyar.ema.v0_5.Node;
const xrNode: xrfs.XRNode = easyarPlugin.createXRNodeFromNodeAnnotation(nodeAnnotation, blockHolder);
const emaName: string = nodeAnnotation.name;
const geometryStr: string = nodeAnnotation.geometry === "cube" ? "cube" : "sphere";
const assetInfo = AnnotationMetaData[nodeAnnotation.id as keyof typeof AnnotationMetaData];
let model: xrfs.Element;
if (assetInfo) {
// GLTF part
} else {
model = scene.createElement(
xrFrameSystem.XRMesh,
{
// Use the occlusion material registered by the plugin
material: "easyar-occlusion",
// Use the built-in geometry of xr-frame, here you can also directly use "cube"
geometry: geometryStr,
name: emaName,
"receive-shadow": "false",
"cast-shadow": "false"
// Do not modify Scale
}
);
xrNode.addChild(model);
}
})
}
<video src="https://doc-asset.easyar.com/develop/wechat/mega/media/occlusion03.mp4" style="width:480px; max-width:100%; height:auto;" muted playsinline controls></video>
> With occlusion implemented, this panda can now dance while hiding behind the wall.
How to set up occlusion for complex geometries
Suitable for scenarios requiring high-precision occlusion, such as irregular devices or unconventional buildings.
You can use Block's dense model for clipping and decimation to obtain the white model needed for occlusion.
In the Unity scene, click on the Mega Block node and record the BlockID in the Inspector panel.
In Mega Studio's Block, select export.
Modify the export options and export.
In the image, 1 represents the LOD level. The lower the level, the simpler the model and the fewer polygons. For the highest precision, choose 2. If you can accept reduced precision to decrease polygon count, choose 1 or 0.
In the image, 2 represents the texture export option. Since we only need the white model for occlusion, textures are unnecessary.
Clip and decimate the exported model in digital content creation software (e.g., Blender) and save it as
Glb.Tip
The example uses Blender's Decimate Modifier.
After clipping and decimation:
Upload the occlusion
Glbfile to a file server to obtain a URL for loading.Load the GLTF as occlusion in the xr-frame mini-program.
First, load the GLTF model for occlusion, then use
scene.createElement(xrFrameSystem.XRGLTF,options)to create the GLTF model.Use
assets.getAsset("material", "easyar-occlusion")to retrieve the material object.Use
model.getComponent(xrFrameSystem.GLTF).meshes.forEach((m: any) => {m.setData({ neverCull: true, material: occlusionMaterial });}to modify the GLTF model's material.Note
The loading, registration, deregistration, and unloading of the
easyar-occulusionmaterial are controlled by the AR Session.
const sampleAssets = {
occlusion1: {
assetId: "occlusion1",
type: "gltf",
src: "url/occlusion1.glb",
options: {}
}
}
async loadAsset() {
if (!scene) {console.error("Empty scene"); return;}
try {
await scene.assets.loadAsset(sampleAssets.occlusion1);
} catch (err) {
console.error(`Failed to load assets: ${err.message}`);
}
},
addOcclusion() {
model = scene.createElement(
xrFrameSystem.XRGLTF,
{
"model": assetInfo.assetId,
"anim-autoplay": assetInfo.animation ? assetInfo.animation : "",
"scale": assetInfo.scale ? assetInfo.scale : "1 1 1",
name: "tree"
}
);
const blockID = "aaaa1234-bbbb-cccc-dddd-eeeeee123456" //Block ID should be filled in here
if (!blockHolder.getBlockById(blockParent.id)) {
// If there is no existing Block node, create one
blockHolder.holdBlock({
id: blockID
})
}
// Retrieve the Block nodes in the xr frame scene
let blockElement = blockHolder.getBlockById(blockParent.id).el;
// Mount the cropped occlusion model under the Block node as its child node
blockElement.addChild(model);
/**
* Due to the different behaviors of the GLTF loader, in order to ensure that the orientation of the model on xr frame is completely consistent with the rendering results of Unity,
* it is sometimes necessary to rotate the loaded model 180 degrees in place around the Y-axis
*/
let modelTransform = model.getComponent(xrFrameSystem.Transform);
let currentRotation = modelTransform.quaternion.clone();
let targetRotation = currentRotation.multiply(new xrFrameSystem.Quaternion().setValue(0, 1, 0, 0));
modelTransform.quaternion.set(targetRotation);
// Note that the material must be modified after modifying the Transform
if (assetInfo.assetId == 'occlusion1') {
// Obtain the occlusion material provided by the Mega plugin
let occlusionMaterial = scene.assets.getAsset("material", "easyar-occlusion");
// Modify the occlusion material
model.getComponent(xrFrameSystem.GLTF).meshes.forEach((m: any) => {
m.setData({ neverCull: true, material: occlusionMaterial });
});
}
}
> [!NOTE]
> When using Mega Block's dense model for clipping as occlusion, there is no need to use annotation to synchronize spatial positioning. This is because in digital content creation software (e.g., Blender), you can decimate and clip the model without altering the coordinate system definition.
>
> If precise placement of a custom GLTF model for occlusion is required, refer to [How to place a spatially aligned occlusion model](./sample.md#wechat-mega-sample-precise-occulusion-model).
For the final real-device running effect, see the video at the top of the article.
Effect expectation of occlusion
The occlusion effect on the xr-frame mini-program is mainly influenced by the following points:
- Accuracy of positioning and tracking itself
- Accuracy of model placement
- Precision of the model itself (if it is not a simple geometric shape)
A misalignment of several centimeters during positioning drift is normal.
Occlusion models with too many polygons can easily affect performance. It is recommended to use them only in necessary areas and to use simple geometric shapes as occlusion whenever possible.