Use the Mega plugin to implement occlusion
Occlusion is a key technology for improving the immersive integration of virtual and real content in AR. This article guides you through implementing occlusion effects in the xr-frame environment through EasyAR cloud localization and annotations.
Before you start
- Be able to use Mega Studio in Unity.
- Be able to create and upload annotations with the Unity editor.
- Be able to create content aligned with the real world.
How occlusion is implemented
Offline modeling: use the Unity editor to create 1:1 matching geometry in the Block coordinate system for real-world entities such as walls, columns, and large equipment; or obtain an optimized model by cropping and reducing the faces of the Block dense model.
Runtime alignment: at xr-frame runtime, align the Block coordinate system with the real space through cloud localization and load the corresponding geometry.
Material replacement: assign special occlusion materials to these geometries.
Visual effect: when the GPU renders other virtual objects, pixels in occluded parts are automatically culled because they fail the depth test, making virtual objects follow the occlusion logic of the real physical space.
How to arrange occlusion with simple geometry
Place box annotations accurately by comparing against the dense model and panorama. After placement, the annotation looks like a "wall" or "column".
Modify the annotation name, such as
occlusion_wall, record the ID, and upload the annotation.In the xr-frame Mini Program, use its built-in geometry to load the annotation used as occlusion.
In the EMA loading callback, use
scene.createElement(xrFrameSystem.XRMesh,{})to create simple geometry and assign theeasyar-occulusionmaterial.Note
The loading, registration, deregistration, and unloading of the
easyar-occulusionmaterial are controlled by AR Session.
```ts
handleEmaResult(ema: easyar.ema.v0_5.Ema) {
let blockHolder: easyar.BlockHolder = session.blockHolder;
ema.blocks.forEach(emaBlock => {
const blockInfo: easyar.BlockInfo = {
id: emaBlock.id
};
// 若 Block 节点不存在，创建 Block 节点
blockHolder.holdBlock(blockInfo, easyarPlugin.toXRFrame(emaBlock.transform));
});
ema.annotations.forEach(annotation => {
if (annotation.type != mega.EmaV05AnnotationType.Node) {
return;
}
const nodeAnnotation = annotation as easyar.ema.v0_5.Node;
const xrNode: xrfs.XRNode = easyarPlugin.createXRNodeFromNodeAnnotation(nodeAnnotation, blockHolder);
const emaName: string = nodeAnnotation.name;
const geometryStr: string = nodeAnnotation.geometry === "cube" ? "cube" : "sphere";
const assetInfo = AnnotationMetaData[nodeAnnotation.id as keyof typeof AnnotationMetaData];
let model: xrfs.Element;
if (assetInfo) {
// GLTF部分
} else {
model = scene.createElement(
xrFrameSystem.XRMesh,
{
// 使用插件注册好的遮挡材质
material: "easyar-occlusion",
// 使用 xr-frame 内置几何体，此处也可以直接使用 "cube"
geometry: geometryStr,
name: emaName,
"receive-shadow": "false",
"cast-shadow": "false"
// 注意不要修改 Scale
}
);
xrNode.addChild(model);
}
})
}
```
<video src="https://doc-asset.easyar.com/develop/wechat/mega/media/occlusion03.mp4" style="width:480px; max-width:100%; height:auto;" muted playsinline controls></video>
> With occlusion, this panda can dance behind the wall.
How to arrange occlusion with complex geometry
This applies to scenarios that require high-precision occlusion, such as irregular devices and irregular buildings.
You can crop and reduce the Block dense model to obtain the white model you need for occlusion.
In the Unity scene, click the Mega Block node and record the BlockID in the Inspector panel.
Select export in Block of Mega Studio.
Modify the export options and export.
In the figure, 1 is the LOD level. The lower the level, the simpler the model and the fewer faces. Select 2 if you need the highest precision, or select 1 or 0 if you can accept reduced precision to reduce the face count.
In the figure, 2 is the texture export option. Because only the white model is needed as occlusion, textures are not needed.
Crop and reduce the exported model in digital content creation software, such as Blender, and save it as
Glb.Tip
The example uses Blender's Decimate Modifier.
After cropping and reduction:
Mount the
Glbfile used for occlusion on a file server to obtain a url for loading.Load the GLTF used as occlusion in the xr-frame Mini Program.
First load the GLTF model used for occlusion, then use
scene.createElement(xrFrameSystem.XRGLTF,options)to create the GLTF model.Use
assets.getAsset("material", "easyar-occlusion")to get the material object.Use
model.getComponent(xrFrameSystem.GLTF).meshes.forEach((m: any) => {m.setData({ neverCull: true, material: occlusionMaterial });}to modify the material of the GLTF model.Note
The loading, registration, deregistration, and unloading of the
easyar-occulusionmaterial are controlled by AR Session.
```ts
const sampleAssets = {
occlusion1: {
assetId: "occlusion1",
type: "gltf",
src: "url/occlusion1.glb",
options: {}
}
}
async loadAsset() {
if (!scene) {console.error("Empty scene"); return;}
try {
await scene.assets.loadAsset(sampleAssets.occlusion1);
} catch (err) {
console.error(`Failed to load assets: ${err.message}`);
}
},
addOcclusion() {
model = scene.createElement(
xrFrameSystem.XRGLTF,
{
"model": assetInfo.assetId,
"anim-autoplay": assetInfo.animation ? assetInfo.animation : "",
"scale": assetInfo.scale ? assetInfo.scale : "1 1 1",
name: "tree"
}
);
const blockID = "aaaa1234-bbbb-cccc-dddd-eeeeee123456" //Fill in the Block ID here
if (!blockHolder.getBlockById(blockParent.id)) {
// If no Block node exists, create one
blockHolder.holdBlock({
id: blockID
})
}
// Get the Block node in the xr-frame scene
let blockElement = blockHolder.getBlockById(blockParent.id).el;
// Attach the clipped occlusion model under the Block node as its child node
blockElement.addChild(model);
/**
* Because GLTF loaders behave differently, to keep the model orientation in xr-frame exactly consistent with the Unity rendering result
* Sometimes the loaded model needs to be rotated 180 degrees around the Y axis in place
*/
let modelTransform = model.getComponent(xrFrameSystem.Transform);
let currentRotation = modelTransform.quaternion.clone();
let targetRotation = currentRotation.multiply(new xrFrameSystem.Quaternion().setValue(0, 1, 0, 0));
modelTransform.quaternion.set(targetRotation);
//Note: the material must be changed after modifying Transform
if (assetInfo.assetId == 'occlusion1') {
//Get the occlusion material provided by the Mega plugin
let occlusionMaterial = scene.assets.getAsset("material", "easyar-occlusion");
//Modify the occlusion material
model.getComponent(xrFrameSystem.GLTF).meshes.forEach((m: any) => {
m.setData({ neverCull: true, material: occlusionMaterial });
});
}
}
```
> [!NOTE]
> Here, using the Mega Block dense model after cropping as occlusion does not require annotation synchronization for spatial position. This is because in digital content creation software, such as Blender, the model can be reduced and cropped without changing the coordinate system definition.
>
> If you need to precisely place your own GLTF model as occlusion, see [How to place an occlusion model aligned with space](./sample.md#wechat-mega-sample-precise-occulusion-model).
The final real-device running effect is shown in the video at the top of this article.
Expected occlusion effect
The occlusion effect in an xr-frame Mini Program is mainly affected by the following:
- The accuracy of localization tracking itself
- The accuracy of model placement
- The accuracy of the model itself, if it is not simple geometry
It is normal for several centimeters of misalignment to occur during localization drift.
Too many faces in the occlusion model can easily affect performance. It is recommended to use it only in necessary areas and use simple geometry as occlusion as much as possible.