Table of Contents

3D spatial content display

When using AR, virtual objects usually need to be displayed. In simple test samples, simple geometric shapes can be used. However, for consumer-facing development, high-precision 3D models and animations generally need to be displayed, and events and interactions may be triggered when model nodes are clicked.

Models

Popular 3D models today are generally composed of triangle mesh. To make a model look realistic, we need to assign material to each triangle. A material usually implements a lighting model, and its content consists of textures and lighting model parameters.

  • Texture determines the base color of each point in a triangle. The textures of all triangles are placed in one or more diffuse textures. In some advanced uses, textures can also be used to represent the normal direction or other parameters of each point.
  • Lighting model defines how an object interacts with light. A common example is PBR. Lighting models are usually implemented with shader. Parameters of a PBR lighting model include color, metallic, roughness, and so on.

In application development, 3D models are usually not loaded directly on OpenGL / Metal / Vulkan / Direct3D. Instead, a 3D engine is used for loading. A 3D engine requires some specific 3D model formats, such as the long-standing and relatively readable obj / mtl format, and the currently popular glTF format.

Animation

To make a model move, skeletal animation is needed. Bones refer to large model nodes in a 3D model that maintain consistent movement during rigid-body motion.

To display animation, the position and pose (transformation matrix) of model nodes need to be continuously updated at runtime. Most 3D engines provide animation features. You only need to edit the animation in animation authoring software and export it in a format supported by the 3D engine, then it can be used in the 3D engine. The glTF format mentioned above also includes animation features.

Interaction

When a user clicks a model node, events and interactions sometimes need to be triggered. Usually, bones in skeletal animation are named, and collision detection or ray detection is used to trigger events on click and return the name of the clicked bone. Event handling can be done with scripts or application code.

Note

If you lack experience using 3D engines, it is strongly recommended that you consider using Unity to develop your application. EasyAR Sense Unity Plugin has good support for Unity. If you use other 3D engines, you may face limited support and fewer available resources.