EasyAR plane detection
EasyAR plane detection automatically detects horizontal or vertical planes in the environment while running EasyAR motion tracking, providing functionalities such as virtual object placement.
EasyAR plane detection principle
EasyAR plane detection is a simple environmental understanding capability synchronously completed during the operation of EasyAR motion tracking. The system continuously models the real environment based on spatiotemporal information obtained from the device camera and inertial sensors, thereby identifying and tracking horizontal and vertical planes in the environment. This provides foundational support for virtual object placement, interactive alignment, and spatial understanding.
The specific process is as follows:
Motion tracking
During motion tracking operation, EasyAR continuously acquires two core types of data: continuous image frames from the RGB camera, and data from the accelerometer and gyroscope. The system estimates the device's continuous 6DoF position and orientation in the world coordinate system through visual-inertial fusion algorithms, providing a stable, low-drift camera trajectory for subsequent spatial modeling and plane analysis.
Feature point detection and triangulation
Based on pose estimation, EasyAR extracts and tracks stable visual feature points (such as corners or texture-salient regions) from the image sequence. These feature points are triangulated using multi-view geometry methods to recover their positions in 3D space, forming a local 3D point cloud representation.
Plane candidate region generation
After obtaining the 3D point cloud, the system performs geometric analysis to discover point clusters that may belong to the same plane. By analyzing the relationship with the gravity direction, the system distinguishes different types of plane candidates:
- Horizontal planes: Normal vectors approximately parallel to the gravity direction (e.g., ground, tabletop);
- Vertical planes: Normal vectors approximately perpendicular to the gravity direction (e.g., walls, pillars).
Plane tracking and detection
EasyAR verifies and updates detected planes across consecutive frames:
- Determines whether newly observed 3D points support existing plane models;
- Dynamically adjusts plane extent, boundaries, and confidence based on observation consistency;
- Eliminates transient or unstable plane candidates. Results are only considered "available planes" when both geometric consistency and temporal stability meet requirements.
Plane coordinate system and virtual content alignment
Once a plane is confirmed, you can achieve more realistic AR effects based on plane detection results:
- Place virtual objects on planes for real-scale and orientation alignment;
- Perform raycasting (Hit Test) to map screen taps to real plane positions;
- Implement plane-based interaction logic, such as object snapping, movement, and occlusion judgment.
Since planes share the same world coordinate system with the motion tracking system, virtual objects maintain stable, continuous spatial consistency as the user moves the device.
Plane detection relies on the stable poses and spatial structure provided by motion tracking, while plane detection results can conversely enhance environmental understanding—for example, assisting content placement and interaction design. Together they form the core foundation of EasyAR's spatial perception capabilities, but remain architecturally decoupled: plane detection does not alter the pose estimation results of motion tracking itself.
Best practices
To ensure optimal user experience with plane detection, following these practices enhances effectiveness:
- Guide users to move slowly, avoiding standing still, rapid motion, or spinning in place.
- Avoid planes with no texture, solid colors, or reflective surfaces that are visually difficult to recognize.
Note
Plane detection is EasyAR's function for identifying horizontal or vertical planes in the environment. Surface tracking does not detect or recognize planar structures in the scene—these should be distinguished.