EasyAR Plane Detection
EasyAR Plane Detection automatically detects horizontal planes in the environment during the operation of EasyAR Motion Tracking, providing functions such as virtual object placement.
EasyAR Plane Detection Principle
EasyAR Plane Detection is a simple environmental understanding ability that is automatically completed synchronously during the operation of EasyAR Motion Tracking. Based on the spatio - temporal information obtained from the device's camera and inertial sensors, the system continuously models the real environment to identify and track horizontal planes in the environment, providing basic support for virtual object placement, interactive alignment, and spatial understanding.
The specific process is as follows:
- Motion Tracking
During the operation of motion tracking, EasyAR continuously obtains the following two types of core data: continuous image frames from the RGB camera, and data from the accelerometer and gyroscope. The system estimates the continuous six - degree - of - freedom position and pose of the device in the world coordinate system through a visual - inertial fusion algorithm, providing a stable and low - drift camera trajectory for subsequent spatial modeling and plane analysis.
- Feature Point Detection and Triangulation
Based on the pose estimation, EasyAR extracts and tracks stable visual feature points (such as corner points or regions with significant textures) from the image sequence, and triangulates these feature points through multi - view geometry methods to recover their positions in three - dimensional space, forming a local three - dimensional point cloud representation.
- Plane Candidate Region Generation
After obtaining the three - dimensional point cloud, the system performs geometric analysis on the point cloud to find point sets that may belong to the same plane. By judging the relationship with the direction of gravity, the system can distinguish horizontal plane candidates, that is, planes whose normal vectors are approximately parallel to the direction of gravity (such as the ground and tabletops).
- Plane Tracking and Detection
EasyAR verifies and updates the detected planes in consecutive frames:
- Determine whether the newly observed three - dimensional points support the existing plane model;
- Dynamically adjust the plane range, boundary, and confidence based on the observation consistency;
- Eliminate plane candidates that appear briefly or are unstable. Only when both geometric consistency and temporal stability meet the requirements will the result be considered an "available plane".
- Plane Coordinate System and Virtual Content Alignment
Once a plane is confirmed, you can achieve a more realistic AR effect based on the plane detection results:
- Place virtual objects on the plane to achieve alignment in real scale and direction;
- Perform a hit test to map screen clicks to real plane positions;
- Implement plane - based interaction logic, such as object adsorption, movement, and occlusion judgment.
Since the plane and the motion tracking system share the same world coordinate system, virtual objects can maintain stable and continuous spatial consistency when the user moves the device.
Plane detection relies on the stable pose and spatial structure provided by motion tracking, and the plane detection results can in turn be used to enhance the environmental understanding ability, such as assisting in content placement and interaction design. The two together form the core foundation of EasyAR's spatial perception ability, but they are decoupled from each other in the system architecture. Plane detection will not change the pose estimation results of motion tracking itself.
Best Practices
To ensure the effectiveness of users' use of plane detection, following the following practices can improve the user experience.
- Guide users to move slowly, avoiding standing still, moving quickly, or rotating in place.
- Avoid planes that are difficult to visually identify, such as those without texture, solid - colored, or mirrored.
Note
Plane detection is a function of EasyAR to identify horizontal planes in the environment. Surface tracking does not detect or identify the plane structure in the scene, and a distinction needs to be made.