Diagnose and fix content jitter and drift in an application

"Virtual content is drifting", "objects are shaking", and "positions are unstable" are common issues developers encounter in AR applications. Unstable content can seriously break immersion and even lead to a poor user experience.

This article helps you understand why content jitters or drifts, and provides a systematic troubleshooting and optimization process.

Distinguish "normal jitter" from "abnormal drift"

First, establish reasonable expectations. Achieving high-precision AR tracking on mobile devices is challenging by nature. The following cases are within the normal range. They cannot be completely eliminated, but they can be optimized:

• Tiny high-frequency jitter:

  • Symptom: The virtual object has subtle millimeter-level shaking.
  • Cause: This is caused by physical noise in device sensors, precision limits of visual tracking algorithms, and tiny hand movements while holding the device.
  • Example: When observing at close range, such as placing a virtual object on a table and moving closer to inspect it, this slight jitter is normal.

• Brief drift:

  • Symptom: When the user moves or rotates the device quickly, the virtual object has a position offset for a short time (0.5-1 second), and then recovers.
  • Cause: During fast motion, accumulated IMU (inertial measurement unit) error and visual localization latency in the device SLAM system can cause brief position drift.
  • Example: This is acceptable in dynamic scenes. If the object can be quickly "pulled back" to the correct position, the system is working effectively.

The following cases are abnormal issues that need troubleshooting and fixing:

• Persistent, large position drift: The virtual object slowly and continuously moves away from where it should be, and does not recover, or only recovers after a long time.
• Severe jumping or flickering: The virtual object jumps significantly on the screen, or appears and disappears.
• Inconsistent relative position to real objects: The virtual object cannot be stably "pinned" to a real object.

Note

There is also an important point to note:
For devices running in 0DoF, 3DoF, or 5DoF modes, the sense of fit and realism is inherently weaker than on 6DoF devices. When users move quickly, turn, or go up and down stairs, virtual objects cannot follow perfectly.
Therefore, in these modes, phenomena such as content "floating in the air" or "position offset" are fundamental limitations of the device capability, rather than the "content jitter or drift" fault discussed in this article.
For the experience differences between different xDoF modes, see the introduction in Navigation best practices.

Systematic troubleshooting process

Troubleshoot in the following order, starting with the most likely causes.

Step 1: External environment and hardware factors (no code changes required)

1. Physical environment check:

   • Texture richness
     Is your test environment too monotonous? Large plain white walls, smooth floors, and glass surfaces can all cause visual localization to fail or produce errors.
   • Dynamic objects
     Are there many moving objects in the environment, such as crowds or moving vehicles? Dynamic objects can interfere with visual localization, but this issue is usually temporary.
   • Scene confusion
     Are there highly confusable areas in the environment, such as elevator lobbies at different entrances? Visually similar areas can affect localization, causing localization results to jump back and forth between similar areas. This type of issue can be avoided by setting appropriate prior information in advance.

2. Device hardware check:

   • Device heat
     After running for a long time, does the device become seriously hot? Overheating can cause CPU/GPU throttling and affect the tracking performance of the device's own SLAM system. This is the most common cause of persistent drift.
   • Device performance
     Some older devices are limited by hardware performance and component precision, making scale drift more likely and causing virtual content to drift with it. You can try comparing with another device to help determine whether the issue is caused by limitations of the device itself.

Step 2: Map and localization quality analysis (using external tools)

1. Use Mega Toolbox:

   • Run Mega Toolbox at the same location and observe the stability of its localization.
   • If Toolbox localization also drifts or jumps: The issue is with the map itself or the current environment is not suitable for localization.
   • If Toolbox localization is stable: The issue is in your application. Continue to step 3.

2. Run EIF data in PC simulation:

   • Replay the EIF data recorded on site.
   • If replay also drifts or jumps: This means the scene itself is not suitable for localization, the map itself has issues, or the device that recorded the EIF has scale drift in its motion tracking.
   • If replay is stable: This means the scene itself is localization-friendly, and the issue may come from factors during real-time running in your application, such as device heat and throttling.

Step 3: Internal application logic check

1. Pose update:

   • Check whether you have applied unnecessary extra smoothing to pose data, such as excessive Lerp or SmoothDamp, which can instead introduce latency and a sense of drift.
   • In most cases, directly using the original Pose returned by Mega is the most stable approach.

2. Coordinate system matching:

   • Confirm that the node relationships between your virtual objects, scene camera, MegaTracker, and similar objects are correct, and that the local transform values of nodes under MegaBlocks have not been modified.
   • Incorrect node setup can lead to incorrect coordinate system transformations, causing unpredictable content rendering behavior.

Special reminder: visual overlay issues on OST headset devices

After checking localization and rendering logic, if you are using an OST (optical see-through) headset device, you also need to consider a special category of issues.

Even if the device itself has good 6DoF motion tracking capability, you may still encounter a poor sense of fit when virtual objects are overlaid on the physical space. This is usually not a fault in the Mega localization service, but an inherent phenomenon caused by the optical principles of OST devices, such as optical alignment errors or differences in eye calibration.

For detailed explanations and judgment methods for this type of issue, see Special notes for OST devices.

Summary and best practices

After the troubleshooting above, you should have identified the root cause of content jitter or drift. To help you quickly review and take action, the following table summarizes common symptoms, possible causes, and best practices. Find the corresponding solution in the table based on your troubleshooting result.

Issue type	Possible cause	Best practice
Tiny jitter	Sensor noise, algorithm limits	This type of tiny jitter is normal and usually does not require much attention
Drift after fast movement	SLAM latency, algorithm correction	Guide users to move the device smoothly. If it cannot recover quickly, pay extra attention
Persistent large drift	SLAM fault, device differences	Use another device for cross-validation
Severe jumping/flickering	Confusing scene, localization-unfriendly environment	Set auxiliary prior information or guide users
Inconsistent relative position to real objects	Localization/map error, code logic error	Test from multiple viewpoints, observe the virtual object's position, and fix possible code errors

If your issue still exists after the troubleshooting and fixes above, submit a detailed report through Issue reporting together with screen recordings, EIF data recordings, and detailed logs.