Mixed usage with planar image tracking

Image cloud recognition can also be combined with planar image tracking to achieve a hybrid mode of "recognition + continuous tracking." This article will introduce how to use it and analyze its advantages and applicable scenarios.

Workflow

The core of hybrid mode is the seamless integration of cloud recognition and local tracking, with the following process:

Cloud recognition phase

1. Send request: The device camera captures the current frame and uploads the image to the CRS server.
2. Cloud matching: CRS searches in the target library and returns the matched target ID and image data (Base64 encoded).
3. Receive result: The client receives the recognition result and triggers subsequent processing logic.

Local tracking phase

1. Image decoding: The client decodes the Base64 data into an image and generates an ImageTarget instance locally based on it.
2. Initialize tracking: Initialize ImageTracker and call the loadTarget method to start planar image tracking.
3. Continuous tracking: The device locally calculates the 6DoF pose, and the virtual content follows the image movement in real-time.

Benefits of hybrid usage

Compared to using cloud recognition alone, the hybrid mode performs better in the following aspects:

• Reduced misrecognition probability
  When using cloud recognition alone, if there are similar images in the target library, it may return incorrect results. With the addition of local image tracking, the local tracker continuously verifies image features. If the actual content of the image does not match the recognition result, the tracking will quickly fail, triggering re-recognition. Therefore, hybrid usage can significantly reduce the misrecognition rate of cloud recognition.

• Supports continuous tracking and interaction
  Using cloud recognition alone only returns the target ID and cannot support continuous interactions such as rotation or scaling. In hybrid mode, after recognition, it immediately switches to local tracking, supporting 6DoF real-time pose updates. Users can move the device or the image, and the virtual content will always follow, making it suitable for scenarios like AR games or product demonstrations.

• Reduced cloud load
  Frequent cloud recognition calls (e.g., once per second) increase server pressure and latency. In hybrid mode, after successful recognition, subsequent tracking is handled locally by the device, eliminating the need for continuous image uploads. Cloud recognition is only re-triggered when tracking is lost, significantly reducing cloud requests and lowering the client's network traffic consumption.

• Adaptability to weak network conditions
  Using cloud recognition alone is prone to timeouts or failures in unstable network conditions. In hybrid mode, once recognition is successful, local tracking can continue to work even if the network is disconnected. Combined with a local target library, it provides a degraded experience for applications before the network recovers.

Best practices

When choosing whether to use cloud recognition, planar image tracking, or hybrid mode, you can evaluate based on the following dimensions:

How to choose features

Application characteristics	Recommended solution	Reason
Number of targets < 100	Planar image tracking	Sufficient local memory, no network dependency
No network or unstable network	Planar image tracking	Avoid recognition failure, ensure offline availability
Targets need real-time updates	Cloud recognition	Takes effect immediately after upload, suitable for dynamic content
Limited device performance	Cloud recognition	Embedded devices or extreme power requirements
No continuous tracking requirement	Cloud recognition	For one-time scan recognition, no tracking needed

When to choose hybrid mode

• Large number of targets (>100): Cloud storage is unlimited, while only the current target is loaded locally, saving memory.
• Continuous interaction required: Such as AR education (rotating 3D models after recognizing textbooks) or AR marketing (viewing 3D demos after identifying products).
• Sensitive to misidentification: Such as in medical or industrial scenarios, where recognition accuracy must be ensured.
• Downgrade needed in weak network environments: Tracking can continue even if the network disconnects after successful recognition.

Summary and extension

The hybrid mode of cloud recognition and planar image tracking combines the large capacity of the cloud with the continuous tracking capability of local devices, making it particularly suitable for complex scenarios requiring high accuracy and interactivity. Developers should flexibly choose between standalone features or hybrid mode based on the number of targets, update frequency, network environment, and interaction requirements.