Combined use with planar image tracking

Cloud image recognition can also be used together with planar image tracking to implement a hybrid "recognition + continuous tracking" mode. This article introduces how to use it and analyzes its advantages and applicable scenarios.

Workflow

The core of the hybrid mode is the seamless connection between cloud recognition and local tracking. The workflow is as follows:

Cloud recognition phase

1. Send request: The device camera captures the current image and uploads it to the CRS server.
2. Cloud matching: CRS searches the target library and returns the matched target ID and image data (Base64 encoded).
3. Receive result: The client receives the recognition result and triggers subsequent processing logic.

Local tracking phase

1. Image decoding: The client decodes the Base64 data into an image and generates an ImageTarget instance locally based on it.
2. Initialize tracking: Initialize ImageTracker and call the loadTarget method to start planar image tracking.
3. Continuous tracking: The device computes the 6DoF pose locally, and virtual content follows the image in real time.

Advantages of combined use

Compared with using cloud recognition alone, the combined mode performs better in the following aspects:

• Reduce the probability of false recognition
  When cloud recognition is used alone, if there are similar images in the target library, an incorrect target may be returned. With local image tracking, local tracking continuously verifies image features. If the actual image content does not match the recognition result, tracking is quickly lost and re-recognition is triggered. Therefore, combined use can greatly reduce the false recognition rate of cloud recognition.

• Support continuous tracking and interaction
  Cloud recognition used alone can only return the target ID and cannot support continuous interactions such as rotation and scaling. In hybrid mode, recognition immediately switches to local tracking, supporting real-time 6DoF pose updates. Users can move the device or image, and virtual content always follows, making it suitable for AR games, product display, and other scenarios.

• Reduce cloud load
  Frequent cloud recognition calls, such as once per second, increase server pressure and latency. In hybrid mode, after recognition succeeds, subsequent tracking is completed locally on the device without continuously uploading images. Cloud recognition is triggered again only when tracking is lost, which can greatly reduce cloud requests and client network traffic.

• Adaptability in weak network environments
  Cloud recognition alone is prone to timeout or failure when the network is unstable. In hybrid mode, once recognition succeeds, local tracking can continue even if the network is disconnected. It can be combined with a local target library to provide a degraded experience before the network recovers.

Best practices

When choosing whether to use cloud recognition, planar image tracking, or hybrid mode, evaluate the following dimensions:

How to choose a feature

Application characteristics	Recommended solution	Reason
Number of targets < 100	Planar image tracking	Sufficient local memory and no network dependency
No network or unstable network	Planar image tracking	Avoid recognition failure and ensure offline availability
Targets need real-time updates	Cloud recognition	Takes effect immediately after upload, suitable for dynamic content
Limited device performance	Cloud recognition	Embedded devices or extreme power consumption requirements
No continuous tracking required	Cloud recognition	Such as one-time scan recognition, with no need for tracking

When to choose hybrid mode

• Large number of targets (>100): The cloud has unlimited storage, and only the current target is loaded locally, saving memory.
• Continuous interaction required: Such as AR education (rotating a 3D model after recognizing teaching materials) and AR marketing (viewing a 3D presentation after recognizing a product).
• Sensitive to false recognition: Such as medical and industrial scenarios where recognition accuracy must be ensured.
• Degradation required in weak network environments: After successful recognition, tracking can continue even when the network is disconnected.

Summary and extension

The hybrid mode of cloud recognition and planar image tracking combines the large capacity of the cloud with the continuous tracking capability of local tracking, and is especially suitable for complex scenarios that require high accuracy and interactivity. Developers should flexibly choose standalone features or hybrid mode according to the number of targets, update frequency, network environment, and interaction requirements.