Mixed usage with image tracking

Cloud image recognition can also be combined with image tracking to achieve a hybrid mode of "recognition + continuous tracking". This article will introduce how to use it and analyze its advantages and applicable scenarios.

Workflow

The core of the hybrid mode is the seamless connection between cloud recognition and local tracking, with the following process:

Cloud recognition phase

1. Send request: The device camera captures the current frame and uploads the image to the CRS server.
2. Cloud matching: CRS searches the target library and returns the ID and image data (Base64-encoded) of the matched target.
3. Result reception: The client receives the recognition result and triggers subsequent processing logic.

Local tracking phase

1. Image decoding: The client decodes the Base64 data into an image and generates a local ImageTarget instance based on it.
2. Initialize tracking: Initialize ImageTracker and call the loadTarget method to start image tracking.
3. Continuous tracking: The device locally calculates the 6DoF pose, and virtual content follows the image movement in real-time.

Advantages of hybrid usage

Compared to using cloud recognition alone, the hybrid mode performs better in the following aspects:

• Reduced false recognition probability
  When using cloud recognition alone, similar images in the target library may cause incorrect matches. With local tracking, the local tracker continuously verifies image features. If the actual content doesn’t match the recognition result, tracking quickly fails and triggers re-identification. Thus, hybrid usage significantly reduces false recognition rates.

• Supports continuous tracking and interaction
  Cloud recognition alone only returns the target ID and cannot support continuous interactions like rotation or scaling. In hybrid mode, recognition immediately switches to local tracking, supporting real-time 6DoF pose updates. Users can move the device or the image while virtual content remains anchored, ideal for AR games or product demonstrations.

• Reduced cloud load
  Frequent cloud recognition requests (e.g., once per second) increase server pressure and latency. In hybrid mode, after successful recognition, tracking is handled locally without continuous image uploads. Cloud recognition is only re-triggered when tracking is lost, drastically reducing cloud requests and client network consumption.

• Adaptability to weak network environments
  Cloud recognition alone is prone to timeouts or failures in unstable networks. In hybrid mode, once recognition succeeds, local tracking continues working even if the network disconnects. Combined with a local target library, this provides a degraded experience before network recovery.

Best practices

When deciding whether to use cloud recognition, image tracking, or hybrid mode, evaluate based on the following dimensions:

How to choose features

Application characteristics	Recommended solution	Reasons
Targets < 100	Image tracking	Sufficient local memory, no network dependency
No network or unstable network	Image tracking	Avoid recognition failures, ensure offline availability
Targets require real-time updates	Cloud recognition	Takes effect immediately after upload, suitable for dynamic content
Limited device performance	Cloud recognition	Embedded devices or extreme power constraints
No continuous tracking needed	Cloud recognition	E.g., one-time scan recognition without tracking

When to choose hybrid mode

• Large target library (>100): Unlimited cloud storage, local loading of only current targets saves memory.
• Continuous interaction required: E.g., AR education (rotating 3D models after recognizing textbooks) or AR marketing (viewing 3D demos after product recognition).
• High sensitivity to false recognition: Scenarios like medical or industrial fields requiring guaranteed accuracy.
• Degraded experience in weak networks: Tracking continues working after successful recognition even if the network disconnects.

Summary and extension

The hybrid mode combining cloud recognition and image tracking leverages the cloud’s large capacity and local continuous tracking capabilities, especially suited for complex scenarios requiring high accuracy and interactivity. Developers should flexibly choose standalone features or hybrid mode based on target quantity, update frequency, network environment, and interaction needs.