Table of Contents

Introduction to 3D object tracking

In addition to planar objects, EasyAR also provides tracking capabilities for 3D objects. This article introduces the core principles, expected effects, and potential limitations of 3D object tracking, helping developers understand the applicable scenarios of this technology.

Basic principles

3D object tracking enables AR content overlay by identifying and tracking three-dimensional objects in the real world (such as toys, sculptures, industrial parts). The core workflow is as follows:

Technical process

  1. Model preparation: Developers provide the 3D model file (Wavefront OBJ format) of the object to be tracked. The system loads the model locally and extracts visual features from different perspectives of the model to generate a unique feature library.
  2. Real-time matching: After the camera captures the real-world scene, the system analyzes the visual features in the frame-by-frame images and matches them with the pre-stored model feature library.
  3. Pose calculation: The 6DoF pose of the object in 3D space is calculated through matching points, driving the virtual content to align precisely with the object.
  4. Continuous tracking: Even if the object is partially occluded or moved, the system can still maintain tracking through the remaining visible feature points.

Core mechanism

  • Local processing: All computations are performed on the device, with models and content managed by the developer, ensuring offline availability.
  • Texture dependency: Object surfaces require rich textures or geometric details, as solid colors or smooth surfaces (e.g., glass, metal) are difficult to track.
  • Model format: Only OBJ format is supported, which must include an MTL material file and at least one texture map (JPEG/PNG). File paths must be relative (absolute paths or spaces are prohibited).

Technical limitations

  • Object type: Only rigid objects (non-deformable) are supported, with predominantly convex geometry (no extensive hollow structures).
  • Size range: Recommended object size is between 10cm and 2m. Objects too small or too large may affect visibility at natural viewing distances.
  • Environmental requirements: Dependent on lighting conditions. Excessive darkness or overexposure may cause detection difficulties or tracking loss.

Effects and expected results

Object tracking relies on the texture of the object itself for visual feature matching, so there are some limitations to the achievable effects. Understanding these effects helps you set reasonable testing standards during development.

Ideal effect

  • Precise binding: Virtual content aligns with the 3D object edges.
  • Real-time response: Ultra-low latency from model loading to successful detection.
  • Occlusion resistance: Maintains tracking even when the object is partially occluded (e.g., 30%).
  • Multi-angle support: Virtual content continuously follows the object when rotated 360° or viewed from different perspectives.

Non-ideal situations and countermeasures

Phenomenon Cause User perception Solution preview (see details in later sections)
Recognition failure Insufficient model texture, incorrect path, non-UTF-8 file encoding Virtual content does not appear Optimize model texture, check file format and path
Tracking jitter Drastic lighting changes, reflective object surfaces Significant shaking of virtual objects Control lighting, avoid reflective surfaces
Frequent loss Fast-moving or fully occluded objects Virtual objects flicker/disappear Increase model size or combine with motion fusion
Loading failure OBJ/MTL file format errors, missing texture maps Tracker initialization fails Verify model specifications, refer to official samples

Expected result verification method

  • Development phase: Use the HelloARObjectTracking sample, import the official test model (and create entities) to verify basic functionality.
  • Test phase: Test tracking stability under different lighting, angles, and occlusion conditions in real-world environments.

Summary and extension

3D object tracking is achieved by locally processing the visual features of the model, making it suitable for offline AR applications targeting non-planar objects. Developers need to ensure the model complies with format specifications and pay attention to the quality of the environment and object textures. Next chapter will detail model preparation and optimization techniques.