Table of Contents

Introduction to 3D object tracking

Besides planar objects, EasyAR also supports tracking 3D objects. This article introduces the core principles, expected outcomes, and potential limitations of 3D object tracking to help developers understand applicable scenarios.

Basic principles

3D object tracking recognizes and tracks three-dimensional objects in the real world (e.g., toys, sculptures, industrial parts) to enable AR content overlay. The core workflow is as follows:

Technical workflow

  1. Model preparation: Developers provide the 3D model file (Wavefront OBJ format) of the target object. The system loads the model locally, extracts visual features from different perspectives, and generates a unique feature database.
  2. Real-time matching: After the camera captures the real-world scene, the system analyzes visual features frame-by-frame and matches them against the pre-stored model feature database.
  3. Pose calculation: The object's 6DoF pose in 3D space is calculated through matched points, ensuring precise alignment between virtual content and the object.
  4. Continuous tracking: Tracking persists even if the object is partially occluded or moved, leveraging remaining visible feature points.

Core mechanisms

  • Local processing: All computations occur on-device. Models and content are managed by developers, ensuring offline availability.
  • Texture dependency: Objects require rich surface textures or geometric details. Solid-color or smooth surfaces (e.g., glass, metal) are difficult to track.
  • Model format: Only OBJ format is supported, requiring MTL material files and at least one texture map (JPEG/PNG). File paths must be relative (absolute paths/spaces prohibited).

Technical limitations

  • Object type: Only rigid objects (non-deformable) with primarily convex geometry (minimal cavities) are supported.
  • Size range: Object size should be between 10cm and 2m. Smaller/larger sizes affect visibility at natural viewing distances.
  • Environment requirements: Dependent on lighting conditions—low light or overexposure may cause detection failure or tracking loss.

Outcomes and expected results

Object tracking relies on the object's inherent textures for visual feature matching, leading to inherent limitations. Understanding these outcomes helps set realistic testing standards during development.

Ideal outcomes

  • Precise alignment: Virtual content aligns accurately with the 3D object's edges.
  • Real-time response: Ultra-low latency from model loading to successful detection.
  • Occlusion resistance: Tracking persists when the object is partially occluded (e.g., 30%).
  • Multi-angle support: Virtual content follows consistently during 360° rotation or from different viewpoints.

Non-ideal scenarios and solutions

Phenomenon Cause User perception Solution preview (details in later chapters)
Detection failure Insufficient model texture, incorrect path, non-UTF-8 encoding Virtual content absent Optimize textures; verify file format/path
Tracking jitter Rapid lighting changes, reflective surfaces Visible virtual object shaking Control lighting; avoid reflective surfaces
Frequent loss Fast object movement or full occlusion Virtual object flickering/disappearing Increase model size or combine with motion fusion
Loading failure OBJ/MTL format errors, missing texture maps Tracker initialization fails Validate model specifications; reference official samples

Verification methods

  • Development phase: Use the HelloARObjectTracking sample with official test models (and physical objects) to validate core functionality.
  • Testing phase: Test tracking stability under real-world conditions (varying lighting, angles, occlusion).

Summary and extensions

3D object tracking uses locally processed visual features of models, making it suitable for offline AR applications targeting non-planar objects. Developers must ensure model format compliance and prioritize environment/texture quality. Next chapter details model preparation and optimization techniques.