Real-Time Tracking of a Minecraft Torch - Computer Vision Exercise

Project Aim:

My goal for the project was to track a real-life object using a matching CAD model. To do this, I used VisionLib integrated with Unity.

Full Video (YouTube)

Expected Outcome:

The input to the system is a live video feed from my cellphone. The video feed features a real-life object that I tracked--a Minecraft torch. The output of the project is a hologram in augmented reality of the torch superimposed over my real-life torch.

Evaluation Criteria:

The resulting live object tracking video would ideally have an intersection over union (IoU) >0.8 (1.0 being perfect). Generally, if the hologram can correctly stick to the real model most of the time, that is a success.

Results:

The Minecraft torch I chose as my model turned out not to be ideal for object tracking. The IoU was unacceptable as shown in the video linked above. The Minecraft torch is similar on all sides, so that is the reason the algorithm had trouble tracking where to map the hologram in augmented reality.

The original paper car model was tracked easily.

Biggest Issue in Development:

I kept running into an error -> VL_ERROR_MODEL_DECODE_FAILED indicating that the file URI could not be decoded. I reread the setup guide and added the text highlighted below:

{
    "type": "VisionLibTrackerConfig",
    "version": 1,
    "meta": {
        "name": "Simple VisionLib MC Torch Tracker",
        "description": "Simple model tracking configuration file",
        "author": "Katie Gonsoulin"
    },
    "tracker": {
        "type": "modelTracker",
        "version": 1,
      "parameters": {
        "modelURI": "project-dir:Torch.obj",
        "useColor": true,
        "metric": "m",
        "showLineModel": true,
        "keyFrameDistance": 5,
        "laplaceThreshold": 1,
        "normalThreshold": 0.3
      }
    }
}

Takeaways:

This project helped me get more familiar with Unity, computer vision, and augmented reality. Real-time model tracking is being implemented in many industries for training, advertising, and entertainment. I anticipate its use will only grow with time.