Overview

This assignment builds upon the previous assignment in which you constructed ASEF filters from video frames. You are to extend that code in this assignment to replicate a basic moving object tracking algorithm based upon David Bolme's correlation filter construction technique.

A key part of this assignment is a live demonstration using video from a laptop camera or other suitable live video source available to you. I will speak to members of each team to make sure that between them they have access to a suitable hardware platform.

When completed, the program being written should draw a box around an object held up in front of the camera and then when the object moves the box should stay centered on the object.

Initialization

You have some choices as to how you wish to initialize tracking. Here are some suggestions. First, implement a basic motion detector and initialize your tracker whenever an object moves very quickly. You can experiment with what 'very quickly' means. Perhaps you rapidly shake the object back and forth. If that approach does not interest you, for this assignment you might constrain yourelf to considering distinctly colored objects and then detecting these colors in several frames with the system is first started and using that information to initialize your tracking window.

Tracking

It is suggested you do not change the size of the tracking window after initialization. That will simplify the task. What your tracker will do is use a develping correlation filter to predict the location of the object being tracked in the next frame of live video.

You will face a question of how quickly to adapt the filter over time. Since the approach to building correlation filters uses multiple frames, you will want to keep some history, i.e. set of past frames and estimated target locations. Exactly how many frames to store in history and how to weight the current versus past frames is an aspect of the tracker you will need to explore emprically.

Whether you want to stick with the ASEF approach or upgrade to the MOSSE strategy is up to you.

What to show

At a bare minimum you need to show each successive frame of video with the corrent estiamted location of the obejct being tracked indicated. Most likely this indication will be a colored bounding box. In addition, it might prove helpful to show additional information such as the current correlation filter and the correlation surface with the next frame. For an example, considder the YouTube video David Bolme prepared and which we reviewed in lecture.

Teams and Grading

You are expected to work in pairs - teams - for this project. Unless worked out between yourselves and approved by me in advance, I strongly suggest you keep your partner from Assignment 3. The grading will be done in the same fashion as for Assignment 3, in face-to-face interviews with the instructor.