Assignment 03 - Features and Person Matching, Due May 8
Overview
The purpose of this assignment is to give you experience using feature matching in OpenCV. In particular, in this assignment you will be matching short sequences of video taken from the 32 PaSC videos with which you have already workedin Assignment 2.
As a consequence of using timely and difficult data, do not have excessively high hopes for the quality of the results. Few people have so far tried to do what you are doing on this small sobuset of PaSC videos. To create a two tiered level of difficult in this assignment, and also to bound the computational burden implied by this project, instead of matching whole videos to whole videos you will instead match 10 frame snippets to 10 frame snippets. Further, there will be two snippets drawn from each video.
As a consequence of this formulation, you are both testing how well snippets seperated by only a few seconds match and also how well snippets on different days and/or with different people match. By way of prediction, hopefully snippets drawn from the same video will match better than others some reasonable fraction of the time. The harder case, expecting a pair of snippets of the same person from different days and locations to match better than pairs for all combinations of different people may be asking two much of the features being used: only through doing this assignment will you find out.
What You are Being Given
In this assignment you are being given individual frames as still images for each of the 32 videos. You are also being provided the motion masks introduced in Assignment 2. You will implicitly or explictly create 64 video snippets consisting of 10 frames starting at frame 60 and 120 in each of the 32 videos.
For the sake of consistency, when describing these snippets use the original video names with "A" or "B" added to the end. Note you are being given all frames just in case you find yourself wanting to experiment with larger snippets or snippets taken from different points in the video. Note that by the instructions you are being given, there will be roughly 2 seconds elapsed between snippets A and B on any given video.
As with Assignment 2, the data is only readable on our unix machines
by accounts in the computer vision group. All students in this class
are now members of this unix group. Also, as we have discussed in
class, this data is provided to us under license and by using it you
are taking responsibility for seeing to it that access to this data
is restricted to members of the vision group. Under no circumstances
may you repost or redistribute this data. The path to the data is:
One last thing, beware the off by one error! The naming convention for the mask file frames versus the video file frames is not consistent; mask zero goes with frame 1.
Your Tasks
Here the major portions of this assignment are broken out into tasks and sub-tasks:Task 0: Infrastructure
You must write code using the OpenCV SURF features and FLANN matching algorithm as a starting point for computing a single number measuring the distance between a pair of video snippets. Here are some of the degrees of freedom available to you as you consider how to generate this pairwise distances between video snippets. Probably the most obvious thing to compare all 10 videos in one snippet to all 10 in another and then compute the averate distance between the top K matching features in all 100 pairwise comparisons. You will write this code and the results will form the basis for Task 1.
You will also solve the problem of generation Cumulative Match Characterist (CMC) curves from your distance matrices. We will discuss CMC curves in class. Further, two variants on the CMC will be generated, one included the same video snippet and the other excluding the same video snippet.
In case it is not yet obvious, you should only be comparing features compatible with the motion masks. Otherwise you will be essentially matching the room backgrounds and your results will be meaningless.
Task 1: Establish a Baseline
Generate a 64 by 64 distance matrix recording the distance between all pairs of video snippets. You must enumerate snippets in lexigraphic order relative to the snippet names. Note this matrix is symmetric with zeroes down the diagonal.
Task 2: Make it Better
There are a whole host of ways to compute a snippet to snippet distance after running pairwise frame comparisons. In Task 2 experiment with different options and also different ways of using the motion mask. You should consider at least 3 variants from the baseline in Task 2.
Task 3: Submitting Results
For this assignment you will both submit a report along the lines of what was required for Assignment 2. You will also submit your best distance matrix as a CSV file.
Submission and Grading
Please be sure your English report is emailed to the instructor before midnight on the due date. Also through email send the CSV file. Grading will be based upon the substance of the work and clarity of the writeup, with the caveat that the substance cannot be graded outside the context of the report. In other words, no report, no credit.
Addendum
No Addendums at this time - April 17 2015
