CS 365: Assignment #5

Assignment 5: Object Recognition Using a Trained Classifer

Due 19 April 2016 (2 weeks)


This project is about learning how to train a classifer for object recogntion. The first task will be to train a Haar cascade classifier to recognize Wendy the Badger given a set of provided training images and then write a program to use the classifier. The second task will be to select an object of your choice, a classifier of your choice, and train it to recognize the object.


Download the training data tar file, which contains images of Wendy selected from the ImageDB we used for project and a file specifying crops from those images that show the front of the doll. You may also want the background image list which contains a listing of all the files in the ImageDB from project 2.

You may also want to read through the Cascade Classifier Training Guide to understand the steps in the process.

Since Wendy is a more or less rigid object, we do not need a large number of positive training samples. However, almost all the training samples are frontal with a small amount of angular variation. If you want the classifier to be able to recognize Wendy at a variety of orientations, then you will need to generate more training data.

As this is the last defined project, you may want to think about this project as a lead-in to your final project.


  1. The first step is to organize all of the necessary information and training samples. You need the positive traning samples (the pictures in wendy.tgz), the annotated list of positive images (wendy-trainingSet-W-small.txt), the negative training image list (bg.txt), and the negative training samples (the ImageDB from project 2). The pathnames in the bg.txt file need to correct, given where you have your ImageDB.

    One helpful tool that will let you avoid having multiple copies of the ImageDB is the unix tool ln. The command:

    ln -s <pathname to ImageDB> ./ImageDB

    creates a link from wherever the ImageDB is located to your current directory. That way it's as if the ImageDB directory is located in your working directory, even if it is not.

    You can set things up however you like, the following is one possibility.

    |        |        |             |
    wendy    bg.txt   ImageDB link  wendy-trainingSet-W-small.txt
    positive training jpgs

    If your data is set up as above, you will want to run the opencv_createsamples and opencv_traincascade commands from the Data directory.

  2. Generate the vector file of training images using opencv_createsamples. The relevant flags are as follows.
    • -info wendy-trainingSet-W-small.txt
    • -vec wendy-small.vec
    • -w 27
    • -h 30

    Note that the last two arguments are the size of the image window used for classification. The crop sizes selected in the annotated training sample list fit this ratio of 9 to 10.

    This should create the output file wendy-small.vec. You can look at the training crops using the command

    opencv_createsamples -vec wendy-small.vec -w 27 -h 30

  3. Train a Haar cascade classifier using opencv_traincascade. The relevant parameters are the following (given the directory structure above).
    • -data wendy
    • -vec wendy-small.vec
    • -bg bg.txt
    • -numPos 19
    • -numNeg 100
    • -featureType Haar
    • -w 27
    • -h 30

    The output classifier should end up in the wendy subdirectory called cascade.xml

  4. Write a program that makes use of the detectMultiScale function and the trained cascade classifier to read in a static image, detect if Wendy is in the image, and identify the bounding box if the detection is positive. Have your program take in one command line argument, which is a file with the paths and names of the images to test. Then test your program on these images.
  5. Extend your program from step 4 to take both a file with the positive test examples and a file with negative examples (no Wendy). You can, for example, use the bg.txt file for the negative examples. Have your program compute true detections, false detections, false negatives, and true negatives and report the results.
  6. Write a new program to detect Wendy in a live video stream.
  7. Pick an object of your choice, preferably a rigid object. Make a training set for the object. Pick a classifier type that you can train. [Note: if you choose to do a cascade classifier, train up two classifiers, one the Haar features and one using Local Binary Pattern [LBP] features.] Train the classifier on your object. You can train it on only an upright frontal pose. Adding more variation in detectable pose and orientation is an extension.
  8. As with the Wendy doll, create a positive test set, use the ImageDB as a negative set and compute the confusion matrix (true positives, false positives, false negatives, and true negatives). Report your results. Make sure the positive test set is different than the positive training set.
  9. Extend your program from step 6 to be able to detect your object in a live video stream.



For this project, make a wiki page that begins by explaining your overall pipeline for OR. Your audience is your fellow CS majors not in the course.

Your audience for this section is other students in the course. Provide some kind of proof that the system is providing useful/reasonable information.

Your project report should include images from each stage: one or two training crops and images showing the recognition working.

If you did any extensions, describe the algorithms and show at least one example for each extension.

Give your wiki page the label: cs365s16project05


Put a zip file or tar file of your code in your Private Courses handin directory.