OpenCV: Haartraining and all that jazz!

Object detection in OpenCV can be done through Haartraining. OpenCV haartraining applications provide the ability to detect objects of interest to us like faces, eyes, moving cars etc. What is required is that we need to identify positive and negative samples and use the utilities that OpenCV provides us to train the application to be able to recognize the objects we want. The positive and negative samples are used to create classifiers which are then utilized to detect the objects that we intend to.

Fortunately the tar file (OpenCV-2.3.1a.tar.bz2) which can be downloaded from the OpenCV site already comes bundled with all the necessary utilities to create a fully trained haar classifier. We just to have to build and run the commands to create our our classifiers.

This post will look at the necessary steps for creating a haarclassifier. To get started with OpenCV please look at my earlier post Computer Vision: Getting started with OpenCV.

Once you have installed OpenCV look under modules/haartraining. All the necessary files are included.

There are 3 steps in this process

1) Create samples   2) Train and create a classified haar 3) Performance testing

 Create Samples (createsamples.cpp) : This is the first utility that has to be executed. This utility takes as input positive samples (images of the object that we are interested in) and negative samples (images that do not include the object that we are interested in). The createsamples utility superimposes the objects that we want to recognize in various degrees of rotation against the background of the negative samples. The composite images file is then used to train the haar application.

The first step is to build createsamples.cpp. Make sure you include ../OpenCV-2.3./modules/haartraining. Build the following files

(createsamples.cpp, cvboost.cpp, cvhaarclassifier.cpp, cvhaartarining.cpp, cvsamples.cpp, cvcommon.cpp)

Once createsamples.cpp successfully builds we can create samples required  for the training.

The command is

$./myhaartraining -img logo.png -vec samples1.jpg -bg bg.txt  -w 20 -h 20

Where I have chosen the OpenCV logo as my object of interest.

The samples are created in samples1.jpg

The bg.txt contains a list of the negative samples included as below

./img/airplane.jpg

./img/baboon.jpg

./img/kid.jpg

./img/lena.jpg

-w stands for the width and -h stands for the height of the samples/

This will create superimposed positive samples in the negative sample background. A few of these a are shown below

Make sure that the width and height are small i.e < 50 otherwise the haartraining application core dumps because of lack of memory.

To check if the samples are created properly run a test round as follows run the command with the following options

$./myhaartraining -img logo.png -vec samples1.jpg -bg bg.txt -n 10 – show  -w 20 -h 20

where -n is the number of samples to be generated (default is 1000) -show will show a series of images with the positive samples superimposed on the negative samples. This can be used to check if createsamples utility is working properly. For a more thorough and detailed explanation see my post Hand detection through Haartraining: A hands-on approach

Haartrainer (haartrainer.cpp): This utility takes as input the samples from the createsamples utility and creates a trained haar classifier. To build the haartrained files, build the haartraining.cpp. As before make sure you include the appropriate files along with the all the opencv libraries. Buid the following files

(haartraining.cpp, cvboost.cpp, cvhaarclassifier.cpp, cvhaartarining.cpp, cvsamples.cpp, cvcommon.cpp)

The command to use is

./haartrainer -data test2 -vec samples1.jpg -bg bg.txt -npos 1 -nneg 4 -nstages 20 -mem 500 -w 20 -h 20

In the above command test2 is the directory name in which the trained classifier is stored. The -vec option denotes the samples that were captured by the createsamples utility above. The bg.txt contains the negative samples file. The width and height have to be the same as used in the create samples utility. As mentioned before if the width and height are too large you the haartrainer will bail out with “Insufficient memory”

If the haartrainer executes successfully the test2 directory will have all the trained files and the directory will also have the haarclassifier as a test2.xml.

Once these two steps go through successfully we have to run the performance step

Performance (performance.cpp): This step is run to ensure that we have trained our application to properly recognize the object we intended to. As before the necessary file to build is performance.cpp. The files to build are

(performance.cpp, cvboost.cpp, cvhaarclassifier.cpp, cvhaartarining.cpp, cvsamples.cpp, cvcommon.cpp)

Once this built successfuly you can now run using the command

./performance -data test2.xml -info bg.txt -w 20 -h 20

Make sure that the bg.txt contains the images from which the object has to be detected in the following

format

[positive filename] [# of objects] [[x y width height]

bg.txt

/img/logo5.jpg 1 145 100 20 20

./img/logo6.jpg 1 145 100 20 20

./img/logo3.jpg 1 145 100 45 45

./img/logo4.jpg 1 145 100 45 45

./img/airplane.jpg 1 145 100 45 45

./img/baboon.jpg 1 145 100 45 45

./img/kid.jpg 1 145 100 45 45

./img/lena.jpg 1 145 100 45 45

./img/opencv-logo2.png 1 145 100 35 35

./img/logo.png 1 145 100 45 45

When this is run we get

ganesh@localhost Debug]$ ./performance -data test3.xml -info bg.txt -w 20 -h 20

+================================+======+======+======+

|            File Name           | Hits |Missed| False|

+================================+======+======+======+

|                 ./img/logo5.jpg|     0|     1|    43|

+——————————–+——+——+——+

|                 ./img/logo6.jpg|     0|     1|    51|

+——————————–+——+——+——+

|                 ./img/logo3.jpg|     1|     0|    37|

+——————————–+——+——+——+

|                 ./img/logo4.jpg|     0|     1|     7|

+——————————–+——+——+——+

|              ./img/airplane.jpg|     1|     0|   226|

+——————————–+——+——+——+

|                ./img/baboon.jpg|     0|     1|   236|

+——————————–+——+——+——+

|                   ./img/kid.jpg|     0|     1|  1291|

+——————————–+——+——+——+

|                  ./img/lena.jpg|     0|     1|   188|

+——————————–+——+——+——+

|          ./img/opencv-logo2.png|     0|     1|     3|

+——————————–+——+——+——+

|                  ./img/logo.png|     0|     1|    33|

+——————————–+——+——+——+

|                           Total|     2|     8|  2115|

+================================+======+======+======+

As can be seen the training has been too accurate. There are hits and misses along with a false positive. The number of positive and negative samples, the co-ordinates and the number of stages all have to fine tuned to get the correct result.

Watch this space!

I will be back! Hasta la vista!

Please do take a look at my sequel to this post Hand detection through Haartraining: A Hands-on approach for a more robust haartraining method

Find me on Google+