Computer Vision: Ramblings on derivatives, histograms and contours


Images can be visualized to be functions of the form f(x,y) where f(x,y) represents the intensity at the pixel position ,y. However images can be grayscale, color or four channels and each channel may consist of integers or floating point numbers. However the changes in the values can be viewed as a continuous function. Here is a nice representation of an image as a continuous function (courtesy: Prof Darrell’s lecture at Berkeley on Filters)

Given that the image can be viewed as a continuous function in 2 or 3 axes we have derivatives that can be taken of the images. The derivates determine the maximum and minimum of this changing function. The key derivatives in image processing are the Sobel, the Scharr and the Laplacian filters. These provide the 1st order or 2nd order derivative and hence can be used for determining edges of an image.

I was keen on playing around with derivatives and also understanding how the histograms look like.

Here is the original image and its histogram. Clearly there is a nice spread of the values.

Sobel filter and its histogram

The output of the Sobel filter on the original image is shown. The edges with Sobel’s derivative somehow are not too pronounced. The Sobel derivate can be used for obtaining the gradient of the image. The corresponding histogram of the Sobel’s gradient is shown.

The code snippet ( The complete code is given below)

    IplImage* out_sobel = cvCreateImage( cvSize(img->width, img->height), IPL_DEPTH_16S, 1);
    cvSobel(in_gray, out_sobel, 1,1,7);
    cvShowImage("Sobel", out_sobel);

    //create an image to hold the histogram
    IplImage* histImage_Sobel = cvCreateImage(cvSize(300,400), 8, 1);

    //create a histogram to store the information from the image
    CvHistogram* histSobel = cvCreateHist(1, &hist_size, CV_HIST_ARRAY, ranges, 1);

    //calculate the histogram and apply to hist
    cvCalcHist( &histImage_Sobel, histSobel, 0, NULL );

    //grab the min and max values and their indeces
    cvGetMinMaxHistValue( histSobel, &min_value, &max_value, &min_idx, &max_idx);

    //scale the bin values so that they will fit in the image representation
    cvScale( histSobel->bins, histSobel->bins, ((double)histImage_Sobel->height)/max_value, 0 );

    //set all histogram values to 255
    cvSet( histImage_Sobel, cvScalarAll(255), 0 );

    //create a factor for scaling along the width
    bin_w = cvRound((double)histImage_Sobel->width/hist_size);

    for( i = 0; i < hist_size; i++ ) {
    //draw the histogram data onto the histogram image
    cvRectangle( histImage_Sobel, cvPoint(i*bin_w, histImage_Sobel->height),
    		   cvPoint((i+1)*bin_w,
    		   histImage_Sobel->height - cvRound(cvGetReal1D(histSobel->bins,i))),
    		   cvScalarAll(0), -1, 8, 0 );
    		//get the value at the current histogram bucket
    		float* bins = cvGetHistValue_1D(histSobel,i);
    		//increment the mean value
    		mean += bins[0];
    	}

    cvShowImage("Hist Sobel",histImage_Sobel);
...
...
(Please see Gavin S. Page's tutorial(vast.uccs.edu/~tboult/CS330/NOTES/OpenCVTutorial_II.ppt) on histograms)

Laplacian and its histogram

The Laplacian provides the 2nd order derivative and hence can be used to determine local maxima and local minima. The Laplacian provides for much more pronounced edges and can be used to extract features of an object of interest. Its corresponding histogram is also included.

Canny filter and Contours

The third filter is cvCanny which is most suitable for obtaining clear edges in an image. The canny is usually used along with cvFindContours to determine the general shape of an object. I used the canny filter which I passed to a contour detecting function. However the contour detecting function identified more than 228 contours most of which were useless except for 1 which had included the complete contour of the hand as shown.

However when I increased the max_depth to 1 I found that it was immediately able to get the complete contour of the hand besides a lot of extraneous contours.

I guess the challenge with the contour function is being able to programmatically reject all those contours which of lesser importance (possibly a future post).

Code for Sobel, Laplacian and Histograms

#include "cv.h"
#include "highgui.h"
#include "stdio.h"

int main(int argc, char** argv)
{
	IplImage* img = cvLoadImage("gazelle.jpg",1);
	IplImage* dst;
	IplImage* in_gray;
	int hist_size=30;
	float gray_ranges[] = { 0, 255 };
	float* ranges[]     = { gray_ranges};
	int min_idx,max_idx;
	float min_value,max_value;
	int bin_w;
	int i;
	float mean,variance;

	cvNamedWindow("Original",CV_WINDOW_AUTOSIZE);
	cvNamedWindow("histogram",CV_WINDOW_AUTOSIZE);
	cvNamedWindow("Sobel",CV_WINDOW_AUTOSIZE);
	cvNamedWindow("Hist Sobel",CV_WINDOW_AUTOSIZE);

	cvNamedWindow("Laplacian",CV_WINDOW_AUTOSIZE);
	cvNamedWindow("Hist Laplace",CV_WINDOW_AUTOSIZE);

	in_gray = cvCreateImage(cvSize(img->width, img->height), IPL_DEPTH_8U, 1);
	cvCvtColor(img, in_gray, CV_BGR2GRAY);
	cvShowImage("Original", in_gray);

	//create a rectangular area to evaluate
	CvRect rect = cvRect(0, 0, 300, 400 );
	//apply the rectangle to the image and establish a region of interest
	cvSetImageROI(in_gray, rect);

	//create an image to hold the histogram
	IplImage* histImage = cvCreateImage(cvSize(300,400), 8, 1);

	//create a histogram to store the information from the image
	CvHistogram* hist = cvCreateHist(1, &hist_size, CV_HIST_ARRAY, ranges, 1);

	//calculate the histogram and apply to hist
	cvCalcHist( &in_gray, hist, 0, NULL );

	//grab the min and max values and their indeces
	cvGetMinMaxHistValue( hist, &min_value, &max_value, &min_idx, &max_idx);

	//scale the bin values so that they will fit in the image representation
	cvScale( hist->bins, hist->bins, ((double)histImage->height)/max_value, 0 );

	//set all histogram values to 255
	cvSet( histImage, cvScalarAll(255), 0 );

	//create a factor for scaling along the width
	bin_w = cvRound((double)histImage->width/hist_size);

	for( i = 0; i < hist_size; i++ ) {
		//draw the histogram data onto the histogram image
		cvRectangle( histImage, cvPoint(i*bin_w, histImage->height),
		   cvPoint((i+1)*bin_w,
		   histImage->height - cvRound(cvGetReal1D(hist->bins,i))),
		   cvScalarAll(0), -1, 8, 0 );
		//get the value at the current histogram bucket
		float* bins = cvGetHistValue_1D(hist,i);
		//increment the mean value
		mean += bins[0];
	}

	//finish mean calculation
	mean /= hist_size;

	//go back through now that mean has been calculated in order to calculate variance
	for( i = 0; i < hist_size; i++ ) {
		float* bins = cvGetHistValue_1D(hist,i);
		variance += pow((bins[0] - mean),2);
	}
	//finish variance calculation
	variance /= hist_size;

	cvShowImage("histogram",histImage);

    IplImage* out_sobel = cvCreateImage( cvSize(img->width, img->height), IPL_DEPTH_16S, 1);
    cvSobel(in_gray, out_sobel, 1,1,7);
    cvShowImage("Sobel", out_sobel);

    //create an image to hold the histogram
    IplImage* histImage_Sobel = cvCreateImage(cvSize(300,400), 8, 1);

    //create a histogram to store the information from the image
    CvHistogram* histSobel = cvCreateHist(1, &hist_size, CV_HIST_ARRAY, ranges, 1);

    //calculate the histogram and apply to hist
    cvCalcHist( &histImage_Sobel, histSobel, 0, NULL );

    //grab the min and max values and their indeces
    cvGetMinMaxHistValue( histSobel, &min_value, &max_value, &min_idx, &max_idx);

    //scale the bin values so that they will fit in the image representation
    cvScale( histSobel->bins, histSobel->bins, ((double)histImage_Sobel->height)/max_value, 0 );

    //set all histogram values to 255
    cvSet( histImage_Sobel, cvScalarAll(255), 0 );

    //create a factor for scaling along the width
    bin_w = cvRound((double)histImage_Sobel->width/hist_size);

    for( i = 0; i < hist_size; i++ ) {
    //draw the histogram data onto the histogram image
    cvRectangle( histImage_Sobel, cvPoint(i*bin_w, histImage_Sobel->height),
    		   cvPoint((i+1)*bin_w,
    		   histImage_Sobel->height - cvRound(cvGetReal1D(histSobel->bins,i))),
    		   cvScalarAll(0), -1, 8, 0 );
    		//get the value at the current histogram bucket
    		float* bins = cvGetHistValue_1D(histSobel,i);
    		//increment the mean value
    		mean += bins[0];
    	}

    cvShowImage("Hist Sobel",histImage_Sobel);

    // Create Laplacian and the histogram for it
    IplImage *output=cvCreateImage( cvSize(img->width, img->height), IPL_DEPTH_16S, 1);
    cvLaplace(in_gray, output, 7);
    cvShowImage("Laplacian", output);

    //create an image to hold the histogram
     IplImage* histImage_Laplace = cvCreateImage(cvSize(300,400), 8, 1);

     //create a histogram to store the information from the image
     CvHistogram* histLaplace = cvCreateHist(1, &hist_size, CV_HIST_ARRAY, ranges, 1);

     //calculate the histogram and apply to hist
     cvCalcHist( &histImage_Laplace, histLaplace, 0, NULL );

     //grab the min and max values and their indeces
     cvGetMinMaxHistValue( histLaplace, &min_value, &max_value, &min_idx, &max_idx);

     //scale the bin values so that they will fit in the image representation
     cvScale( histLaplace->bins, histLaplace->bins, ((double)histImage_Laplace->height)/max_value, 0 );

     //set all histogram values to 255
     cvSet( histImage_Laplace, cvScalarAll(255), 0 );

     //create a factor for scaling along the width
     bin_w = cvRound((double)histImage_Laplace->width/hist_size);

     for( i = 0; i < hist_size; i++ ) {
     //draw the histogram data onto the histogram image
     cvRectangle( histImage_Laplace, cvPoint(i*bin_w, histImage_Laplace->height),
     		   cvPoint((i+1)*bin_w,
     		   histImage_Laplace->height - cvRound(cvGetReal1D(histLaplace->bins,i))),
     		   cvScalarAll(0), -1, 8, 0 );
     		//get the value at the current histogram bucket
     		float* bins = cvGetHistValue_1D(histLaplace,i);
     		//increment the mean value
     		mean += bins[0];
     	}

     cvShowImage("Hist Laplace",histImage_Laplace);

	cvWaitKey(0);

	printf("Mean= %f\n",mean);
	printf("variance=%f\n",variance);

	//clean up images
	cvReleaseImage(&histImage_Laplace);
	cvReleaseImage(&histImage_Sobel);
	cvReleaseImage(&histImage);
	cvReleaseImage(&in_gray);
	cvReleaseImage(&img);

	//remove windows
	cvDestroyWindow("Original");

	cvDestroyWindow("histogram");
}

Code for Canny & Contours
#include "cv.h"
#include "highgui.h"

#define CVX_RED		CV_RGB(0xff,0x00,0x00)
#define CVX_GREEN	CV_RGB(0x00,0xff,0x00)
#define CVX_BLUE	CV_RGB(0x00,0x00,0xff)

int main(int argc, char* argv[])
{
	CvSeq* c;
	int i;
	cvNamedWindow("Original", 1 );
	cvNamedWindow("Canny_Edge", 1 );
	cvNamedWindow("Contours", 1 );
	IplImage* img_8uc1 = cvLoadImage( argv[1], CV_LOAD_IMAGE_GRAYSCALE );
	IplImage* img_edge = cvCreateImage( cvGetSize(img_8uc1), 8, 1 );
	IplImage* img_8uc3 = cvCreateImage( cvGetSize(img_8uc1), 8, 3 );
	cvThreshold( img_8uc1, img_edge, 128, 255, CV_THRESH_BINARY );
	CvMemStorage* storage =cvCreateMemStorage(0);

	CvSeq* first_contour = NULL;

	int Nc;
	int n=0;

	cvShowImage("Original", img_8uc1);

    IplImage *out_canny=cvCreateImage( cvSize(img_8uc1->width, img_8uc1->height), IPL_DEPTH_8U, 1);
	cvCanny(img_8uc1, out_canny, 50.0 ,100.0, 3);
	cvShowImage("Canny_Edge", out_canny);

/*	Nc = cvFindContours(
			img_edge,
			storage,
			&first_contour,
			sizeof(CvContour),
			CV_RETR_LIST,
			CV_CHAIN_APPROX_SIMPLE,
			cvPoint(0,0)// Try all four values and see what happens
	);*/

	Nc = cvFindContours(
			out_canny,
			storage,
			&first_contour,
			sizeof(CvContour),
			CV_RETR_TREE,
			CV_CHAIN_APPROX_SIMPLE,
			cvPoint(0,0)// Try all four values and see what happens
	);

	printf("Total contours detected: %d\n",Nc);

	for(c=first_contour; c!=NULL; c=c->h_next )
	{
		cvCvtColor( img_8uc1, img_8uc3, CV_GRAY2BGR );
		cvDrawContours(
					img_8uc3,
					c,
					CVX_RED,
					CVX_BLUE,
					1,        // Try different values of max_level, and see what happens
					2,
					8,
					cvPoint(0,0));
			printf("Contour #%d\n",n);

			cvShowImage("Contours", img_8uc3 );
			printf(" %d elements: \n",c->total);

			for(i=0; i<c->total; ++i ) {
				CvPoint* p = CV_GET_SEQ_ELEM( CvPoint, c, i );
				printf("(%d,%d)\n",p->x,p->y);

			}
			cvWaitKey(0);
			n++;
	}
	printf("Finished all contours\n");
	cvCvtColor( img_8uc1, img_8uc3, CV_GRAY2BGR );
	cvShowImage( argv[0], img_8uc3 );
	cvWaitKey(0);
	cvDestroyWindow( argv[0] );
	cvReleaseImage( &img_8uc1 );
	cvReleaseImage( &img_8uc3 );
	cvReleaseImage( &img_edge );
	return 0;
}

 
 

Find me on Google+

Advertisements

2 thoughts on “Computer Vision: Ramblings on derivatives, histograms and contours

  1. Hello,

    I have been working with opencv for a little over a year (this blog has often been very helpful) and have run, far too often, into the problem you mention of being able to programatically reject (with a good enough accuracy, say >95%) the useless countours or lines (when using houghlines) I have a couple of implementations to help me achieve such a goal but most hover at the 85% accuracy mark.
    If you are serious about coming up with a reliable mehod or algorithm and would like some help do let me know, you can contact me at: jaime_apATyahooDOTcom.

  2. Pingback: My 3 video presentations on “Essential R” | Giga thoughts …

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s