Difference between revisions of "OpenCV Tennis balls recognizing tutorial"

Revision as of 05:49, 18 June 2009

This tutorial demonstrates how to use an Elphel (or perhaps another) camera to perform some basic computer vision tasks, such as identifying objects. For this example, we will be recognizing tennis balls.

Prerequisites

OpenCV

OpenCV is a C library designed to help with computer vision programs. It provides quite a few useful functions that can save a lot typing when performing operations on images.

V4L/AVLD

OpenCV provides a V4L API that can be used to acquire images from Elphel cameras. This is done using AVLD.

Image acquisition

The first step is to get some images into OpenCV and display them. This assumes you have AVLD and V4L already set up. Below is a fairly minimal example that captures and displays images. It can be compiled with gcc -o main main.c $(pkg-config --libs --cflags opencv).

#include <opencv/cv.h>
#include <opencv/highgui.h>
#include <X11/keysym.h>

int main(int argc, char **argv)
{
	/* Start the CV system and get the first v4l camera */
	cvInitSystem(argc, argv);
	CvCapture *cam = cvCreateCameraCapture(0);

	/* Create a window to use for displaying the images */
	cvNamedWindow("img", 0);
	cvMoveWindow("img", 200, 200);

	/* Display images until the user presses q */
	while (1) {
		cvGrabFrame(cam);
		IplImage *img = cvRetrieveFrame(cam);
		cvShowImage("img", img);
		if (cvWaitKey(10) == XK_q)
			return 0;
		cvReleaseImage(&img);
	}
}

Processing

Color space

A good first step in many CV algorithms is to convert the image to HSV (or another similar color space). This make picking out objects based on colors a bit simpler as will be seen later. We'll make a copy of the original image so that we can display it at the end. Note that OpenCV stores images in BGR format by default.

	CvSize size = cvGetSize(img);
	IplImage *hsv = cvCreateImage(size, IPL_DEPTH_8U, 3);
	cvCvtColor(img, hsv, CV_BGR2HSV);  

Masks

The next step is to select all pixels that we think might be part of a tennis ball. We'll do this based purely on their HSV values. OpenCV provides a InRanage function that can be used to pick out pixels based on their values. This generates a mask; a binary image where the foreground (white) pixels were with in the specified range. We're done with the HSV image after this, so we can free it's memory.

Picking the ranges for creating mask is one of the more complicated parts of a CV algorithm. For now, manually tuning the values is easiest. You could also use a machine learning algorithm to pick the ranges automatically.

	CvMat *mask = cvCreateMat(size.height, size.width, CV_8UC1);
	cvInRangeS(hsv, cvScalar(0.11*256, 0.60*256, 0.20*256, 0),
	                cvScalar(0.14*256, 1.00*256, 1.00*256, 0), mask);
	cvReleaseImage(&hsv);

Morphological operations

No matter how good your ranges are when generating a mask, there will almost always be noise in the mask. In our example, the white lines on the tennis ball don't show up because they don't fit the hue range. Much of this nose can be eliminated by using a series of morphological operations. Two commonly uses operation are opening and closing, which are in turn comprised of dilate and erode operations. The table below summarizes these operations.

Morphological operations are performed with a Structuring Element. In computer vision, this is typically a oval or a rectangle of some specific size. Note that using rectangles results in faster code but can also cause poorer results.

Below, we use a large rectangular structuring element along with a close to remove the black lines that show up in the tennis balls. Afterwards we perform an open with a smaller structuring element to eliminate some additional nose from the image.

	IplConvKernel *se21 = cvCreateStructuringElementEx(21, 21, 10, 10, CV_SHAPE_RECT, NULL);
	IplConvKernel *se11 = cvCreateStructuringElementEx(11, 11, 5,  5,  CV_SHAPE_RECT, NULL);
	cvClose(mask, mask, se21);
	cvOpen(mask, mask, se11);
	cvReleaseStructuringElement(&se21);
	cvReleaseStructuringElement(&se11);

Hough transform

The real work in finding tennis balls is done by a Hough transform. The specifics of this are beyond the scope of this tutorial. We'll just treat it as a black box function that finds circular objects in a gray input image.

The OpenCV Hough function performs a Canny edge detection on the input image before doing running the actual Hough transform. Due to this and the way the Hough transform works, it is beneficial to do quite a bit of smoothing to get a nice gradient around the edge of the circles before passing the image to the Hough function. Many of the parameters to the Hough function can also be tuned to provide better results.

	/* Copy mask into a gray scale image */
	IplImage *hough_in = cvCreateImage(size, 8, 1);
	cvCopy(mask, hough_in, NULL);
        cvSmooth(hough_in, hough_in, CV_GAUSSIAN, 15, 15, 0, 0);

	/* Run the Hough function */
	CvMemStorage *storage = cvCreateMemStorage(0);
	CvSeq *circles = cvHoughCircles(hough_in, storage,
		CV_HOUGH_GRADIENT, 4, size.height/10, 100, 40, 0, 0);
	cvReleaseMemStorage(&storage);

Output

Acknowledgments