ElphelWiki - User contributions [en]

OpenCV Tennis balls recognizing tutorial

2009-06-20T16:10:20Z

Andy753421:

This tutorial demonstrates how to use an Elphel (or perhaps another) camera to perform some basic computer vision tasks, such as identifying objects. For this example, we will be recognizing tennis balls. A [[Image:Tennis.tar.gz|finished example]] is available for GNU/Linux systems.

== Prerequisites ==
;OpenCV
:[http://opencv.willowgarage.com/wiki/ OpenCV] is a C library designed to help with computer vision programs. It provides quite a few useful functions that can save a lot typing when performing operations on images.

;V4L/AVLD
:OpenCV provides a V4L API that can be used to acquire images from Elphel cameras. This is done using [[AVLD_-_Another_Video_Loopback_Device|AVLD]].

== Image acquisition ==
[[Image:Tennis-input.jpg|thumb|Example input image]]

The first step is to get some images into OpenCV and display them. This assumes you have AVLD and V4L already set up. Below is a fairly minimal example that captures and displays images. It can be compiled with '''gcc -o main main.c $(pkg-config --libs --cflags opencv)'''.

<pre>
#include <opencv/cv.h>
#include <opencv/highgui.h>
#include <X11/keysym.h>

int main(int argc, char **argv)
{
/* Start the CV system and get the first v4l camera */
cvInitSystem(argc, argv);
CvCapture *cam = cvCreateCameraCapture(0);

/* Create a window to use for displaying the images */
cvNamedWindow("img", 0);
cvMoveWindow("img", 200, 200);

/* Display images until the user presses q */
while (1) {
cvGrabFrame(cam);
IplImage *img = cvRetrieveFrame(cam);
cvShowImage("img", img);
if (cvWaitKey(10) == XK_q)
return 0;
cvReleaseImage(&img);
}
}
</pre>

== Processing ==
=== Color space ===
[[Image:Tennis-hsv.jpg|thumb|Example HSV output]]

A good first step in many CV algorithms is to convert the image to [http://en.wikipedia.org/wiki/HSL_and_HSV HSV] (or another similar [http://en.wikipedia.org/wiki/Color_space color space]). This makes picking out objects based on colors a bit simpler, as will be seen later. We'll make a copy of the original image so that we can display the original at the end. Note that OpenCV stores images in BGR format by default.
<pre>
CvSize size = cvGetSize(img);
IplImage *hsv = cvCreateImage(size, IPL_DEPTH_8U, 3);
cvCvtColor(img, hsv, CV_BGR2HSV);
</pre>

=== Masks ===
[[Image:Tennis-mask.png|thumb|Example mask]]

The next step is to select all pixels that we think might be part of a tennis ball. We'll do this based purely on their HSV values. OpenCV provides a InRanage function that can be used to pick out pixels based on their values. This generates a [http://en.wikipedia.org/wiki/Mask_(computing) mask]; a binary image where the foreground pixels (white) were within the specified range. We're done with the HSV image after this, so we can free it's memory.

Picking the ranges for creating masks is one of the more complicated parts of a CV algorithm. For now, manually tuning the ranges is easiest. You might also use one of the OpenCV machine learning algorithms to pick the ranges automatically.

<pre>
CvMat *mask = cvCreateMat(size.height, size.width, CV_8UC1);
cvInRangeS(hsv, cvScalar(0.11*256, 0.60*256, 0.20*256, 0),
cvScalar(0.14*256, 1.00*256, 1.00*256, 0), mask);
cvReleaseImage(&hsv);
</pre>

=== Morphological operations ===
[[Image:Tennis-morph.png|thumb|Mask after morphological operations]]

No matter how good your ranges are when generating a mask, there will almost always be noise in the mask. In our example, the white lines on the tennis ball don't show up because they don't fit the hue range. Much of this nose can be eliminated by using a series of [http://en.wikipedia.org/wiki/Mathematical_morphology morphological operations]. Two commonly uses operation are [http://en.wikipedia.org/wiki/Opening_(morphology) opening] and [http://en.wikipedia.org/wiki/Closing_(morphology) closing], which are in turn comprised of [http://en.wikipedia.org/wiki/Dilation_(morphology) dilate] and [http://en.wikipedia.org/wiki/Erosion_(morphology) erode] operations. The table below summarizes these operations.

{| border=1
! Operation !! Effect / Use
|-
| Dilate || Expand the foreground
|-
| Erode || Contract the foreground (~ expand background)
|-
| Close || Dilation followed by erosion, removes specks of background, fills in foreground areas.
|-
| Open || Erosion followed by dilation, removes specks of foreground, fills in background areas.
|}

Morphological operations are performed with a [http://en.wikipedia.org/wiki/Structuring_element Structuring Element]. In computer vision, this is typically a oval or a rectangle of some specific size. Note that using rectangles results in faster code but can also cause poorer results.

Below, we use a large rectangular structuring element along with a close to remove the black lines that show up in the tennis balls. Afterwards, we perform an open with a smaller structuring element to eliminate some additional nose from the image.

<pre>
IplConvKernel *se21 = cvCreateStructuringElementEx(21, 21, 10, 10, CV_SHAPE_RECT, NULL);
IplConvKernel *se11 = cvCreateStructuringElementEx(11, 11, 5, 5, CV_SHAPE_RECT, NULL);
cvClose(mask, mask, se21); // See completed example for cvClose definition
cvOpen(mask, mask, se11); // See completed example for cvOpen definition
cvReleaseStructuringElement(&se21);
cvReleaseStructuringElement(&se11);
</pre>

=== Hough transform ===
The real work in finding tennis balls is done by a [http://en.wikipedia.org/wiki/Hough_transform Hough transform]. The specifics of this are beyond the scope of this tutorial. We'll just treat it as a black box function that finds circular objects in an input image.

The OpenCV Hough function performs a [http://en.wikipedia.org/wiki/Canny_edge_detector Canny edge detection] on the input image before the actual Hough transform. Due to this and the way the Hough transform works, it is beneficial to do quite a bit of smoothing to get a nice gradient around the edge of the circles before passing the image to the Hough function. Many of the parameters to the Hough function can also be tuned to provide better results.

<pre>
/* Copy mask into a grayscale image */
IplImage *hough_in = cvCreateImage(size, 8, 1);
cvCopy(mask, hough_in, NULL);
cvSmooth(hough_in, hough_in, CV_GAUSSIAN, 15, 15, 0, 0);

/* Run the Hough function */
CvMemStorage *storage = cvCreateMemStorage(0);
CvSeq *circles = cvHoughCircles(hough_in, storage,
CV_HOUGH_GRADIENT, 4, size.height/10, 100, 40, 0, 0);
cvReleaseMemStorage(&storage);
</pre>

== Output ==
[[Image:Tennis-output.jpg|thumb|Example output on a particularly nice image]]

The output of the Hough function can then be used in a variety of ways. For now we'll just draw some circles and centers onto the original input image before displaying it.

<pre>
int i;
for (i = 0; i < circles->total; i++) {
float *p = (float*)cvGetSeqElem(circles, i);
CvPoint center = cvPoint(cvRound(p[0]),cvRound(p[1]));
CvScalar val = cvGet2D(mask, center.y, center.x);
if (val.val[0] < 1) continue;
cvCircle(img, center, 3, CV_RGB(0,255,0), -1, CV_AA, 0);
cvCircle(img, center, cvRound(p[2]), CV_RGB(255,0,0), 3, CV_AA, 0);
cvCircle(mask, center, 3, CV_RGB(0,255,0), -1, CV_AA, 0);
cvCircle(mask, center, cvRound(p[2]), CV_RGB(255,0,0), 3, CV_AA, 0);
}
</pre>

== Acknowledgments ==
Some of the code provided as part of this example was developed by Jon Nibert and Andy Spencer as part of the Image Recognition course taught at Rose-Hulman Institute of Technology. All examples are provided under the GNU GPLv3.

Talk:OpenCV Tennis balls recognizing tutorial

2009-06-18T14:42:22Z

Andy753421:

I don't currently have an Elphel camera at my disposal anymore, so I was unable to test to make sure the image acquisition code in the tarball actually works. --[[User:Andy753421|Andy753421]]

OpenCV Tennis balls recognizing tutorial

2009-06-18T14:40:06Z

Andy753421:

This tutorial demonstrates how to use an Elphel (or perhaps another) camera to perform some basic computer vision tasks, such as identifying objects. For this example, we will be recognizing tennis balls. A [[Image:Tennis.tar.gz|finished example]] is available for GNU/Linux systems.

== Prerequisites ==
;OpenCV
:[http://opencv.willowgarage.com/wiki/ OpenCV] is a C library designed to help with computer vision programs. It provides quite a few useful functions that can save a lot typing when performing operations on images.

;V4L/AVLD
:OpenCV provides a V4L API that can be used to acquire images from Elphel cameras. This is done using [[AVLD_-_Another_Video_Loopback_Device|AVLD]].

== Image acquisition ==
[[Image:Tennis-input.jpg|thumb|Example input image]]

The first step is to get some images into OpenCV and display them. This assumes you have AVLD and V4L already set up. Below is a fairly minimal example that captures and displays images. It can be compiled with '''gcc -o main main.c $(pkg-config --libs --cflags opencv)'''.

<pre>
#include <opencv/cv.h>
#include <opencv/highgui.h>
#include <X11/keysym.h>

int main(int argc, char **argv)
{
/* Start the CV system and get the first v4l camera */
cvInitSystem(argc, argv);
CvCapture *cam = cvCreateCameraCapture(0);

/* Create a window to use for displaying the images */
cvNamedWindow("img", 0);
cvMoveWindow("img", 200, 200);

/* Display images until the user presses q */
while (1) {
cvGrabFrame(cam);
IplImage *img = cvRetrieveFrame(cam);
cvShowImage("img", img);
if (cvWaitKey(10) == XK_q)
return 0;
cvReleaseImage(&img);
}
}
</pre>

== Processing ==
=== Color space ===
[[Image:Tennis-hsv.jpg|thumb|Example HSV output]]

A good first step in many CV algorithms is to convert the image to [http://en.wikipedia.org/wiki/HSL_and_HSV HSV] (or another similar [http://en.wikipedia.org/wiki/Color_space color space]). This make picking out objects based on colors a bit simpler as will be seen later. We'll make a copy of the original image so that we can display it at the end. Note that OpenCV stores images in BGR format by default.
<pre>
CvSize size = cvGetSize(img);
IplImage *hsv = cvCreateImage(size, IPL_DEPTH_8U, 3);
cvCvtColor(img, hsv, CV_BGR2HSV);
</pre>

=== Masks ===
[[Image:Tennis-mask.png|thumb|Example mask]]

The next step is to select all pixels that we think might be part of a tennis ball. We'll do this based purely on their HSV values. OpenCV provides a InRanage function that can be used to pick out pixels based on their values. This generates a [http://en.wikipedia.org/wiki/Mask_(computing) mask]; a binary image where the foreground (white) pixels were with in the specified range. We're done with the HSV image after this, so we can free it's memory.

Picking the ranges for creating mask is one of the more complicated parts of a CV algorithm. For now, manually tuning the values is easiest. You could also use a machine learning algorithm to pick the ranges automatically.

<pre>
CvMat *mask = cvCreateMat(size.height, size.width, CV_8UC1);
cvInRangeS(hsv, cvScalar(0.11*256, 0.60*256, 0.20*256, 0),
cvScalar(0.14*256, 1.00*256, 1.00*256, 0), mask);
cvReleaseImage(&hsv);
</pre>

=== Morphological operations ===
[[Image:Tennis-morph.png|thumb|Mask after morphological operations]]

No matter how good your ranges are when generating a mask, there will almost always be noise in the mask. In our example, the white lines on the tennis ball don't show up because they don't fit the hue range. Much of this nose can be eliminated by using a series of [http://en.wikipedia.org/wiki/Mathematical_morphology morphological operations]. Two commonly uses operation are [http://en.wikipedia.org/wiki/Opening_(morphology) opening] and [http://en.wikipedia.org/wiki/Closing_(morphology) closing], which are in turn comprised of [http://en.wikipedia.org/wiki/Dilation_(morphology) dilate] and [http://en.wikipedia.org/wiki/Erosion_(morphology) erode] operations. The table below summarizes these operations.

{| border=1
! Operation !! Effect / Use
|-
| Dilate || Expand the foreground
|-
| Erode || contract the foreground (~ expand background)
|-
| Close || Dilation followed by erosion, removes specks of background, fills in foreground areas.
|-
| Open || Erosion followed by dilation, removes specks of foreground, fills in background areas.
|}

Morphological operations are performed with a [http://en.wikipedia.org/wiki/Structuring_element Structuring Element]. In computer vision, this is typically a oval or a rectangle of some specific size. Note that using rectangles results in faster code but can also cause poorer results.

Below, we use a large rectangular structuring element along with a close to remove the black lines that show up in the tennis balls. Afterwards we perform an open with a smaller structuring element to eliminate some additional nose from the image.

<pre>
IplConvKernel *se21 = cvCreateStructuringElementEx(21, 21, 10, 10, CV_SHAPE_RECT, NULL);
IplConvKernel *se11 = cvCreateStructuringElementEx(11, 11, 5, 5, CV_SHAPE_RECT, NULL);
cvClose(mask, mask, se21);
cvOpen(mask, mask, se11);
cvReleaseStructuringElement(&se21);
cvReleaseStructuringElement(&se11);
</pre>

=== Hough transform ===
The real work in finding tennis balls is done by a [http://en.wikipedia.org/wiki/Hough_transform Hough transform]. The specifics of this are beyond the scope of this tutorial. We'll just treat it as a black box function that finds circular objects in a gray input image.

The OpenCV Hough function performs a [http://en.wikipedia.org/wiki/Canny_edge_detector Canny edge detection] on the input image before doing running the actual Hough transform. Due to this and the way the Hough transform works, it is beneficial to do quite a bit of smoothing to get a nice gradient around the edge of the circles before passing the image to the Hough function. Many of the parameters to the Hough function can also be tuned to provide better results.

<pre>
/* Copy mask into a gray scale image */
IplImage *hough_in = cvCreateImage(size, 8, 1);
cvCopy(mask, hough_in, NULL);
cvSmooth(hough_in, hough_in, CV_GAUSSIAN, 15, 15, 0, 0);

/* Run the Hough function */
CvMemStorage *storage = cvCreateMemStorage(0);
CvSeq *circles = cvHoughCircles(hough_in, storage,
CV_HOUGH_GRADIENT, 4, size.height/10, 100, 40, 0, 0);
cvReleaseMemStorage(&storage);
</pre>

== Output ==
[[Image:Tennis-output.jpg|thumb|Example output on a particularly nice image]]

The output of the Hough function can then be used in a variety of ways. For now we'll just draw some circles and centers onto the original input image before displaying it.

<pre>
int i;
for (i = 0; i < circles->total; i++) {
float *p = (float*)cvGetSeqElem(circles, i);
CvPoint center = cvPoint(cvRound(p[0]),cvRound(p[1]));
CvScalar val = cvGet2D(mask, center.y, center.x);
if (val.val[0] < 1) continue;
cvCircle(img, center, 3, CV_RGB(0,255,0), -1, CV_AA, 0);
cvCircle(img, center, cvRound(p[2]), CV_RGB(255,0,0), 3, CV_AA, 0);
cvCircle(mask, center, 3, CV_RGB(0,255,0), -1, CV_AA, 0);
cvCircle(mask, center, cvRound(p[2]), CV_RGB(255,0,0), 3, CV_AA, 0);
}
</pre>

== Acknowledgments ==
Some of the code provided as part of this example was developed by Jon Nibert and Andy Spencer as part of the Image Recognition course taught at Rose-Hulman Institute of Technology. All examples are provided under the GNU GPLv3.

OpenCV Tennis balls recognizing tutorial

2009-06-18T14:30:15Z

Andy753421:

This tutorial demonstrates how to use an Elphel (or perhaps another) camera to perform some basic computer vision tasks, such as identifying objects. For this example, we will be recognizing tennis balls. A [[Image:Tennis.tar.gz|finished example]] is available for GNU/Linux systems.

== Prerequisites ==
;OpenCV
:[http://opencv.willowgarage.com/wiki/ OpenCV] is a C library designed to help with computer vision programs. It provides quite a few useful functions that can save a lot typing when performing operations on images.

;V4L/AVLD
:OpenCV provides a V4L API that can be used to acquire images from Elphel cameras. This is done using [[AVLD_-_Another_Video_Loopback_Device|AVLD]].

== Image acquisition ==
[[Image:Tennis-input.jpg|thumb|Example input image]]

The first step is to get some images into OpenCV and display them. This assumes you have AVLD and V4L already set up. Below is a fairly minimal example that captures and displays images. It can be compiled with '''gcc -o main main.c $(pkg-config --libs --cflags opencv)'''.

<pre>
#include <opencv/cv.h>
#include <opencv/highgui.h>
#include <X11/keysym.h>

int main(int argc, char **argv)
{
/* Start the CV system and get the first v4l camera */
cvInitSystem(argc, argv);
CvCapture *cam = cvCreateCameraCapture(0);

/* Create a window to use for displaying the images */
cvNamedWindow("img", 0);
cvMoveWindow("img", 200, 200);

/* Display images until the user presses q */
while (1) {
cvGrabFrame(cam);
IplImage *img = cvRetrieveFrame(cam);
cvShowImage("img", img);
if (cvWaitKey(10) == XK_q)
return 0;
cvReleaseImage(&img);
}
}
</pre>

== Processing ==
=== Color space ===
[[Image:Tennis-hsv.jpg|thumb|Example HSV output]]

A good first step in many CV algorithms is to convert the image to HSV (or another similar color space). This make picking out objects based on colors a bit simpler as will be seen later. We'll make a copy of the original image so that we can display it at the end. Note that OpenCV stores images in BGR format by default.
<pre>
CvSize size = cvGetSize(img);
IplImage *hsv = cvCreateImage(size, IPL_DEPTH_8U, 3);
cvCvtColor(img, hsv, CV_BGR2HSV);
</pre>

=== Masks ===
[[Image:Tennis-mask.png|thumb|Example mask]]

The next step is to select all pixels that we think might be part of a tennis ball. We'll do this based purely on their HSV values. OpenCV provides a InRanage function that can be used to pick out pixels based on their values. This generates a mask; a binary image where the foreground (white) pixels were with in the specified range. We're done with the HSV image after this, so we can free it's memory.

Picking the ranges for creating mask is one of the more complicated parts of a CV algorithm. For now, manually tuning the values is easiest. You could also use a machine learning algorithm to pick the ranges automatically.

<pre>
CvMat *mask = cvCreateMat(size.height, size.width, CV_8UC1);
cvInRangeS(hsv, cvScalar(0.11*256, 0.60*256, 0.20*256, 0),
cvScalar(0.14*256, 1.00*256, 1.00*256, 0), mask);
cvReleaseImage(&hsv);
</pre>

=== Morphological operations ===
[[Image:Tennis-morph.png|thumb|Mask after morphological operations]]

No matter how good your ranges are when generating a mask, there will almost always be noise in the mask. In our example, the white lines on the tennis ball don't show up because they don't fit the hue range. Much of this nose can be eliminated by using a series of morphological operations. Two commonly uses operation are [http://en.wikipedia.org/wiki/Opening_(morphology) opening] and [http://en.wikipedia.org/wiki/Closing_(morphology) closing], which are in turn comprised of [http://en.wikipedia.org/wiki/Dilation_(morphology) dilate] and [http://en.wikipedia.org/wiki/Erosion_(morphology) erode] operations. The table below summarizes these operations.

{| border=1
! Operation !! Effect / Use
|-
| Dilate || Expand the foreground
|-
| Erode || contract the foreground (~ expand background)
|-
| Close || Dilation followed by erosion, removes specks of background, fills in foreground areas.
|-
| Open || Erosion followed by dilation, removes specks of foreground, fills in background areas.
|}

Morphological operations are performed with a [http://en.wikipedia.org/wiki/Structuring_element Structuring Element]. In computer vision, this is typically a oval or a rectangle of some specific size. Note that using rectangles results in faster code but can also cause poorer results.

Below, we use a large rectangular structuring element along with a close to remove the black lines that show up in the tennis balls. Afterwards we perform an open with a smaller structuring element to eliminate some additional nose from the image.

<pre>
IplConvKernel *se21 = cvCreateStructuringElementEx(21, 21, 10, 10, CV_SHAPE_RECT, NULL);
IplConvKernel *se11 = cvCreateStructuringElementEx(11, 11, 5, 5, CV_SHAPE_RECT, NULL);
cvClose(mask, mask, se21);
cvOpen(mask, mask, se11);
cvReleaseStructuringElement(&se21);
cvReleaseStructuringElement(&se11);
</pre>

=== Hough transform ===
The real work in finding tennis balls is done by a Hough transform. The specifics of this are beyond the scope of this tutorial. We'll just treat it as a black box function that finds circular objects in a gray input image.

The OpenCV Hough function performs a Canny edge detection on the input image before doing running the actual Hough transform. Due to this and the way the Hough transform works, it is beneficial to do quite a bit of smoothing to get a nice gradient around the edge of the circles before passing the image to the Hough function. Many of the parameters to the Hough function can also be tuned to provide better results.

<pre>
/* Copy mask into a gray scale image */
IplImage *hough_in = cvCreateImage(size, 8, 1);
cvCopy(mask, hough_in, NULL);
cvSmooth(hough_in, hough_in, CV_GAUSSIAN, 15, 15, 0, 0);

/* Run the Hough function */
CvMemStorage *storage = cvCreateMemStorage(0);
CvSeq *circles = cvHoughCircles(hough_in, storage,
CV_HOUGH_GRADIENT, 4, size.height/10, 100, 40, 0, 0);
cvReleaseMemStorage(&storage);
</pre>

== Output ==
[[Image:Tennis-output.jpg|thumb|Example output on a particularly nice image]]

The output of the Hough function can then be used in a variety of ways. For now we'll just draw some circles and centers onto the original input image before displaying it.

<pre>
int i;
for (i = 0; i < circles->total; i++) {
float *p = (float*)cvGetSeqElem(circles, i);
CvPoint center = cvPoint(cvRound(p[0]),cvRound(p[1]));
CvScalar val = cvGet2D(mask, center.y, center.x);
if (val.val[0] < 1) continue;
cvCircle(img, center, 3, CV_RGB(0,255,0), -1, CV_AA, 0);
cvCircle(img, center, cvRound(p[2]), CV_RGB(255,0,0), 3, CV_AA, 0);
cvCircle(mask, center, 3, CV_RGB(0,255,0), -1, CV_AA, 0);
cvCircle(mask, center, cvRound(p[2]), CV_RGB(255,0,0), 3, CV_AA, 0);
}
</pre>

== Acknowledgments ==
Some of the code provided as part of this example was developed by Jon Nibert and Andy Spencer as part of the Image Recognition course taught at Rose-Hulman Institute of Technology. All examples are provided under the GNU GPLv3.

File:Tennis-output.jpg

2009-06-18T14:25:03Z

Andy753421: Output of OpenCV tennis ball finder

Output of OpenCV tennis ball finder

File:Tennis-morph.png

2009-06-18T14:24:46Z

Andy753421: Cleaned up mask for OpenCV tennis ball finder

Cleaned up mask for OpenCV tennis ball finder

File:Tennis-mask.png

2009-06-18T14:24:28Z

Andy753421: Mask for OpenCV tennis ball finder

Mask for OpenCV tennis ball finder

File:Tennis-hsv.jpg

2009-06-18T14:24:11Z

Andy753421: HSV for OpenCV tennis ball finder

HSV for OpenCV tennis ball finder

File:Tennis-input.jpg

2009-06-18T14:23:48Z

Andy753421: Input for OpenCV tennis ball finder

Input for OpenCV tennis ball finder

File:Tennis.tar.gz

2009-06-18T14:14:14Z

Andy753421: Example OpenCV code to find tennis balls

Example OpenCV code to find tennis balls

OpenCV Tennis balls recognizing tutorial

2009-06-18T13:55:21Z

Andy753421: /* Acknowledgments */

This tutorial demonstrates how to use an Elphel (or perhaps another) camera to perform some basic computer vision tasks, such as identifying objects. For this example, we will be recognizing tennis balls.

== Prerequisites ==
;OpenCV
:[http://opencv.willowgarage.com/wiki/ OpenCV] is a C library designed to help with computer vision programs. It provides quite a few useful functions that can save a lot typing when performing operations on images.

;V4L/AVLD
:OpenCV provides a V4L API that can be used to acquire images from Elphel cameras. This is done using [[AVLD_-_Another_Video_Loopback_Device|AVLD]].

== Image acquisition ==
The first step is to get some images into OpenCV and display them. This assumes you have AVLD and V4L already set up. Below is a fairly minimal example that captures and displays images. It can be compiled with '''gcc -o main main.c $(pkg-config --libs --cflags opencv)'''.

<pre>
#include <opencv/cv.h>
#include <opencv/highgui.h>
#include <X11/keysym.h>

int main(int argc, char **argv)
{
/* Start the CV system and get the first v4l camera */
cvInitSystem(argc, argv);
CvCapture *cam = cvCreateCameraCapture(0);

/* Create a window to use for displaying the images */
cvNamedWindow("img", 0);
cvMoveWindow("img", 200, 200);

/* Display images until the user presses q */
while (1) {
cvGrabFrame(cam);
IplImage *img = cvRetrieveFrame(cam);
cvShowImage("img", img);
if (cvWaitKey(10) == XK_q)
return 0;
cvReleaseImage(&img);
}
}
</pre>

== Processing ==
=== Color space ===
A good first step in many CV algorithms is to convert the image to HSV (or another similar color space). This make picking out objects based on colors a bit simpler as will be seen later. We'll make a copy of the original image so that we can display it at the end. Note that OpenCV stores images in BGR format by default.
<pre>
CvSize size = cvGetSize(img);
IplImage *hsv = cvCreateImage(size, IPL_DEPTH_8U, 3);
cvCvtColor(img, hsv, CV_BGR2HSV);
</pre>

=== Masks ===
The next step is to select all pixels that we think might be part of a tennis ball. We'll do this based purely on their HSV values. OpenCV provides a InRanage function that can be used to pick out pixels based on their values. This generates a mask; a binary image where the foreground (white) pixels were with in the specified range. We're done with the HSV image after this, so we can free it's memory.

Picking the ranges for creating mask is one of the more complicated parts of a CV algorithm. For now, manually tuning the values is easiest. You could also use a machine learning algorithm to pick the ranges automatically.

<pre>
CvMat *mask = cvCreateMat(size.height, size.width, CV_8UC1);
cvInRangeS(hsv, cvScalar(0.11*256, 0.60*256, 0.20*256, 0),
cvScalar(0.14*256, 1.00*256, 1.00*256, 0), mask);
cvReleaseImage(&hsv);
</pre>

=== Morphological operations ===
No matter how good your ranges are when generating a mask, there will almost always be noise in the mask. In our example, the white lines on the tennis ball don't show up because they don't fit the hue range. Much of this nose can be eliminated by using a series of morphological operations. Two commonly uses operation are [http://en.wikipedia.org/wiki/Opening_(morphology) opening] and [http://en.wikipedia.org/wiki/Closing_(morphology) closing], which are in turn comprised of [http://en.wikipedia.org/wiki/Dilation_(morphology) dilate] and [http://en.wikipedia.org/wiki/Erosion_(morphology) erode] operations. The table below summarizes these operations.

{| border=1
! Operation !! Effect / Use
|-
| Dilate || Expand the foreground
|-
| Erode || contract the foreground (~ expand background)
|-
| Close || Dilation followed by erosion, removes specks of background, fills in foreground areas.
|-
| Open || Erosion followed by dilation, removes specks of foreground, fills in background areas.
|}

Morphological operations are performed with a [http://en.wikipedia.org/wiki/Structuring_element Structuring Element]. In computer vision, this is typically a oval or a rectangle of some specific size. Note that using rectangles results in faster code but can also cause poorer results.

Below, we use a large rectangular structuring element along with a close to remove the black lines that show up in the tennis balls. Afterwards we perform an open with a smaller structuring element to eliminate some additional nose from the image.

<pre>
IplConvKernel *se21 = cvCreateStructuringElementEx(21, 21, 10, 10, CV_SHAPE_RECT, NULL);
IplConvKernel *se11 = cvCreateStructuringElementEx(11, 11, 5, 5, CV_SHAPE_RECT, NULL);
cvClose(mask, mask, se21);
cvOpen(mask, mask, se11);
cvReleaseStructuringElement(&se21);
cvReleaseStructuringElement(&se11);
</pre>

=== Hough transform ===
The real work in finding tennis balls is done by a Hough transform. The specifics of this are beyond the scope of this tutorial. We'll just treat it as a black box function that finds circular objects in a gray input image.

The OpenCV Hough function performs a Canny edge detection on the input image before doing running the actual Hough transform. Due to this and the way the Hough transform works, it is beneficial to do quite a bit of smoothing to get a nice gradient around the edge of the circles before passing the image to the Hough function. Many of the parameters to the Hough function can also be tuned to provide better results.

<pre>
/* Copy mask into a gray scale image */
IplImage *hough_in = cvCreateImage(size, 8, 1);
cvCopy(mask, hough_in, NULL);
cvSmooth(hough_in, hough_in, CV_GAUSSIAN, 15, 15, 0, 0);

/* Run the Hough function */
CvMemStorage *storage = cvCreateMemStorage(0);
CvSeq *circles = cvHoughCircles(hough_in, storage,
CV_HOUGH_GRADIENT, 4, size.height/10, 100, 40, 0, 0);
cvReleaseMemStorage(&storage);
</pre>

== Output ==
The output of the Hough function can then be used in a variety of ways. For now we'll just draw some circles and centers onto the original input image before displaying it.

<pre>
int i;
for (i = 0; i < circles->total; i++) {
float *p = (float*)cvGetSeqElem(circles, i);
CvPoint center = cvPoint(cvRound(p[0]),cvRound(p[1]));
CvScalar val = cvGet2D(mask, center.y, center.x);
if (val.val[0] < 1) continue;
cvCircle(img, center, 3, CV_RGB(0,255,0), -1, CV_AA, 0);
cvCircle(img, center, cvRound(p[2]), CV_RGB(255,0,0), 3, CV_AA, 0);
cvCircle(mask, center, 3, CV_RGB(0,255,0), -1, CV_AA, 0);
cvCircle(mask, center, cvRound(p[2]), CV_RGB(255,0,0), 3, CV_AA, 0);
}
</pre>

== Acknowledgments ==
Some of the code provided as part of this example was developed by Jon Nibert and Andy Spencer as part of the Image Recognition course taught at Rose-Hulman Institute of Technology. All examples are provided under the GNU GPLv3.

OpenCV Tennis balls recognizing tutorial

2009-06-18T13:51:46Z

Andy753421: /* Output */

This tutorial demonstrates how to use an Elphel (or perhaps another) camera to perform some basic computer vision tasks, such as identifying objects. For this example, we will be recognizing tennis balls.

== Prerequisites ==
;OpenCV
:[http://opencv.willowgarage.com/wiki/ OpenCV] is a C library designed to help with computer vision programs. It provides quite a few useful functions that can save a lot typing when performing operations on images.

;V4L/AVLD
:OpenCV provides a V4L API that can be used to acquire images from Elphel cameras. This is done using [[AVLD_-_Another_Video_Loopback_Device|AVLD]].

== Image acquisition ==
The first step is to get some images into OpenCV and display them. This assumes you have AVLD and V4L already set up. Below is a fairly minimal example that captures and displays images. It can be compiled with '''gcc -o main main.c $(pkg-config --libs --cflags opencv)'''.

<pre>
#include <opencv/cv.h>
#include <opencv/highgui.h>
#include <X11/keysym.h>

int main(int argc, char **argv)
{
/* Start the CV system and get the first v4l camera */
cvInitSystem(argc, argv);
CvCapture *cam = cvCreateCameraCapture(0);

/* Create a window to use for displaying the images */
cvNamedWindow("img", 0);
cvMoveWindow("img", 200, 200);

/* Display images until the user presses q */
while (1) {
cvGrabFrame(cam);
IplImage *img = cvRetrieveFrame(cam);
cvShowImage("img", img);
if (cvWaitKey(10) == XK_q)
return 0;
cvReleaseImage(&img);
}
}
</pre>

== Processing ==
=== Color space ===
A good first step in many CV algorithms is to convert the image to HSV (or another similar color space). This make picking out objects based on colors a bit simpler as will be seen later. We'll make a copy of the original image so that we can display it at the end. Note that OpenCV stores images in BGR format by default.
<pre>
CvSize size = cvGetSize(img);
IplImage *hsv = cvCreateImage(size, IPL_DEPTH_8U, 3);
cvCvtColor(img, hsv, CV_BGR2HSV);
</pre>

=== Masks ===
The next step is to select all pixels that we think might be part of a tennis ball. We'll do this based purely on their HSV values. OpenCV provides a InRanage function that can be used to pick out pixels based on their values. This generates a mask; a binary image where the foreground (white) pixels were with in the specified range. We're done with the HSV image after this, so we can free it's memory.

Picking the ranges for creating mask is one of the more complicated parts of a CV algorithm. For now, manually tuning the values is easiest. You could also use a machine learning algorithm to pick the ranges automatically.

<pre>
CvMat *mask = cvCreateMat(size.height, size.width, CV_8UC1);
cvInRangeS(hsv, cvScalar(0.11*256, 0.60*256, 0.20*256, 0),
cvScalar(0.14*256, 1.00*256, 1.00*256, 0), mask);
cvReleaseImage(&hsv);
</pre>

=== Morphological operations ===
No matter how good your ranges are when generating a mask, there will almost always be noise in the mask. In our example, the white lines on the tennis ball don't show up because they don't fit the hue range. Much of this nose can be eliminated by using a series of morphological operations. Two commonly uses operation are [http://en.wikipedia.org/wiki/Opening_(morphology) opening] and [http://en.wikipedia.org/wiki/Closing_(morphology) closing], which are in turn comprised of [http://en.wikipedia.org/wiki/Dilation_(morphology) dilate] and [http://en.wikipedia.org/wiki/Erosion_(morphology) erode] operations. The table below summarizes these operations.

{| border=1
! Operation !! Effect / Use
|-
| Dilate || Expand the foreground
|-
| Erode || contract the foreground (~ expand background)
|-
| Close || Dilation followed by erosion, removes specks of background, fills in foreground areas.
|-
| Open || Erosion followed by dilation, removes specks of foreground, fills in background areas.
|}

Morphological operations are performed with a [http://en.wikipedia.org/wiki/Structuring_element Structuring Element]. In computer vision, this is typically a oval or a rectangle of some specific size. Note that using rectangles results in faster code but can also cause poorer results.

Below, we use a large rectangular structuring element along with a close to remove the black lines that show up in the tennis balls. Afterwards we perform an open with a smaller structuring element to eliminate some additional nose from the image.

<pre>
IplConvKernel *se21 = cvCreateStructuringElementEx(21, 21, 10, 10, CV_SHAPE_RECT, NULL);
IplConvKernel *se11 = cvCreateStructuringElementEx(11, 11, 5, 5, CV_SHAPE_RECT, NULL);
cvClose(mask, mask, se21);
cvOpen(mask, mask, se11);
cvReleaseStructuringElement(&se21);
cvReleaseStructuringElement(&se11);
</pre>

=== Hough transform ===
The real work in finding tennis balls is done by a Hough transform. The specifics of this are beyond the scope of this tutorial. We'll just treat it as a black box function that finds circular objects in a gray input image.

The OpenCV Hough function performs a Canny edge detection on the input image before doing running the actual Hough transform. Due to this and the way the Hough transform works, it is beneficial to do quite a bit of smoothing to get a nice gradient around the edge of the circles before passing the image to the Hough function. Many of the parameters to the Hough function can also be tuned to provide better results.

<pre>
/* Copy mask into a gray scale image */
IplImage *hough_in = cvCreateImage(size, 8, 1);
cvCopy(mask, hough_in, NULL);
cvSmooth(hough_in, hough_in, CV_GAUSSIAN, 15, 15, 0, 0);

/* Run the Hough function */
CvMemStorage *storage = cvCreateMemStorage(0);
CvSeq *circles = cvHoughCircles(hough_in, storage,
CV_HOUGH_GRADIENT, 4, size.height/10, 100, 40, 0, 0);
cvReleaseMemStorage(&storage);
</pre>

== Output ==
The output of the Hough function can then be used in a variety of ways. For now we'll just draw some circles and centers onto the original input image before displaying it.

<pre>
int i;
for (i = 0; i < circles->total; i++) {
float *p = (float*)cvGetSeqElem(circles, i);
CvPoint center = cvPoint(cvRound(p[0]),cvRound(p[1]));
CvScalar val = cvGet2D(mask, center.y, center.x);
if (val.val[0] < 1) continue;
cvCircle(img, center, 3, CV_RGB(0,255,0), -1, CV_AA, 0);
cvCircle(img, center, cvRound(p[2]), CV_RGB(255,0,0), 3, CV_AA, 0);
cvCircle(mask, center, 3, CV_RGB(0,255,0), -1, CV_AA, 0);
cvCircle(mask, center, cvRound(p[2]), CV_RGB(255,0,0), 3, CV_AA, 0);
}
</pre>

== Acknowledgments ==

OpenCV Tennis balls recognizing tutorial

2009-06-18T13:49:26Z

Andy753421: /* Processing */

This tutorial demonstrates how to use an Elphel (or perhaps another) camera to perform some basic computer vision tasks, such as identifying objects. For this example, we will be recognizing tennis balls.

== Prerequisites ==
;OpenCV
:[http://opencv.willowgarage.com/wiki/ OpenCV] is a C library designed to help with computer vision programs. It provides quite a few useful functions that can save a lot typing when performing operations on images.

;V4L/AVLD
:OpenCV provides a V4L API that can be used to acquire images from Elphel cameras. This is done using [[AVLD_-_Another_Video_Loopback_Device|AVLD]].

== Image acquisition ==
The first step is to get some images into OpenCV and display them. This assumes you have AVLD and V4L already set up. Below is a fairly minimal example that captures and displays images. It can be compiled with '''gcc -o main main.c $(pkg-config --libs --cflags opencv)'''.

<pre>
#include <opencv/cv.h>
#include <opencv/highgui.h>
#include <X11/keysym.h>

int main(int argc, char **argv)
{
/* Start the CV system and get the first v4l camera */
cvInitSystem(argc, argv);
CvCapture *cam = cvCreateCameraCapture(0);

/* Create a window to use for displaying the images */
cvNamedWindow("img", 0);
cvMoveWindow("img", 200, 200);

/* Display images until the user presses q */
while (1) {
cvGrabFrame(cam);
IplImage *img = cvRetrieveFrame(cam);
cvShowImage("img", img);
if (cvWaitKey(10) == XK_q)
return 0;
cvReleaseImage(&img);
}
}
</pre>

== Processing ==
=== Color space ===
A good first step in many CV algorithms is to convert the image to HSV (or another similar color space). This make picking out objects based on colors a bit simpler as will be seen later. We'll make a copy of the original image so that we can display it at the end. Note that OpenCV stores images in BGR format by default.
<pre>
CvSize size = cvGetSize(img);
IplImage *hsv = cvCreateImage(size, IPL_DEPTH_8U, 3);
cvCvtColor(img, hsv, CV_BGR2HSV);
</pre>

=== Masks ===
The next step is to select all pixels that we think might be part of a tennis ball. We'll do this based purely on their HSV values. OpenCV provides a InRanage function that can be used to pick out pixels based on their values. This generates a mask; a binary image where the foreground (white) pixels were with in the specified range. We're done with the HSV image after this, so we can free it's memory.

Picking the ranges for creating mask is one of the more complicated parts of a CV algorithm. For now, manually tuning the values is easiest. You could also use a machine learning algorithm to pick the ranges automatically.

<pre>
CvMat *mask = cvCreateMat(size.height, size.width, CV_8UC1);
cvInRangeS(hsv, cvScalar(0.11*256, 0.60*256, 0.20*256, 0),
cvScalar(0.14*256, 1.00*256, 1.00*256, 0), mask);
cvReleaseImage(&hsv);
</pre>

=== Morphological operations ===
No matter how good your ranges are when generating a mask, there will almost always be noise in the mask. In our example, the white lines on the tennis ball don't show up because they don't fit the hue range. Much of this nose can be eliminated by using a series of morphological operations. Two commonly uses operation are [http://en.wikipedia.org/wiki/Opening_(morphology) opening] and [http://en.wikipedia.org/wiki/Closing_(morphology) closing], which are in turn comprised of [http://en.wikipedia.org/wiki/Dilation_(morphology) dilate] and [http://en.wikipedia.org/wiki/Erosion_(morphology) erode] operations. The table below summarizes these operations.

{| border=1
! Operation !! Effect / Use
|-
| Dilate || Expand the foreground
|-
| Erode || contract the foreground (~ expand background)
|-
| Close || Dilation followed by erosion, removes specks of background, fills in foreground areas.
|-
| Open || Erosion followed by dilation, removes specks of foreground, fills in background areas.
|}

Morphological operations are performed with a [http://en.wikipedia.org/wiki/Structuring_element Structuring Element]. In computer vision, this is typically a oval or a rectangle of some specific size. Note that using rectangles results in faster code but can also cause poorer results.

Below, we use a large rectangular structuring element along with a close to remove the black lines that show up in the tennis balls. Afterwards we perform an open with a smaller structuring element to eliminate some additional nose from the image.

<pre>
IplConvKernel *se21 = cvCreateStructuringElementEx(21, 21, 10, 10, CV_SHAPE_RECT, NULL);
IplConvKernel *se11 = cvCreateStructuringElementEx(11, 11, 5, 5, CV_SHAPE_RECT, NULL);
cvClose(mask, mask, se21);
cvOpen(mask, mask, se11);
cvReleaseStructuringElement(&se21);
cvReleaseStructuringElement(&se11);
</pre>

=== Hough transform ===
The real work in finding tennis balls is done by a Hough transform. The specifics of this are beyond the scope of this tutorial. We'll just treat it as a black box function that finds circular objects in a gray input image.

The OpenCV Hough function performs a Canny edge detection on the input image before doing running the actual Hough transform. Due to this and the way the Hough transform works, it is beneficial to do quite a bit of smoothing to get a nice gradient around the edge of the circles before passing the image to the Hough function. Many of the parameters to the Hough function can also be tuned to provide better results.

<pre>
/* Copy mask into a gray scale image */
IplImage *hough_in = cvCreateImage(size, 8, 1);
cvCopy(mask, hough_in, NULL);
cvSmooth(hough_in, hough_in, CV_GAUSSIAN, 15, 15, 0, 0);

/* Run the Hough function */
CvMemStorage *storage = cvCreateMemStorage(0);
CvSeq *circles = cvHoughCircles(hough_in, storage,
CV_HOUGH_GRADIENT, 4, size.height/10, 100, 40, 0, 0);
cvReleaseMemStorage(&storage);
</pre>

== Output ==

== Acknowledgments ==

OpenCV Tennis balls recognizing tutorial

2009-06-18T13:06:13Z

Andy753421:

This tutorial demonstrates how to use an Elphel (or perhaps another) camera to perform some basic computer vision tasks, such as identifying objects. For this example, we will be recognizing tennis balls.

== Prerequisites ==
;OpenCV
:[http://opencv.willowgarage.com/wiki/ OpenCV] is a C library designed to help with computer vision programs. It provides quite a few useful functions that can save a lot typing when performing operations on images.

;V4L/AVLD
:OpenCV provides a V4L API that can be used to acquire images from Elphel cameras. This is done using [[AVLD_-_Another_Video_Loopback_Device|AVLD]].

== Image acquisition ==
The first step is to get some images into OpenCV and display them. This assumes you have AVLD and V4L already set up. Below is a fairly minimal example that captures and displays images. It can be compiled with '''gcc -o main main.c $(pkg-config --libs --cflags opencv)'''.

<pre>
#include <opencv/cv.h>
#include <opencv/highgui.h>
#include <X11/keysym.h>

int main(int argc, char **argv)
{
/* Start the CV system and get the first v4l camera */
cvInitSystem(argc, argv);
CvCapture *cam = cvCreateCameraCapture(0);

/* Create a window to use for displaying the images */
cvNamedWindow("img", 0);
cvMoveWindow("img", 200, 200);

/* Display images until the user presses q */
while (1) {
cvGrabFrame(cam);
IplImage *img = cvRetrieveFrame(cam);
cvShowImage("img", img);
if (cvWaitKey(10) == XK_q)
return 0;
cvReleaseImage(&img);
}
}
</pre>

== Processing ==
=== Color space ===
=== Masks ===
=== Morphological operations ===
=== Hough transform ===

== Output ==

== Acknowledgments ==

OpenCV Tennis balls recognizing tutorial

2009-06-18T12:42:03Z

Andy753421: templating some things out

This tutorial demonstrates how to use an Elphel (or perhaps another) camera to perform some basic computer vision tasks, such as identifying objects. For this example, we will be recognizing tennis balls.

== Prerequisites ==
=== OpenCV ===
=== V4L/AVLD ===

== Image acquisition ==

== Processing ==
=== Color space ===
=== Masks ===
=== Morphological operations ===
=== Hough transform ===

== Output ==

== Acknowledgments ==

Rose-Hulman Robotics Team

2008-10-23T01:37:09Z

Andy753421: Updating with information for our net website

The [http://rhitrobotics.org Rose-Hulman Robotics Team] is in the process of constructing an autonomous ground vehicle and will be competing in the [http://www.igvc.org IGVC] competition. We attended the competition during the summer of 2008 but were unable to compete due issues with our motor controllers. We plan on attending the 2009 competition this coming year and expect to do better.

All of our software, hardware, and electronics designs are licensed under the GNU GPLv3 and are available from our [https://rhitrobotics.org/svn/rhrt/trunk/ subversion server]. Documentation is provided on our Trac development site. If you are interested in helping with our code please contact one us and we can set up a user account so that you can have write access out our systems.

== Contact ==
* Our primary communication mechanism is our Trac website, see External Links below.
* We also regularly converse on [http://rhitrobotics.org/wiki/irc IRC] in #rhrt on irc.freenode.net
* We also maintain a [http://mailman.rose-hulman.edu/mailman/listinfo/arc mailing list], but this is primarily used for meeting reminders.

== External Links ==
* http://rhitrobotics.org/ - Main development site based on Trac
* http://rhitrobotics.org/wiki/igvc - Wiki page for our efforts on the IGVC
* http://rhitrobotics.org/browser/trunk - Subversion browser for our codebase
** http://rhitrobotics.org/browser/trunk/software/rb/camera/elphel.py - Code to access Elphel 353 cameras
** http://rhitrobotics.org/browser/trunk/software/rb/vision - Image processing routines
* http://rhitrobotics.org/report/14 - Active tickets, organized by component.

Rose-Hulman Robotics Team

2008-02-14T05:50:14Z

Andy753421:

The [http://rhitrobotics.org Rose-Hulman Robotics Team] is in the process of constructing an autonomous ground vehicle and will be competing in the [http://www.igvc.org IGVC] competition.

We are currently in the process of making our internal documentation available to the public. Currently read-only access is provided to anonymous users. If you are interested in helping contact one us and we can set up a user account so that you can make changes.

== External Links ==
* https://rhitrobotics.org/ - "External" site, aimed at the general public
* https://project.rhitrobotics.org/wiki/ - Main development site/wiki
* https://project.rhitrobotics.org/trac/rt/ - Trac install for the software team
* https://project.rhitrobotics.org/svn/software/ - SVN repository used by the software team

Rose-Hulman Robotics Team

2008-01-31T08:02:43Z

Andy753421:

The [http://rhitrobotics.org Rose-Hulman Robotics Team] is in the process of constructing an autonomous ground vehicle and will be competing in the [http://www.igvc.org IGVC] competition.

UserProjects

2008-01-31T07:53:25Z

Andy753421: adding rhitrobotics link

== Research Projects ==
There are several research and educational projects that use Elphel cameras. Some are already active, others just starting or only in planning stage. We believe that scientific research is the application area where the open nature of our products is the most beneficial. The shortage of the powerful and at the same time open and flexible hardware was that "itch" that led me to start Elphel.

This page is started to provide resources to those who already uses or plans to use Elphel cameras. This is wiki, so please feel free to add you project description/link below.--[[User:Andrey.filippov|Andrey.filippov]] 14:59, 29 April 2007 (CDT)

=== [[SCINI: Submersible Capable of under Ice Navigation and Imaging]] ===

=== [[EPFL: Auto focus System]] ===

=== [[Rose-Hulman Robotics Team]] ===