One of the most interesting
projects I've worked on in the past couple of years was a project about
processing. The goal was to develop a system to be able to recognize Coca-Cola
'cans' (note that I'm stressing the word 'cans', you'll see
why in a minute). You can see a sample below, with the can recognized in the
green rectangle with scale and
rotation.
matching">
Some constraints on the
project:
- The background
could be very noisy. - The can could
have any scale or rotation or even orientation
(within reasonable limits). - The image could
have some degree of fuzziness (contours might not be entirely
straight). - There could be Coca-Cola bottles in the image,
and the algorithm should only detect the
can! - The brightness of the image
could vary a lot (so you can't rely "too much" on color
detection). - The can could be partly
hidden on the sides or the middle and possibly partly hidden behind a
bottle. - There could be no can at all
in the image, in which case you had to find nothing and write a message saying
so.
So you could end up
with tricky things like this (which in this case had my algorithm totally
fail):
fail">
I did this project a while
ago, and had a lot of fun doing it, and I had a decent implementation. Here are some
details about my
implementation:
Language:
Done in C++ using OpenCV
library.
Pre-processing:
For the image pre-processing, i.e. transforming the image into a more raw form to give
to the algorithm, I used 2
methods:
- Changing color
domain from RGB to rel="noreferrer">HSV and filtering based on "red" hue, saturation above a
certain threshold to avoid orange-like colors, and filtering of low value to avoid dark
tones. The end result was a binary black and white image, where all white pixels would
represent the pixels that match this threshold. Obviously there is still a lot of crap
in the image, but this reduces the number of dimensions you have to work
with.
- Noise filtering using median
filtering (taking the median pixel value of all neighbors and replace the pixel by this
value) to reduce noise. - Using
Detection Filter to get the contours of all items after 2 precedent
steps.
Algorithm:
The algorithm itself I chose for this task was taken from
rel="noreferrer">this awesome book on feature extraction and called
rel="noreferrer">Generalized Hough Transform (pretty different from the
regular Hough Transform). It basically says a few
things:
- You can describe
an object in space without knowing its analytical equation (which is the case
here). - It is resistant to image deformations such as
scaling and rotation, as it will basically test your image for every combination of
scale factor and rotation factor. - It uses a
base model (a template) that the algorithm will
"learn". - Each pixel remaining in the contour image will
vote for another pixel which will supposedly be the center (in terms of gravity) of your
object, based on what it learned from the
model.
In the end, you
end up with a heat map of the votes, for example here all the pixels of the contour of
the can will vote for its gravitational center, so you'll have a lot of votes in the
same pixel corresponding to the center, and will see a peak in the heat map as
below:
alt="GHT">
Once you have that, a simple
threshold-based heuristic can give you the location of the center pixel, from which you
can derive the scale and rotation and then plot your little rectangle around it (final
scale and rotation factor will obviously be relative to your original template). In
theory at
least...
Results:
Now, while this approach worked in the basic cases, it was severely lacking in some
areas:
- It is
extremely slow! I'm not stressing this enough. Almost a
full day was needed to process the 30 test images, obviously because I had a very high
scaling factor for rotation and translation, since some of the cans were very
small. - It was completely lost when bottles were in the
image, and for some reason almost always found the bottle instead of the can (perhaps
because bottles were bigger, thus had more pixels, thus more
votes) - Fuzzy images were also no good, since the votes
ended up in pixel at random locations around the center, thus ending with a very noisy
heat map. - In-variance in translation and rotation was
achieved, but not in orientation, meaning that a can that was not directly facing the
camera objective wasn't
recognized.
Can you help
me improve my specific algorithm, using
exclusively OpenCV features, to resolve the
four specific issues
mentioned?
I hope some people will
also learn something out of it as well, after all I think not only people who ask
questions should learn. :)
No comments:
Post a Comment