Installing OpenCV: The Saga

ocv

I’ve started setting up OpenCV on a Raspberry Pi (RPI) for my degree show work! It’s not simple, but is the premiere open source computer vision package out there. I want it to be on a RPI because the smaller the computer, the better. Because it’s quite complex and information is disparate I thought I would write up my experience. It’s not a tutorial as such, but will hopefully help people know what is involved.

I’m considering making my robot-pump-machine-thing react to seeing its own image. Another idea is occult/alien symbols, or perhaps another robot, but I think there’s something cute/creepy/interesting about a robot who’s only function is to recognise itself.

I’ve previously used face detection libraries from the OpenCV portion of OpenFrameworks (OF), but now that I want to do something more elaborate and idiosyncratic I need to make use of algorithms not included in OF. I have actually previously tried installing OpenCV (OCV) on a Raspberry Pi, but that was before I learnt C++ and I was quickly scared away by the huge amount of know how you needed.

Now, armed with just enough knowledge to get myself in trouble, I’m trying again! I could use the Python bindings for OCV, but what I’m doing will probably be quite intensive so I need every scrap of performance I can wrench from the soon-to-be-exhausted ARMv8 processor. Python bindings means that the Python code you write isn’t actually running as Python, it’s directly calling C++ functions under the hood, so it’s not actually that much less efficient. But, any native Python code is still going to slow it down little bit, and seeing as I have a fair amount of experience with C++, I’ve decided to stick with that. I’m also running the RPI without a graphical display so I can funnel as much memory as possible into my vain attempt to make something “self-aware”.

Installation starts by following the guide for Linux here. However, there is a problem with installing these packages:

libtbb2 libtbb-dev

These are for using Intels Thread Building Blocks library which allows you to do some fancy multi-threading. You can safely choose to compile OCV without it, but it does apparently make a significant difference to performance when used on a multi-core processor. I have a multi-core processor and an obsession with performance, so I needed to compile it from scratch. Luckily, the very helpful “danielchalef” on StackExchange has already worked out the details.

The algorithm I am going to experiment with first is called SURF. This is a faster version of SIFT. It essentially identifies interesting points (feature detection) and then creates a way of describing those points (descriptor extraction). Interesting, in this case, means points with a high degree of variation around them, which usually means “corners”. This is why you’ll see the phrase corner-detection thrown around a lot. If something has a lot of variance, you have much more information with which to compare it to other points and so you can check if one image is similar to another image!

Because of legal reasons (*cough**patents**cough*), SURF has been relegated to the opencv-contrib repo, a separate directory of source files. In my excitement to spend a whole day compiling, I failed to realise this, and so am currently going through my second day of compiling with the new library. Actually, as of right this second I am being informed that my device has run out of room. So if you do this yourself, don’t do what I did, and make sure you check you have enough room before compiling.

You should learn how cmake works. the basic idea is quite simple. It’s a platform independent way of setting lots of project settings before building source code into executable programs! You need to pass quite a few settings to cmake when you build OCV, so here’s what I had to put in for reference:

cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local -DOPENCV_EXTRA_MODULES_PATH=~/opencv-contrib/modules -DWITH_TBB:BOOL=TRUE ..

This is run from inside a folder called “build” that I created in the root opencv source folder. The two dots on the end are important. They tell cmake that the root cmake information is in the folder above. Keeping everything produced by the compiling process in a separate folder (the “build” folder) is an important concept in cmake that is called “out of source building”. Basically, it means you don’t have the detritus of compiling messing up your source files, and you can safely delete everything in the build folder if you want to start over. With that done you can “make” and then “make install” OCV! It’s not over yet though.

OCV uses “X” to display images on linux. Unfortunately, lite-install Raspberry Pis do not have X! I could install it, but then loose all the advantages of having a light install. As well as memory, it takes up a lot of disk (card?) space, which as noted above, is already becoming a problem (for what it’s worth, my card is 16GB).

The answer is SDL! This is a library for 2D graphics and input, and possibly sound as well. From my experiences so far it’s super easy to use, and I quite like the syntax. It can use X, but also supports other ways of using images. This lovely man has written an easy-to-follow tutorial on installing it on RPI.

The difficult part is getting OCV data formats into the SDL graphics pipeline. It took me long hours of frustration to work it out, and here’s my solution! Well, not entirely mine. It’s mostly not mine. It’s taken from here and I would have been lost with out it!


void matToTex(SDL_Texture * tex, cv::Mat const &mat)
{
    unsigned char * textureData = NULL;
    int texturePitch = 0;

    SDL_LockTexture(tex, 0, (void**)&textureData, &texturePitch);
    memcpy(textureData, (void *) mat.data, mat.cols * mat.rows * mat.channels());
    SDL_UnlockTexture(tex);
}


My change was to completely remove the IplImage. IplImage is a format developed by Intel for OCV and used to be the standard format used. It is now a legacy format and has been replaced in function by the Mat data structure. “GotNoSpirits” code casts the Mat argument as a IplImage pointer, but this threw an error for me. It worked when I changed it to instead copy the data, but obviously copying every image before displaying it is not going to be feasible (PERFORMANCE!!!!). I realised, however, that there’s no reason why you can’t access the data from mat, rather than IplImage, and so I skipped out the casting stage altogether. Hopefully  there isn’t a reason why it was done this way that is going to come back to haunt me later on.

This is the first time I have used the memcpy function, and is the key to simply and cleanly converting the data types! It’s a low level C function that copies the byte data from one structure to another. Both must be cast as pointers to void, and the type does not matter! It seems like a potentially dangerous operation, because you are manipulating memory with function that doesn’t check that what you’re doing isn’t insane. Because the type does not matter, we can simply transfer the mat data to the SDL texture, because we know both are stored as unsigned chars! At least, I do because the mat and the texture I pass to the function were declared that way. You absolutely MUST check that the data of both is the same type and length and fits exactly.

Now I’m going to sleep before I have to submit myself to another round of compiling.
 

1 thought on “Installing OpenCV: The Saga”

  1. Update:
    I was actually only using half of the SD-card. sudo raspi-config, advanced options, expand file system was all I had to do.

Leave a Reply

Your email address will not be published. Required fields are marked *