Blancer.com Tutorials and projects

Microsoft’s Kinect has been out for a few months now and has become a fairly popular accessory for the Xbox 360. Let’s face it though, using the Kinect for what it was intended didn’t end up being the most exciting part of this new toy. What has become far more interesting is seeing the various hacks developed that makes the device so much more than simply an input mechanism for games. Now it’s your turn to do something amazing, and this tutorial will get you started. Today I’m going to get your Kinect up and running and demonstrate how to get the camera and depth information into your very own C# application.

Above is some example output that our app will produce. In this case it’s the corner of my office with some bookshelves and a guitar. The RGB data is on the left and the depth information is on the right. The darker things are, the closer they are to the camera.

1. Setup libfreenect

openkinect.org is going to be your best friend for this portion of the project. We’re going to be depending on libfreenect for our drivers and the library used to communicate with the Kinect. They’ve got pretty good instructions for all platforms, but since we’re doing C#, you’ll want to follow the Windows installation guide. I found the instructions to be fairly straight forward and worked without too much hassle. After you’ve followed all of those instructions return here to continue the tutorial. This portion of the project will probably take a while to complete – so be patient.

2. Setup the C# Application

Since our plan with this tutorial is just to display output, we can get away with a basic WPF application, which actually performs surprisingly well. If your app is going to do some really incredible things, you may want to consider something like DirectX or OpenGL.

Bundled as part of the libfreenect source are a set of wrappers for various languages. We’re going to want the C# one (\libfreenect\wrappers\csharp). Go ahead and add the wrapper project to your new solution.

You should now be able to build the solution without any errors. Of course, since we haven’t written any code, nothing will happen when you run the app. Also, our application now depends on the freenect.dll file that was created as part of step 1. Depending on how you installed it, you may have to copy this file somewhere where you app can load it (somewhere in the path, or the project’s output directory).

3. Writing some Code

Now we’re at the meat of this tutorial, writing some code to retrieve the Kinect’s output. The first thing we’re going to do is setup the Kinect and tell it to start recording data. I did all of this in my MainWindow’s constructor.

As you read through the code, it should be very self-explanatory. Basically we’re just connecting to a Kinect and telling it to start recording video and depth information. In order to events to be processed and raised by the Kinect library, we have to periodically call Kinect.ProcessEvents. Since we can’t block our main thread doing that, I just created a simple worker thread using the ThreadPool.

Now that we’ve got the Kinect setup, let’s take a look at the event handler, VideoCamera_DataReceived. This is where we’re going to receive the raw RGB data and convert it to something that can be displayed on the screen.

Fortunately for us, this part of the project is a breeze. The Kinect library returns RGB data in a form that can be directly fed into a BitmapSource object. I added some very basic protection against this event handler being called before the previous one had completed. I noticed that as the app ran longer and longer, it got slower and slower. This simple fix seemed to address that bug. All we have to do now is feed the raw data into BitmapSource.Create and give that to our Image control, which was added to our MainWindow in the XAML. The most complicated part of the Create call is the last parameter – stride. This argument represents the number of bytes in a single row of the image. Since our image contains 3 bytes per pixel, the number of bytes in a row will by the width multiplied by 3.

Now on to depth. This one is slightly more complicated, but still not too bad.

Depth data comes from the Kinect as 11 bits per pixel. The Kinect library packs that into a little friendlier 16 bits per pixel before passing it up to the C# wrapper and then on to us. What makes this difficult is that the bit order is big endian, whereas our Windows box needs little endian. In order to fix this, we need to read each and every pixel from the buffer and convert the endianness. Fortunately, .NET has a helper function designed for networking that comes in handy for this task – IPAddress.HostToNetworkOrder. After we’ve got all the pixels out and converted, we simply do the same thing as before to create our image. Instead of RGB format, this time we need to use Gray16, since each pixel is now represented by 2 bytes (16 bits).

And that’s it for retrieving Kinect data. If you combine this code with our XAML:

and a little cleanup code:

you should now have a working app that displays output very similar to the image below.

All of the libraries and wrappers used for this tutorial are in constant flux. Please refer to the most up-to-date documentation before starting to make sure nothing has changed. The libraries are also pretty buggy – so be prepared for things to not work correctly right out of the box.

Hopefully this tutorial will help save you some time bootstrapping your awesome Kinect hack. As we get more time to work with the libraries, we’ll be creating some more compelling demos. If you happen to make something neat, please drop us a line so we can check it out. If you have questions or comments, feel free to leave them below.

As you may have guessed by now, here at Switch On The Code, we are very big advocates of the .Net framework and all it has to offer. Sometimes however, things can be a little bit on the tricky side. Recently I ran into a tricky solution when I was trying to get image buttons in a DataGridView control, and in this tutorial I will go over the solution I came up with, which may just give you the edge the next time you need a fancy DataGridView.

DataGridViews do offer a few different column types, including fairly simple ways to brew up your own custom column types. However, none of the default column types offer a truly good solution for clickable images. What we want here is an image that does something when we click it. Sure we can have a button with an image in it if we use a Button Column, but that is not really what I am looking for. This is where the tricky solution comes in.

For a moment, we have to step back and consider all the options a DataGridView offers, especially the events that can be captured by it. One of these events happens to be CellClick, which is the key to our solution. Using this event we can capture which row and column is clicked, and therefore we can determine if one of our image cells is being clicked. Even better, we can even tell which actual cell was clicked. Using this information we can have image cells that act like buttons, and the best part is that it is not that complicated to get working.

So the first part is pretty strait forward, we need a DataGridView with some image columns. It doesn’t matter what images you use, or how you set up your columns, but you just have to keep track of the column names. Once you have your DataGridView all set up, we need to give it a couple test rows to work with, which we will do during initialization:

If you run the project and start clicking away, you will notice that nothing happens when you click on the image cells. This is not what we are after, but with some simple event parsing, we can get a click event going for these cells. In fact, we are going to hook to the CellClick event:

Basically what we are doing here is looking for specific columns when a cell is clicked. Technically, in this respect, you could use this method to tell if any specific column is clicked, like a text column as well. However, for this tutorial we just want something to happen when the user clicks on an image cell, and this code does exactly that. When the use clicks on a cell, this event is fired and we check the column of the cell that was clicked. If it is a column we are looking for, we do something, in this case we show a message.

There is one issue with this code though, it fires if ANY cell is clicked, including header cells (row and column headers). This a a fairly big issue, but one that can be fixed very easily. With a simple check, we can fix the problem:

With one simple if we now have a solid solution for image buttons in a DataGridView. Each time a cell in one of these columns is clicked, the corresponding message is displayed:

That is about it for this tutorial. I hope this quick solution helps you in your DataGridView endeavors, and just remember that when you need coding help all you have to do is Switch On The Code.