Kinect on the BeagleBone (and Ubuntu)

[UPDATE 8 Mar 2016: This is now an old post and many of the links have had to be removed because they no longer work. Things have generally moved on and so the information here is almost certainly out of date. I’ve left this here for historical amusement only.]

[Updated 17 May 2012] A few weeks ago I managed to get my Ubuntu 11.04-running laptop properly configured to use the Kinect, using OpenNI and whatnot. It took a bit of doing. Now I’m trying to repeat the process with my BeagleBone (also running Ubuntu 11.04).

I’ve got a long way down the road, and have got it working, but I think it’s a little flaky and there are some issues. This post will get updated as & when I sort the problems.

There’s nothing inherently difficult about installing OpenNI. I’m learning about using the Kinect from the very fine book ’Making Things See’ by Greg Borenstein. Following his installation instructions, I had the device running on my MacBook Pro in a couple of minutes. I probably could have done the same on Windows, if I did Windows. But when it comes to Linux, Borenstein pretty much says, ‘if you use Linux, you’ll know how to do this’ and leaves you to it.

I found a very useful blog post on ‘Daves thoughts on stuff’ [sic]. He detailed a 29-step process to getting the Kinect working on Linux. If you want to follow along with what I do here, you’ll want to have that page open too. I’m not going to cut’n’paste Dave’s copy, just add my own notes where my experience differed from his. The numbers of the steps below correspond to Dave’s steps.

I also found a very handy post by Jatin Sharma about getting OpenNI working on the BeagleBoard. [Link now broken.]

Except where specified, all these notes apply equally to the Intel laptop and BeagleBone Arm installations.

So here we go…

[1] First problem is that libglut3-dev has been replaced by freeglut3-dev. Also, Dave doesn’t specify some libraries and apps that you’re going to need installed (possibly because he already had then installed). The following is a more comprehensive list of stuff you’re going to need to install, as detailed by Jatin. These all installed fine on my BeagleBoard:

# apt-get install git-core cmake libglut3-dev pkg-config gcc g++ build-essential libxmu-dev libxi-dev libusb-1.0-0-dev doxygen graphviz git

It should be noted, perhaps, than I’m logged into the BeagleBone as root. In fact, this is how I use the ‘Bone. It makes life simpler and, frankly, worrying about ‘security’ on my robot is just a little silly. So the hash mark above is my command-line prompt, not part of the command. Also, if you’re not logged in as root, you’ll need to use ‘sudo’ for a lot of this stuff.

You’re also going to need Java JDK, so for my BeagleBone I used:

# apt-get install openjdk-6-jdk

(Yeah, I know OpenJDK 7 is available, but I hit problems with that).

[2-3] Okay.

[4] The repo has changed. Dave says to cd to OpenNI/Platform/Linux-x86/Build, but for Intel platforms ‘Linux-x86’ has been replaced with Linux. So:

# cd OpenNI/Platform/Linux/Build

[5] ~/kinect/OpenNI/Platform/Linux/CreateRedist/RedistMaker doesn’t have execute permissions, so need to chmod before doing make && make install.

Jatin’s experience is that trying to use the Arm, rather than Intel, version throws up too many errors, so he suggests using the Intel version but with some modifications to ~/kinect/OpenNI/Platform/Linux-x86/Build/CommonMakefile. But things seem to have changed quite a lot with the repo. Make worked fine on my BeagleBone.

On Intel laptop: ‘make install’ looks for the install.sh script in ~/kinect/OpenNI/Platform/Linux/Redist but in fact it, and other files, are in a further subdir. This was OpenNI-Bin-Dev-Linux-x64-v1.5.2.23 on my laptop – obviously, your filename may vary). You could just manually go in there, once make install fails, and run install.sh, but I copied all the files in that subdir to ~/kinect/OpenNI/Platform/Linux/Redist/

On BeagleBone: ‘make install’ was rather more of a problem. It calls the ~/kinect/OpenNI/Platform/Linux/CreateRedist/RedistMaker script (just a wrapper to the Redist_OpenNi.py Python program). But this throws up an error on Arm. Presumably, you need instead to run RedistMaker.Arm (first having chmod’d it to give it execute permissions) instead. Jatin didn’t seem to have this problem (although he had another that involved installing Mono as the fix. I installed Mono as a precautionary measure.) But even running the RedistMaker.Arm script caused a problem. The Python script attempts to calculate the number of compile jobs and passes this to make using the ’-j <num>’ option. But in my case, it calculated the number as 0, and make requires -j to have a positive integer. I edited the Redist_OpenNi.py script, commenting out the line:

MAKE_ARGS += ’ -j’ + calc_jobs_number()

(Sidenote: In an attempt to get the BeagleBone installation running I’ve gone back and tried installing from SensorKinect. In that case I had to run RedistMaker manually using:

RedistMaker Arm

This threw up a similar problem with the -j switch. In this case RedistMaker itself is the Python script. This has the line:

make -j

I commented out this line then added another:

End of sidenote).

The, while in the ~/kinect/OpenNI/Platform/Linux/CreateRedist directory, I ran:

# python Redist_OpenNi.py Arm

This appeared to work fine.

Next problem: make install looks for the install.sh script in ~/kinect/OpenNI/Platform/Linux/Redist but in fact it, and other files, are in a further subdir – OpenNI-Bin-Dev-Linux-x64-v1.5.2.23 on the laptop and OpenNI-Bin-Dev-Linux-Arm-v1.5.2.23 on the BeagleBone (obviously, your filename may vary). You could just manually go in there run install.sh, but I copied all the files in that subdir to ~/kinect/OpenNI/Platform/Linux/Redist/. Running make install then worked.

[6] Okay.

[7 – Updated 17 May 2012] Dave says to get Sensor from git clone [Link now broken]. This worked fine on my laptop. Jatin says to go for [Link now broken], which produced fewer errors on the BeagleBoard-XM. I cloned both repos to give me options. The first time I tried this with the BeagleBoard, I went with Sensor. Of course, that didn’t work out so well. So I tried again, this time with SensorKinect. So the comments below refer to Sensor on the Intel Laptop and SensorKinect on the Arm-based BeagelBoard.

[8] cd into Sensor or SensorKinect.

[9] It seems as though the ‘checkout’ process was unnecessary (both versions).

[10] With SensorKinect, the subdir name is ‘Linux’, not ‘Linux-x86’.

[11] Intel: On the Intel laptop: CommonMakefile – which is looked for by the Sensor package – has been renamed CommonCppMakefile. The advice is to cd to /usr/include/ni and create a symbolic link:

# ln -s CommonCppMakefile CommonMakefile

But you might find even CommonCppMakefile is missing from /usr/include/ni, in which case, copy these makefiles from the downloaded OpenNI files:

# cp ~/kinect/OpenNI/Platform/Linux/Build/Common/* /usr/include/ni

I had to edit ~/kinect/Sensor/Platform/Linux-x86/Build/XnFormats/Makefile – the line that says

LIB_USED = XnCore OpenNI

should read

USED_LIBS = XnCore OpenNI

Actually, I just added that extra line. The make then worked, but it still wouldn’t make install.

That’s because it was looking for a file in the dir ~/kinect/Sensor/Platform/Linux-x86/Bin/Release. But the files it was looking for were actually in dirs called x64-Release on the Intel laptop and Arm-Release on the BeagleBone. So on both machines I created a dir called ~/kinect/Sensor/Platform/Linux-x86/Bin/Release and copied (using cp -vR) all the files from x64-Release/Arm-Release into it. Using a symlink would no doubt have been more elegant.

Arm [updated 17 May 2012]: ‘make’ worked fine, but ‘make install’ complained of an unknown architecture. So here’s what I did:

* cd to ~/kinect/SensorKinect/Platform/Linux/CreateRedist

* ran:

chmod u+x RedistMaker

./RedistMaker Arm

cd ~/kinect/SensorKinect/Platform/Linux/Redist/Sensor-Bin-Linux-Arm-v5.1.0.25

chmod u+x install.sh

./install.sh

(Remember, I’m still doing all of this as root.)

According to Jatin, I should now be able to plug in my Kinect and get it working. That didn’t happen. See the ‘next steps’ section below for more details.

[12] The NITE files you’re looking for are now at: [Link now broken].

[13-15] Went fine.

[16] There was no MapOutputMode entry in the XML files.

[17-21] Okay.

[22] Are these binaries? No make && make install necessary.

[25] This subdir & file don’t exist in the version I downloaded & installed.

[26] As detailed in the section below, the SensorKinect package installs its own udev rules, so I didn’t need to do this. But I did edit the SensorKinect version.

Next steps – BeagleBone

On the BeagleBone, when trying to run some of the NITE samples I got:

Open failed: USB interface is not supported!

It wasn’t the gspca_kinect module problem, because I’d already taken the steps below on the BeagleBoard.

Using lsusb tells me that the Kinect is connected. The output includes:

Bus 001 Device 009: ID 045e:02b0 Microsoft Corp. Xbox NUI Motor
Bus 001 Device 010: ID 045e:02ad Microsoft Corp. Xbox NUI Audio
Bus 001 Device 011: ID 045e:02ae Microsoft Corp. Xbox NUI Camera

And dmesg | tail also showed me that the Kinect had connected properly.

[13498.772869] usb 1-1: new high-speed USB device number 3 using musb-hdrc

[13498.913178] usb 1-1: New USB device found, idVendor=0409, idProduct=005a

[13498.913228] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0

[13498.920702] hub 1-1:1.0: USB hub found

[13498.920965] hub 1-1:1.0: 3 ports detected

[13499.502849] usb 1-1.2: new full-speed USB device number 4 using musb-hdrc

[13499.612244] usb 1-1.2: New USB device found, idVendor=045e, idProduct=02b0

[13499.612295] usb 1-1.2: New USB device strings: Mfr=1, Product=2, SerialNumber=0

[13499.612334] usb 1-1.2: Product: Xbox NUI Motor

[13499.612363] usb 1-1.2: Manufacturer: Microsoft

[13501.042841] usb 1-1.1: new high-speed USB device number 5 using musb-hdrc

[13501.145079] usb 1-1.1: New USB device found, idVendor=045e, idProduct=02ad

[13501.145129] usb 1-1.1: New USB device strings: Mfr=1, Product=2, SerialNumber=3

[13501.145171] usb 1-1.1: Product: Xbox Kinect Audio, © 2011 Microsoft Corporation. All rights reserved.

[13501.145211] usb 1-1.1: Manufacturer: Microsoft

[13501.145240] usb 1-1.1: SerialNumber: A44887B03785042A

[13502.833003] usb 1-1.3: new high-speed USB device number 6 using musb-hdrc

[13502.936174] usb 1-1.3: New USB device found, idVendor=045e, idProduct=02ae

[13502.936226] usb 1-1.3: New USB device strings: Mfr=2, Product=1, SerialNumber=3

[13502.936265] usb 1-1.3: Product: Xbox NUI Camera

[13502.936294] usb 1-1.3: Manufacturer: Microsoft

[13502.936321] usb 1-1.3: SerialNumber: A00367914376042A

My first thought was to check udev rules. SensorKinect installs its own rules at /etc/udev/rules.d/55-primesense-usb.rules. I noticed that these effectively duplicated the rules I’d set up manually following Dave’s instructions. I removed the manually installed ones. I also noticed that the SensorKinect rules included the settings: OWNER=“xxx”,GROUP=“users”. I tried changing OWNER to “root” (as that’s how this BeagleBone is going to be run). I also replaced all instances of the deprecated SYFS with ATTR. Then I ran:

/etc/init.d/udev restart

Then I tried editing /usr/etc/primesense/GlobalDefaultsKinect.ini to uncomment the line:

UsbInterface=2

On the Arm platform, UsbInterface defaults to 2, so I thought I’d try it with a value of 1 because I’d read of that working with Android on various Arm-based devices. I tried running the sample program Sample-NiSimpleRead.

It worked!

Then it stopped working. I Ctrl-C’d out of the program after which any attempt to re-run it gave the message the device wasn’t connected. Unplugging and reconnecting didn’t help. I ran lsusb and just got error messages (invalid pointer). Restarting udev didn’t help and subsequent attempts to run the sample program resulted in a segfault. Soooo….

Rebooted the BeagleBone and it worked several times. Then any further attempts to run the sample program returned the repeated message:

UpdateData failed: A timeout has occurred when waiting for new data!

So I’m going to have to look into that.

Another problem: the Kinect isn’t recognised until I unplug it & plug it back in again. Obviously I don’t want to have to do that on the robot, so it would be useful if I could get the system to see it programmatically.

Also, as of this writing, I’m running the BeagleBoard headless, so I can’t see the visual output. However, my DVI cape should arrive tomorrow.

Next steps – Intel laptop

A lot of the Nite stuff had to be altered because of the different version. (Actually, this applied to the BeagleBone installation too – mainly just a problem with some dirs being prefixed with ‘Arm-’).

I hit the error:

Xlib: extension “GLX” missing on display “:0”

This seems to be a freeglut problem – OpenGL GLX extension not supported by display “:0”

I removed Nvidia drivers (I think this involved apt-get remove nvidia-common or some such, but this is the one step I forgot to document). Then I got a USB problem. Was getting the message: