Friday, December 10, 2010

HOWTO: Kinect + OpenNI/NITE skeleton tracking and gesture recognition in gentoo

Thanks to the folks at PrimeSense libraries are now available for skeleton tracking and gesture recognition.
UPDATE: Check here if you've gotten NITE working and want to try using the kinect as a Minority Report style mouse.
UPDATE:I've added a description of how to track multiple hands under the Sample-PointViewer description.

Here's how I got things working in gentoo:


(1) mkdir ~/kinect && cd ~/kinect
(2) git clone https://github.com/OpenNI/OpenNI.git
(3) cd OpenNI/Platform/Linux-x86/Build
(4) make && sudo make install
(5) cd ~/kinect/
(6) git clone https://github.com/boilerbots/Sensor.git
(7) cd Sensor
(8) git checkout kinect
(9) cd Platform/Linux-x86/Build
(10) make && sudo make install
(11) go to this page at openNI to download the latest NITE release for your platform: NITE download page or for the impatient:
32-bit
64-bit
UPDATE: download links now point to openNI and should work again
(12)Save the NITE tarball to ~/kinect and untar it
(13) cd ~/kinect/NITE/Nite-1.3.0.17/Data
(14)Open Sample-User.xml and replace the existing License line with the line below:
NOTE: this is case sensitive!

< License vendor="PrimeSense" key="0KOIk2JeIBYClPWVnMoRKn5cdY4="/>

(15)Repeat step 14 for Sample-Scene.xml and Sample-Tracking.xml
(16)Open Sample-User.xml and replace the existing MapOutputMode line with the line below.
NOTE: this is case sensitive!

< MapOutputMode xRes="640" yRes="480" FPS="30"/>

(17)Repeat step 16 for Sample-Scene.xml and Sample-Tracking.xml
(18)niLicense PrimeSense 0KOIk2JeIBYClPWVnMoRKn5cdY4=
(19)cd ~/kinect/NITE/Nite-1.3.0.17/
(20)sudo ./install.bash
(21)make && sudo make install
(22)cd ~/kinect/NITE/Nite-1.3.0.17/Samples/Bin

Now finally you should be sitting in a directory with all the sample binaries that you can play with. Here's what they should look like:

Sample-TrackPad:
This app will track your hand and show it's relative position on a grid. Run it and wave your hand,one of the squares on the grid should turn yellow to indicate your hand's location as seen below:



you should also get some debug output in your console:



Sample-Players
This app demonstrates the skeletal tracking. After starting it up, move around or wave until your body changes to blue (subsequent players will be other colors, e.g. player 2 is green, 3 yellow,etc.). At this point your viewer window should look vaguely like this:



and you should see something like this in your console:
Look for pose
Found pose "Psi" for user 1


Now, hold your arms out to your sides bent 90 degrees at the elbows as shown below until a skeleton is overlayed on the image of your body:


At this point something like this should have appeared in your console:


Calibration started
Calibration done [1] successfully
Writing 217.596 50 50 78.4388 64.6762
Matching for existing calibration
Read 217.596 50 50 78.4388 64.6762

Sample-SingleControl
This seems to do some sort of gesture recognition and dynamically adjusts the camera resolution, so it's probably zooming in on an area of interest. When it starts out it asks you to perform a focus gesture. The NITE documentation doesn't seem to define what this would be but simply sticking one hand out in front of you seems to make it happy and you'll see the following output:



SamplePointViewer
This app does handtracking. UPDATE: to allow multiple hands to be tracked you will need to edit /usr/etc/primesense/XnVHandGenerator/Nite.ini by uncommenting the two config parameters it contains. Basically remove the semicolons at the start of each line so that Nite.ini looks like this:

[HandTrackerManager]
AllowMultipleHands=1
TrackAdditionalHands=1


To persistently track different hands in your code you can make use of the XnVHandPointContext.nID in your OnPointUpdate callback.



Sample-Boxes
This example allows you to click one of three boxes, your hand motion is tracked by a slider and depending on the context, up, left, right gestures will be recognized.



Sample-CircleControl
Wave to make the border of the window turn green. Then I think you need to send a focus gesture and then if you trace out a circle in the air with your hand the onscreen circle will follow your hand as seen below. In other words if you draw a clockwise circle in the air, the clock hand will also spin clockwise and vice versa. For some reason, this appears to be annoyingly inconsistent.



Sample-SceneAnalysis
This seems to just do player detection without skeleton tracking:

93 comments:

  1. Hi, Thanks for all this. I am attempting to install on Ubuntu 10.10 (64bit). I thought I was home free but at line (19) the make cannot find XnVSessionManager.h which I confirmed is in Nite-1.3.0.17/Include. I am not that experienced with make but I took a look at the Makefile and found that INC_DIR = ../../../Include. I edited this line in various ways thinking the problem might be there; but am still getting the error. Do you have some suggestions... do you want to see the entire output of make. I am running make from Nite-1.3.0.17.

    Thanks for any help.

    ReplyDelete
  2. This comment has been removed by a blog administrator.

    ReplyDelete
  3. circlec, i forgot to include the step where you run install.bash in the NITE/Nite-1.3.0.17/ directory. That script copies all the NITE headers into /usr/include/nite and should fix your problem. I think that install.bash might also obviate the need for the make && sudo make install, but I'm too lazy to verify this. Basically reread the updated steps in my post and you should be good to go.

    ReplyDelete
  4. Ben, Thanks for your help. Oh so close. I am embarrassed to say that I missed a step.. but I went back and rebuilt everything from scratch.. no errors until (21) "make && sudo make install". No missing includes but the last step gave this error:

    make: *** No rule to make target `Install', needed by `install-usb-rules'. Stop.

    All the output was there in Samples/Bin but when I run them this is the error:

    InitFromXml failed: Can't create any node of the requested type!

    Not sure what this is.

    Again thanks for your patience. I know I am out of my depth.It has been years since I worked in C-C++ but this project really interests me. I am an artist and see great potential (stating the obvious) in this..Would love to connect this up to Bender for instance.

    Kent

    ReplyDelete
  5. trtg...Yes that was the step I missed also. If you read my next post you will see another little glitch that I have come upon.

    kent

    ReplyDelete
  6. Hi,
    I went ahead and followed your install instructions which worked out
    smoothly on Ubuntu 9.10 machine too.
    What I try to execute any of the Samples in Nite-1.3.0.17/Samples/Bin
    I do get below errors though. Any idea why so?
    thx, D.

    pangerci@lapradig95:~/ros/ni_cirlcles/Nite-1.3.0.17/Samples/Bin$ sudo
    ./Sample-Boxes
    InitFromXml failed: Can't create any node of the requested type!
    pangerci@lapradig95:~/ros/ni_cirlcles/Nite-1.3.0.17/Samples/Bin$ sudo
    ./Sample-CircleControl
    Couldn't initialize from file: Can't create any node of the requested type!
    pangerci@lapradig95:~/ros/ni_cirlcles/Nite-1.3.0.17/Samples/Bin$ sudo
    ./Sample-Players
    InitFromXml failed: Can't create any node of the requested type!
    pangerci@lapradig95:~/ros/ni_cirlcles/Nite-1.3.0.17/Samples/Bin$ sudo
    ./Sample-PointServer
    usage: ./Sample-PointServer section_name
    pangerci@lapradig95:~/ros/ni_cirlcles/Nite-1.3.0.17/Samples/Bin$ sudo
    ./Sample-PointViewer
    InitFromXmlFile failed: Can't create any node of the requested type!
    pangerci@lapradig95:~/ros/ni_cirlcles/Nite-1.3.0.17/Samples/Bin$ sudo
    ./Sample-SceneAnalysis
    InitFromXml failed: Can't create any node of the requested type!
    pangerci@lapradig95:~/ros/ni_cirlcles/Nite-1.3.0.17/Samples/Bin$ sudo
    ./Sample-SingleControl
    Couldn't initialize: Can't create any node of the requested type!
    pangerci@lapradig95:~/ros/ni_cirlcles/Nite-1.3.0.17/Samples/Bin$ sudo
    ./Sample-TrackPad
    Error initializing: Can't create any node of the requested type!

    ReplyDelete
  7. That looks you've got a few issues going on- for one don't use sudo to run the examples, you don't need to and will probably get errors about freeglut being unable to open the display if you do. So either run the examples as yourself or actually fully switch to root (by using su), but don't use sudo in this case. I also made a mistake where I was showing how to modify the config XML files. SyntaxHighlighter made everything lower case, but those lines are case sensitive, they have to look the way I've shown in the post- e.g. MapOutputMode not mapoutputmode.

    What kind of processor do you have? If you have a pentium 4 or older you will need to change NITE/Nite-1.3.0.17/CommonMakefile by setting SSE_GENERATION=2 instead of 3.

    Are you sure that openNI built successfully in the first few steps?

    What version of GCC are you using? The NITE shared library will only work with >= 4.4 I believe.

    Lastly, what happens if you do:
    niLicense -l
    you should see:

    VENDOR KEY
    ====== ===
    PrimeSense 0KOIk2JeIBYClPWVnMoRKn5cdY4=

    ReplyDelete
  8. Hi trgt,

    I fixed casing problem. For that to take into effect only proceeding steps have to be redone right?

    My processor is Intel(R) Core(TM)2 Duo CPU T9600 @ 2.80GHz.

    gcc --version
    gcc (Ubuntu 4.4.1-4ubuntu9) 4.4.1

    I only now noticed that the step 21 indeed failed at copying includes:
    http://pastie.org/1371520
    Are those actually important?

    Last but not least, I do not have the same directory structure as you. Are the absolute paths assumed at some moment?

    thx a lot for your help, D.

    ReplyDelete
  9. I forgot to add, but the output of niLicense -l seems to be correct:
    niLicense -l
    VENDOR KEY
    ====== ===
    PrimeSense 0KOIk2JeIBYClPWVnMoRKn5cdY4=

    ReplyDelete
  10. Hi trtd, Like dejanpan, I also have made all the suggested changes.
    I also get:
    niLicense -l
    VENDOR KEY
    ====== ===
    PrimeSense 0KOIk2JeIBYClPWVnMoRKn5cdY4=

    I get the following error for step (21) "sudo make install":
    make: *** No rule to make target `Install', needed by `install-usb-rules'. Stop.

    I noticed that in OpenNI discussions group people trying to build for Windows are getting the same error I am getting when I run Sample programs:

    InitFromXml failed: Can't create any node of the requested type!

    Someone seems to solve the problem by removing the other Kincect Drivers he had installed on his system and starting over. I also have installed libfreenect drivers on my Ubuntu 10.10 box and successfully accessed the Kinect.

    I am not sure how to uninstall them... perhaps this is the problem in my case.

    Thanks again for the help, trtg.

    kent

    ReplyDelete
  11. This comment has been removed by the author.

    ReplyDelete
  12. I have run uninstall.bash followed by install.bash.
    The directories:

    /usr/include/nite
    /usr/include/ni
    /usr/etc/primesense

    all exist and are populated by many files.

    Here is the PROBLEM:
    These files:
    usr/lib/XnFormats.so
    /usr/lib/XnDeviceSensorV2.so
    /usr/lib/XnDeviceFile.so
    /usr/lib/XnDDK.so
    /usr/lib/XnCore.so

    are NOT there or on the system at all.


    My install directory is ~kinect-openNI and I believe I have been consistent in its use.

    kent

    ReplyDelete
  13. Hi again,

    maybe you've already seen this, but if you're getting "InitFromXml" error, Radu Bogdan Rusu says that

    * either ldd on the bins/libs will tell you that stuff is not linked properly. As far as I know, the only way to fix
    that problem for relative directories is the way we did it in the ROS metapackages;

    * either that sample needs an .ini/.xml configuration file badly, and it can't find it because it has an absolute path
    hardcoded. Again, you can check the fixes for ps_engine/openni in the ROS metapackages.

    see this link for the discussion:

    http://groups.google.com/group/openni-dev/browse_thread/thread/ee33b530b2c854cf/eefb7a5a475c83d7#eefb7a5a475c83d7

    Ben

    ReplyDelete
  14. circlec, if those libraries are missing from your system it means that you did not correctly install Sensor, in other words something between steps 6 and 10 went wrong. In particular step 10 is where those libraries would normally get copied into /usr/lib/
    What happened on your system during step 10?

    Note that I did not have to do anything with ROS to get things working, you should NOT have to do any of the things that Ben has mentioned. The whole point of this post is to avoid having to go through the even more involved process of installing ROS. Perhaps on ubuntu installing ROS is easy but on gentoo it is non-trivial.

    ReplyDelete
  15. Hi trtg,

    it seems that on my system (Ubuntu 9.10) the libraries are called:
    /usr/lib/libXnFormats.so
    /usr/lib/libXnDeviceSensorV2.so
    /usr/lib/libXnDeviceFile.so
    /usr/lib/libXnDDK.so
    /usr/lib/libXnCore.so
    Could this be a problem maybe?

    D.

    ReplyDelete
  16. Hi tgrt,
    one more question, as per instruction step (8) git checkout kinect - this is suppose to switch to branch kinect in the repository, but I only have a master branch. Is there something missing?

    cheers, d.

    ReplyDelete
  17. Hi again,
    I'm using Ubuntu 10.04 32-bit and I followed your steps (yes, without using anything related to ROS) and I'm getting exactly the same error with dejanpan.

    And, instead of having

    /usr/lib/XnFormats.so
    /usr/lib/XnDeviceSensorV2.so
    /usr/lib/XnDeviceFile.so
    /usr/lib/XnDDK.so
    /usr/lib/XnCore.so

    I, too have the following shared libraries

    /usr/lib/libXnFormats.so
    /usr/lib/libXnDeviceSensorV2.so
    /usr/lib/libXnDeviceFile.so
    /usr/lib/libXnDDK.so
    /usr/lib/libXnCore.so

    Another thing is, for ./Sample-Boxes I get:

    InitFromXml failed: Can't create any node of the requested type!

    However for ./Sample-Trackpad I get:

    Error initializing: Can't create any node of the requested type!

    I don't know the difference between InitFromXml error and Error initializing error, but probably they are both caused by the same problem...

    I just thought I should share...

    Ben

    ReplyDelete
  18. Sorry for the confusion regarding my earlier comment. In the process of copying and pasting I inadvertently mangled the library names. They should indeed have a preceding "lib" in the name. My earlier comment (which I've deleted) should have looked like what I've repasted below.


    If you ran the sudo install.bash in step 20, you should probably be okay as far as header files because that should have created a /usr/include/nite/ directory and copied all the headers there. Make sure that the following directories exist for you to verify that the install didn't get messed up:

    /usr/include/nite
    /usr/include/ni
    /usr/etc/primesense

    Make sure all those directories exist and have files in them. Also make sure that these files exist:
    /usr/lib/libXnFormats.so
    /usr/lib/libXnDeviceSensorV2.so
    /usr/lib/libXnDeviceFile.so
    /usr/lib/libXnDDK.so
    /usr/lib/libXnCore.so

    Note that wherever I used the directory ~/kinect/ you could have used whatever directory you want instead, just make sure you use it consistently through all the steps.

    ReplyDelete
  19. dejanpan, if you did not successfully switch Sensor to the kinect branch, nothing will work at all. Unless master picked up boilerbots changes recently, master does not have any support for the kinect. So in ~/kinect/Sensor (or your equivalent directory) if you type
    git branch
    you should see this:
    * kinect
    master

    Probably the last bits of advice I can give you are as follows:
    try running ~/kinect/OpenNI/Platform/Linux-x86/Bin/Release/NiViewer

    if that gives you the same sort of InitFromXml error as before you'll have to track down which node is not getting enumerated and why. A good first step would be to build the samples with debugging enabled, do
    gdb NiViewer
    then set a breakpoint in
    ~/kinect/OpenNI/Samples/NiViewer/NiViewer.cpp around line 592 which should be inside an if statement that looks like:

    if (rc == XN_STATUS_NO_NODE_PRESENT)
    {
    XnChar strError[1024]; errors.ToString(strError, 1024);
    printf("%s\n", strError);
    closeSample(ERR_DEVICE);
    return (rc);
    }

    You'll need to check the call stack when you hit this breakpoint and work your way backward to figure out what node is failing to enumerate.
    This process isn't particularly difficult but it would be very tedious for me to walk you through it, so I'm afraid this is probably the extent of assistance I'll be able to give you.

    ReplyDelete
  20. Ben, sorry for doing this but I deleted one of your earlier comments because people seemed to think that I had written it and were thus getting even more confused.

    ReplyDelete
  21. Oh that's completely fine. It's good that you deleted it, we want people to get things working right?

    Don't worry :)

    ReplyDelete
  22. Dear trgt,
    I indeed only get the master branch cloned from the Sensor repo. I'll have to check it tomorrow why is it so.
    I hope that this will then do it.

    ReplyDelete
  23. Hi tgrt,
    I solved the issue with kinect branch, had to "git checkout -b kinect origin/kinect" it.
    D.

    ReplyDelete
  24. hi tgrt, it seems that the action above brought me a bit further. I however am getting the following error now:

    pangerci@lapradig95:~/kinect/NITE/Nite-1.3.0.17/Samples/Bin$ ./Sample-PointViewer
    Setting resolution to QVGA
    Warning: USB events thread - failed to set priority. This might cause loss of data...
    InitFromXmlFile failed: Device Protocol: Bad Parameter sent!

    D.

    ReplyDelete
  25. dejanpan, that error message suggests you didn't change the MapOutputMode line as described in steps 16 and 17. The change needs to be made for the config files of all samples that you want to run. There are 3 config files for the NITE examples: Sample-User,Sample-Scene.xml, and Sample-Tracking.xml. Just make the same change in all of them- resolution needs to be set to 640 x 480.

    ReplyDelete
  26. Hi trtg,
    yup that was it. Now it all works like a charm, thx a lot for helping me out.
    D.

    ReplyDelete
  27. I've got it working and tried all the samples, but I wasn't able to get the skeleton overlayed to my body despite many trials holding both arms up. Any hint on the right pose to calibrate?

    ReplyDelete
  28. Hantarex, note that only the Sample-Players example will demo skeletal tracking (as far as I know the scene analyis one will not). If you look back at my post I've add a picture which should clarify the calibration pose. If the console output indicates that calibration was unsuccessful you may need to move further away or closer, restart the example, wave again and hold the calibration pose again.

    ReplyDelete
  29. This comment has been removed by the author.

    ReplyDelete
  30. Hey,
    thx! your post really helped me installing a kinect camera and getting things work.
    Using Ubuntu 10.10 I had to install a few additional librarys to be able to compile and install the drivers and openNI.
    On http://www.felix-huelsmann.de/Blog/MSKinectmitopenNIunterUbuntuLinux.php I postet a list containing necessary librarys and so on (in German). Maybe that helps someone.

    Does anyone have experiences installing the kinect stuff on Ubuntu 10.04 in an Oracle Virtual Box? Installation works, but I do not receive any signal although I gave my VirtualMachine access to my USB-Ports,

    ReplyDelete
  31. Hi,
    I followed your guide and everything seems to be installed right.

    But when I launch the sample programs it says

    Warning: USB events thread - failed to set priority. This might cause loss of data...

    and then nothing happens.
    The program is blocked and I must kill it.

    I'm working on Ubuntu with VMWare Fusion, could it be the problem?
    I also installed freenect and it works very well with kinect...

    ReplyDelete
  32. Muffo, the "failed to set priority" warning can be eliminated by running the program as root. Note that I'm referring to fully switching to root via su. If you just use sudo you might have problems opening an x window. As far as VMWare is concerned, I can't help you. I did all this on a native linux system. I have no clue what sorts of issues using VMWare might present.

    ReplyDelete
  33. hi all
    i've tryed do instale nite but linux return me this error

    miss operand i think....

    make[1]: uscita dalla directory «/home/matteo/kinect/NITE/Nite-1.3.0.17/Samples/Players»
    cp /usr/local/include
    cp: manca l'operando per il file di destinazione dopo `/usr/local/include'
    Usare `cp --help' per ulteriori informazioni.
    make: *** [install-includes] Errore 1

    ReplyDelete
  34. Regarding the multiple hand tracking:

    You have to uncomment the two lines in

    /usr/etc/primesense/XnVHandGenerator/Nite.ini

    and it should work :-)

    Have fun!

    ReplyDelete
  35. Matteo, try just running install.bash and forget about the make command. If you'd rather use the makefile you'll need to edit the Makefile and make the following change INST_INC=/usr/include/

    ReplyDelete
  36. Firstname, thanks for the tip, I was planning to add that as part of an MPX cursor control example, but in the meantime I've added it to the Sample-PointViewer description above.

    ReplyDelete
  37. Hi,
    I followed your guide to install Kinect + OpenNI/NITE on Ububuntu 10.10.
    Every think works fine except step 21.
    I get the following error: cp: missing destination file operand after `/usr/include'.

    Cause you mentioned that this is maybe unnecessary i've still tried the samples.
    But when I launch them the programs says either "Setting resolution to VGA" and then nothing happens or nothing happens anyway.

    All mentioned files exists, sensor branch is set to kinect and niLicense -l returns the right key.

    Any idea?

    ReplyDelete
  38. Paul, is "Setting resolution to VGA" the only output you see? No error messages? For some of the examples, if you see "Setting resolution to VGA" it means the hand or feature tracking is maximizing its field of view because it can't see you or your hands. Waving(at least 5 times) or doing a focus/click gesture should grab the kinect's attention. Also, are you using Ubuntu natively, or running under VMWare? If using VMWare, I have no clue- all sorts of stuff could be going wrong.

    ReplyDelete
  39. Paul, one last possibility if you tried to run the examples using sudo, depending on how your system is configured you may not be able to open an X window. If this is the case you should either run the examples as yourself or as full-on root but not by using sudo. Normally, however, if this is the problem you're encountering you should see an error message in your console saying something like "Unable to open display".

    ReplyDelete
  40. Trtg, yes "Setting resolution to VGA" is the only output. I´ve tried waving, sudo, su and my own account. If i run the examples with my account nothing happens and with all other i get "Setting resolution to VGA" and then nothing happens.

    So the problem must be using VMWare.

    ReplyDelete
  41. Hi, I would like to report that I finally successfully compiled the code on Ubuntu 10.10 using the instructions above. I am not sure what I did this time around... I was unable to do it two weeks ago after considerable effort and frustration.. but today everything went well and all the examples are running now. I did omit STEP 21.

    thanks for your help
    k

    ReplyDelete
  42. awesome.. followed exact same steps, and works perfect on Ubuntu 10.10

    ReplyDelete
  43. dose any one know what this means and how to fix it?

    --

    registering module 'libXnDeviceSensorV2.so' with OpenNI..../install.sh: line 57: 16036 Illegal instruction niReg -r $INSTALL_LIB/$module $INSTALL_ETC
    /home/william/kinect/Sensor/Platform/Linux-x86/Build
    --

    ReplyDelete
  44. This comment has been removed by the author.

    ReplyDelete
  45. william, that error just says your system doesn't have a /usr/local/include/ directory. This was discussed earlier, just run install.bash and forget about the make command and you should be fine.

    ReplyDelete
  46. how do i run the sample-trackpad file? i went into the directory and double clicked it but it doesn't open.

    ReplyDelete
  47. Hi,
    installation went fine (Ubuntu 10.10) omitting step 21. But running the examples I got an 'illegal instruction' error. Any idea how I could fix this? Thanks in advance.

    ReplyDelete
  48. Hi all,
    openni.org is not available since two day, so I can't do step 11 to 21. Some examples are present in OpenNi/Platform/Linux-x86/Bin/Release/.

    When executing Sample-NiUserTracker I get this message :
    Find user generator failed: Can't create any node of the requested type!

    (NiViewer work fine);

    Could someone tell me what exacly Nite is for ?
    Can we execute Sample-NiUserTracker without it ?

    Thanks.
    Damien

    ReplyDelete
  49. Hi all.
    Its me againe. After using kinect + openni under vmware with ubuntu 10 doesn´t work. I set up an native ubuntu and followed this guide again. Now it works nearly perfect. Only the Sample-TrackPad example doesn´t work. The program outputs "Setting resolution to VGA" opens a new black window and then nothing happens.
    I´ve tried waving my hands but the windows still remains black. Any ideas?

    ReplyDelete
  50. The links for: NITE 1.3 Beta build 17 SDK for Linux x64
    &
    NITE 1.3 Beta build 17 SDK for Linux x86

    Are down, could someone upload that to somewhere? I'm stuck here and I would like to try this out!

    ReplyDelete
  51. Ros, not sure if it helps you, but I've mirrored the 32-bit version of NITE here:

    https://dl.dropbox.com/u/11217419/NITE-Bin-Ubuntu-x86-1.3.0.17.tar.bz2

    Just a quick disclaimer- I'm not associated with PrimeSense, I simply uploaded this file to help those who might need it. It is an unmodified copy of the original. Sorry, I don't have the 64-bit version.

    ReplyDelete
  52. For people getting illegal instruction errors:
    By default the makefile works for sse3 instructions, if your proc supports sse2 but not sse3 you have to tweak the make file :

    in
    .../Platform/Linux-x86/Build/CommonMakefile
    change SSE_GENERATION from 3 to 2

    ReplyDelete
  53. I'm getting the Illegal Instruction error on 64-bit Ubuntu. I tried changing the SSE_GENERATION to 2 but still yielded the same results. Any one got any ideas?

    ReplyDelete
  54. Johnny V8, if you have a 64 bit processor it's quite likely new enough to support SSE3. You could check by doing:

    cat /proc/cpuinfo | grep -i sse

    Only other thing I can think of is make sure you're using the 64-bit version of NITE.

    ReplyDelete
  55. Don't know if this helps to diagnose the problem at all:
    Program received signal SIGILL, Illegal instruction.
    0x00007ffff39a3313 in NAGeneralData::Downscale2x2SSE(unsigned short const*, unsigned short*, short, short, Box2D*) () from /usr/lib/libXnVHandGenerator.so

    ReplyDelete
  56. @trtg Thanks for the help. Turns out it must be a bug in the current stable version. I pulled down the unstable 1.3.0.18 and it seemed to clear up the issue. Strange?!?!

    ReplyDelete
  57. ./Sample-PointViewer works but i can't get the ./Sample-TrackPad working properly - i get the "Setting resolution to VGA" output & the screen shows up but stays black without showing a grid...
    similar with ./Sample-Boxes - i also cannot click on the boxes there.
    thanks for help

    ReplyDelete
  58. sesselastronaut, you should wave your hand like in the Sample-PointViewer.

    ReplyDelete
  59. Hi guy, I need your help,

    After the first Make && sudo Make install i got this error:


    ../../../../Source/OpenNI/Linux-x86/XnUSBLinux-x86.cpp:30:31: error: libusb-1.0/libusb.h: Nessun file o directory
    In file included from ../../../../Source/OpenNI/Linux-x86/XnUSBLinux-x86.cpp:31:
    ../../../../Source/OpenNI/Linux-x86/XnUSBLinux-x86.h:46: error: ISO C++ forbids declaration of ‘libusb_device_handle’ with no type
    ../../../../Source/OpenNI/Linux-x86/XnUSBLinux-x86.h:46: error: expected ‘;’ before ‘*’ token
    .
    .
    .
    .
    .
    make[1]: *** [Release/XnUSBLinux-x86.o] Errore 1
    make[1]: uscita dalla directory «/home/omar/kinect/OpenNI/Platform/Linux-x86/Build/OpenNI»
    make: *** [OpenNI] Errore 2


    I think i have some problem with the libusb.h because i don't have it... what should i do?

    ReplyDelete
  60. Denvoizer, openNI's sensor module uses libusb for its backend - you must have libusb installed in order to use NITE. I don't know what distro of linux you're using but it should almost certainly have a libusb package available, either install that or get the source code for the latest libusb and install it that way.

    ReplyDelete
  61. problem solved :D
    I had to install this :

    sudo apt-get install git-core cmake libglut3-dev pkg-config build-essential libxmu-dev libxi-dev libusb-1.0-0-dev doxygen graphviz

    ReplyDelete
  62. Sorry trtg, I didn't see your answer. But thank you verrt much though.
    and thanks to all of you guys...
    but I have another problem :(
    when i tried to install it in my desktop (ubuntu 10.04) and at the point : niLicense ... it gives me an :illegal instruction

    same thing with : sudo ./install , : ./install.bash: riga 9: 10080 illegal instruction niReg /usr/lib/libXnVFeatures.so /usr/etc/primesense/XnVFeatures

    thank you guys! you are the best!

    ReplyDelete
  63. ej sleepmancer works now flawlessly - thanks for the indication!

    ReplyDelete
  64. to denvoizer :
    check if your cpu supports ssse3 :

    cat /proc/cpuinfo | grep ssse3

    Nite requires ssse3 support, you can compile OpenNi without it, but you are stuck with Nite as it is a binary

    Hope that helps

    ReplyDelete
  65. Just to say that I followed these instructions and got things working under Debian 6.0 (Squeeze). The pose detection for the skeleton seems a bit hit and miss - sometimes it'll spot me doing the pose and then track the skeleton fine, whereas at other times I can get stiff arms trying to hold the pose whilst it does nothing. Impressive when it works though - thanks for the conf instructions!

    ReplyDelete
  66. Solution for find user generator failed: can’t create node of the requested type...

    After days of "merde" i found how to use all the samples not only simple viewer and Niviewer ....

    Change all the .xml files in the openNI file (in the the subfolder sensor and nite)
    Open them all with notepad and replace all the information off licence and resolution... save and that it !!!!

    ReplyDelete
  67. Replacing the lines as in step 14 and 15 with:

    < License vendor="PrimeSense" key="0KOIk2JeIBYClPWVnMoRKn5cdY4="/>

    and

    < MapOutputMode xRes="640" yRes="480" FPS="30"/>

    gave me, on Ubuntu 10.10, the error:

    InitFromXmlFile failed: Device Protocol: Bad Parameter sent!

    I had to remove the little spaces after the first bracket in order for it to work, after doing that there was no errors (probably xml parse errors). The reason why they are writting like this in this blog is probably because Blogger wont display them if they are written properly in the editor. It's fairly easy to make this mistake if you are just copying everything verbatim.

    Really neat stuff! Thanks for the HOWTO!

    Tom

    ReplyDelete
  68. Hey thanks a lot for the HOWTO. Worked perfectly for me. Except I had to use sudo to run the examples initially because of "libusb couldn't open USB device /dev/bus/usb/002/005: Permission denied.". But otherwise, great :-).

    ReplyDelete
  69. I have a strange problem I haven't been able to track down. I had this all working under Ubuntu 10.04 and life was great :-) I then installed the RGBDemo
    from http://nicolas.burrus.name/index.php/Research/KinectRgbDemoV5 Unfortunately, this broke the openNI I had installed (the RGBDemo also uses this, so I know something got installed that clobbered by earlier installation). The error I get when I try to run any of the Sample-* programs is
    InitFromXmlFile failed: OpenNI library can't find any module!

    When I try to reinstall the openNI from scratch, the niReg steps fail with a
    Failed: Failed to write to the file!

    I haven't been able to track down the conflict yet :-(
    As I continue to try, I was hoping that maybe some other Kinect/Ubuntu user had done this and figured out the solution.

    Thanks,
    kenny
    kmoorman@transy.edu

    ReplyDelete
  70. Hi,

    I'm trying to install NITE by typing ./install.sh but I'm getting the following error:

    root@padawan:~/kinect/Nite# ./install.sh
    Package exported to: /usr/lib/mono/2.0/XnVNite.net_1_3_1.dll -> ../gac/XnVNite.net_1_3_1/1.3.1.3__399870656fc6c787/XnVNite.net_1_3_1.dll
    Installed Bin/XnVNite.net_1_3_1.dll into the gac (/usr/lib/mono/gac)
    Please enter your PrimeSense license key: 0KOIk2JeIBYClPWVnMoRKn5cdY4=
    make -C Samples/PointServer CFG=Release -f PointServer.mak ; make -C Samples/SingleControl CFG=Release -f SingleControl.mak ; make -C Samples/PointViewer CFG=Release -f PointViewer.mak ; make -C Samples/Boxes CFG=Release -f Boxes.mak ; make -C Samples/TrackPad CFG=Release -f TrackPad.mak ; make -C Samples/CircleControl CFG=Release -f CircleControl.mak ; make -C Samples/SceneAnalysis CFG=Release -f SceneAnalysis.mak ; make -C Samples/Players CFG=Release -f Players.mak ; make -C Samples/Boxes.net CFG=Release -f Boxes.net.mak ;
    make[1]: Entering directory `/media/5074D70A74D6F228/Documentos/Mestrado/IME/Tese/Software/OpenNI+NITE+Sensor/Nite-1.3.1.3/Samples/PointServer'
    g++ -malign-double -O3 -fno-tree-pre -fno-strict-aliasing -DNDEBUG -msse3 -mssse3 -I/usr/include/nite -I/usr/include/ni -DXN_SSE -M -MF Release/main.d -MT "./Release/main.o Release/main.d" ../PointServer/main.cpp
    make[1]: Leaving directory `/media/5074D70A74D6F228/Documentos/Mestrado/IME/Tese/Software/OpenNI+NITE+Sensor/Nite-1.3.1.3/Samples/PointServer'
    make[1]: Entering directory `/media/5074D70A74D6F228/Documentos/Mestrado/IME/Tese/Software/OpenNI+NITE+Sensor/Nite-1.3.1.3/Samples/PointServer'
    g++ -c -malign-double -O3 -fno-tree-pre -fno-strict-aliasing -DNDEBUG -msse3 -mssse3 -I/usr/include/nite -I/usr/include/ni -DXN_SSE -o Release/main.o ../PointServer/main.cpp
    g++ -o ../Bin/Sample-PointServer ./Release/main.o ./Release/signal_catch.o ./Release/kbhit.o -L../Bin -lOpenNI -lXnVNite_1_3_1
    /usr/lib/gcc/i486-linux-gnu/4.4.3/../../../../lib/libXnVNite_1_3_1.so: undefined reference to `std::_List_node_base::_M_unhook()@GLIBCXX_3.4.14'
    /usr/lib/gcc/i486-linux-gnu/4.4.3/../../../../lib/libXnVNite_1_3_1.so: undefined reference to `std::_List_node_base::_M_hook(std::_List_node_base*)@GLIBCXX_3.4.14'
    collect2: ld returned 1 exit status
    make[1]: *** [../Bin/Sample-PointServer] Error 1

    How can I fix it?

    Regards,
    Caio

    ReplyDelete
  71. I have the same problem too,thanks to help us

    ReplyDelete
  72. i cant even get install.sh running, it says in line 20, its an illegal instruction. What do i do from there?

    ReplyDelete
  73. @Caio
    I had the same problem and I have it working now. The last change that I made before it started working was match the OS type for both NITE and OpenNI. Earlier, I had OpenNI 32-bit installed and NITE 64-bit. Once I installed NITE 32-bit, install.sh started working fine. Make also runs fine (step 21), but make install doesn't.

    Most of the samples are now running fine for me, except for niusertracker which I'm currently trying to fix.

    Hope this helps.

    ReplyDelete
  74. @Caio
    Oh and I also uninstalled libfreenect and re-built Sensor and OpenNI

    ReplyDelete
  75. Hello,

    I am still having trouble with the installation. My version of NITE, 1.3.1.3 does not have MapOutputMode values in the three files specified in instructions 16/17. On Ubuntu 11.xxxx

    So, I'm searching elsewhere for the resolution, or will have to install an older version of nite. Any suggestions?

    ReplyDelete
  76. Same problem as Caio here: when I run NITE's install.sh I get the errors:
    /usr/lib/gcc/i486-linux-gnu/4.4.3/../../../../lib/libXnVNite_1_3_1.so: undefined reference to `std::_List_node_base::_M_unhook()@GLIBCXX_3.4.14'
    /usr/lib/gcc/i486-linux-gnu/4.4.3/../../../../lib/libXnVNite_1_3_1.so: undefined reference to `std::_List_node_base::_M_hook(std::_List_node_base*)@GLIBCXX_3.4.14'
    collect2: ld returned 1 exit status
    make[1]: *** [../Bin/Sample-CircleControl] Error 1
    make[1]: Leaving directory `/media/DATA/downloads/programmi/openni/nite/Nite-1.3.1.3/Samples/CircleControl'
    make[1]: Entering directory `/media/DATA/downloads/programmi/openni/nite/Nite-1.3.1.3/Samples/SceneAnalysis'
    g++ -o ../Bin/Sample-SceneAnalysis ./Release/main.o ./Release/SceneDrawer.o ./Release/kbhit.o ./Release/signal_catch.o -L../Bin -lglut -lOpenNI -lXnVNite_1_3_1
    /usr/lib/gcc/i486-linux-gnu/4.4.3/../../../../lib/libXnVNite_1_3_1.so: undefined reference to `std::_List_node_base::_M_unhook()@GLIBCXX_3.4.14'
    /usr/lib/gcc/i486-linux-gnu/4.4.3/../../../../lib/libXnVNite_1_3_1.so: undefined reference to `std::_List_node_base::_M_hook(std::_List_node_base*)@GLIBCXX_3.4.14'
    collect2: ld returned 1 exit status
    make[1]: *** [../Bin/Sample-SceneAnalysis] Error 1
    make[1]: Leaving directory `/media/DATA/downloads/programmi/openni/nite/Nite-1.3.1.3/Samples/SceneAnalysis'
    make[1]: Entering directory `/media/DATA/downloads/programmi/openni/nite/Nite-1.3.1.3/Samples/Players'
    g++ -o ../Bin/Sample-Players ./Release/main.o ./Release/SceneDrawer.o ./Release/kbhit.o ./Release/signal_catch.o -L../Bin -lglut -lOpenNI -lXnVNite_1_3_1
    /usr/lib/gcc/i486-linux-gnu/4.4.3/../../../../lib/libXnVNite_1_3_1.so: undefined reference to `std::_List_node_base::_M_unhook()@GLIBCXX_3.4.14'
    /usr/lib/gcc/i486-linux-gnu/4.4.3/../../../../lib/libXnVNite_1_3_1.so: undefined reference to `std::_List_node_base::_M_hook(std::_List_node_base*)@GLIBCXX_3.4.14'
    collect2: ld returned 1 exit status
    make[1]: *** [../Bin/Sample-Players] Error 1
    make[1]: Leaving directory `/media/DATA/downloads/programmi/openni/nite/Nite-1.3.1.3/Samples/Players'
    make: *** [all] Error 2
    teo@XXX:/media/DATA/downloads/programmi/openni/nite/Nite-1.3.1.3$

    Any idea?

    ReplyDelete
  77. (16)Open Sample-User.xml and replace the existing MapOutputMode line with the line below.
    NOTE: this is case sensitive!

    < MapOutputMode xRes="640" yRes="480" FPS="30"/>

    (17)Repeat step 16 for Sample-Scene.xml and Sample-Tracking.xml

    None of those files have any line nor anything vaguely similar

    ReplyDelete
  78. Oh the usual comment-parsing *@€#! I wrote "any XXX line" where XXX was the abovementioned xml tag "MapOutputMode"

    ReplyDelete
  79. Does anyone know where to get NITE from? The links doesn't work :(

    ReplyDelete
  80. Hi

    Real nice documentation. However, I get a problem at step 11 during the install process for the sensor. When I run make && sudo make install, it gives:
    make: Entering directory `/home/—–/kinect/Sensor/Platform/Linux-x86/Build’
    make -C XnCore CFG=Release
    make[1]: Entering directory `/home/—–/kinect/Sensor/Platform/Linux-x86/Build/XnCore’
    ../EngineLibMakefile:24: /usr/include/ni/CommonMakefile: No such file or directory
    make[1]: *** No rule to make target `/usr/include/ni/CommonMakefile’. Stop.
    make[1]: Leaving directory `/home/——/kinect/Sensor/Platform/Linux-x86/Build/XnCore’
    make: *** [XnCore] Error 2
    make: Leaving directory `/home/——/kinect/Sensor/Platform/Linux-x86/Build’

    From a different post they had advised to provide a link from file /usr/include/ni/CommonMakefile to /usr/include/ni/CommonCppMakefile. I did this using:
    cd /usr/include/ni
    sudo ln -s CommonCppMakefile CommonMakefile
    However, after this when I try the installation(make && sudo make install) it gives this:
    …….
    …….
    ./Release/jutils.o: In function `jcopy_sample_rows’:
    jutils.c:(.text+0x9c): undefined reference to `xnOSMemCopy’
    ./Release/jutils.o: In function `jzero_far’:
    jutils.c:(.text+0×36): undefined reference to `xnOSMemSet’
    ./Release/jutils.o: In function `jcopy_block_row’:
    jutils.c:(.text+0×50): undefined reference to `xnOSMemCopy’
    collect2: ld returned 1 exit status
    make[1]: *** [../../Bin/Release/libXnFormats.so] Error 1
    make[1]: Leaving directory `/home/——/kinect/Sensor/Platform/Linux-x86/Build/XnFormats’
    make: *** [XnFormats] Error 2
    make: Leaving directory `/home/—–/kinect/Sensor/Platform/Linux-x86/Build’

    Any ideas whats wrong?
    I tried installing the sensor modules from the OpenNI repo and continued with the remaining installation but the final release binaries did not work and kept saying ‘InitXmlfiles failed…..’
    For your info:
    OS: Linux 10.10
    Pro: intel xenon

    ReplyDelete
  81. hi,
    I have the same problem as Anon. Can you help please?

    ReplyDelete
  82. Hi, we solve the problem. It is really tricky.
    Makefile in /Sensor/Platform/Linux-x86/Build/XnFormat should be changed like this:

    SRC_FILES = \
    ../../../../Source/XnFormats/*.cpp \
    ../../../../Source/External/LibJPEG/*.c

    LIB_NAME = XnFormats
    USED_LIBS = XnCore OpenNI
    DEFINES = XN_FORMATS_EXPORTS
    INC_DIRS = ../../../../Source/External/LibJPEG

    include ../EngineLibMakefile

    in the original file it looks like this:

    SRC_FILES = \
    ../../../../Source/XnFormats/*.cpp \
    ../../../../Source/External/LibJPEG/*.c

    LIB_NAME = XnFormats
    LIB_USED = XnCore OpenNI
    DEFINES = XN_FORMATS_EXPORTS
    INC_DIRS = ../../../../Source/External/LibJPEG

    include ../EngineLibMakefile

    somehow it works for me

    ReplyDelete
  83. Hi,the new NITE downloadlink is here http://www.openni.org/downloadfiles/opennimodules/openni-compliant-middleware-binaries/latest-unstable/175-primesense-nite-unstable-build-for-ubuntu-10-10-x64-64-bit-v1-4-1

    ReplyDelete
  84. Hey everyone,
    I can't do this step:

    git clone https://github.com/boilerbots/Sensor.git

    I get this error:

    fatal: could not switch to 'http://github.com/boilertbots': no such file or directory.

    ReplyDelete
  85. Another think, I can't find the line MapOutputMode to change the values. In the files that the tut mention that line is not there.

    ReplyDelete
  86. Hi Best.....

    I am also facing the same problem. The latest NITE doesn't contain < MapOutputMode> line. What shall we do.

    Can any body send us the content of old file so that we may update it in ours.

    Same black screen problem is here.

    ReplyDelete
  87. Hi,
    In case you're running gentoo you might be interested in
    my (wip) ebuilds for openni, openni-sensor, nite and lib2realkinectwrapper which I just comitted to the gamerlay overlay. build for me but haven't tested yet.

    ReplyDelete
  88. Has anyone still solve the following problem. I did the what previous step said i am still getting the same error


    Real nice documentation. However, I get a problem at step 11 during the install process for the sensor. When I run make && sudo make install, it gives:
    make: Entering directory `/home/—–/kinect/Sensor/Platform/Linux-x86/Build’
    make -C XnCore CFG=Release
    make[1]: Entering directory `/home/—–/kinect/Sensor/Platform/Linux-x86/Build/XnCore’
    ../EngineLibMakefile:24: /usr/include/ni/CommonMakefile: No such file or directory
    make[1]: *** No rule to make target `/usr/include/ni/CommonMakefile’. Stop.
    make[1]: Leaving directory `/home/——/kinect/Sensor/Platform/Linux-x86/Build/XnCore’
    make: *** [XnCore] Error 2
    make: Leaving directory `/home/——/kinect/Sensor/Platform/Linux-x86/Build’

    ReplyDelete
  89. Same as above, can't find that CommonMakefile

    ReplyDelete
  90. Hello, maybe my question may result kinda stupid, but: How do I compile the samples or even my own code (OpenNI only) in Linux with g++?
    Most of documentation files in the web are for visual studio and the documentation of OpenNI is no help on that sense.
    Thanks a lot

    ReplyDelete
  91. Hello all,
    i followed the instructions and i installed Nite, Openni and sensor Kinect on ubuntu 11.4 but When i tryed running examples everything works fine except the sample-Players, it gives me this error:

    355 INFO New log started on 2012-01-15 15:56:18
    397 INFO OpenNI version is 1.5.2 (Build 23)-Linux-x86 (Dec 28 2011 17:53:48)
    408 INFO --- Filter Info --- Minimum Severity: UNKNOWN
    InitFromXml failed: This operation is invalid!

    Do you have any idea how to fix it ?
    Thanks in advance..

    ReplyDelete
  92. I am having the same problem. Does anybody have a solution?

    Thank you

    ReplyDelete