I’ve written a ROS node for the Nuitrack SDK, which publishes 2D and 3D tracking info, and provides markers in RVIZ. Code posted on Github at Shinsel Robots · GitHub
Videos:
Sheldon Robot:
Turtlebot:
Overview:
Simulation in RVIZ:
Following me around the house
Code for Nuitrack ROS Node:
Hello dshinsel
I am installing your ROS node for the Nuitrack skeletal tracking (trial version). I am getting an error in finding " #include “nuitrack/Nuitrack.h”. Is that included in the Nuitrack SDK or the Nuitrack linux drivers? I have been looking for these two different files all over the internet and I only found the Nuitrack SDK. Where are placed the linux drivers?
Thank you for your help.
Your sincerely,
Daniel.
Hello there,
I am having the same issue and can’t find the “nuitrack/Nuitrack.h” file. @Daniel, did you ever get this to work?
Thanks in advance everyone!
-Josh
Never-mind, I got it to install. For those having trouble, make sure you install the SDK from Nuitrack’s main page under the API linkas well as install the drivers as in the instructions in this section.
Then change the “NUITRACK_SDK_PATH” parameter in the ROS package’s CMakeLists.txt to the location where you saved the SDK.
UPDATE: Added ability to share Depth and RGB images with other ROS Nodes!
Unfortunately, Nuitrack accesses the depth camera directly, so ROS can’t access the camera. (there may be a way to do this with OpenNI? I don’t know…)
I wanted other nodes to be able to access the color and depth streams, so I have added the ability to publish image messages for both Color and Depth.
I initially was going to use OpenCV, but ended up not using it, as ROS Kinetic image transport uses OpenCV 3.0, while NuiTrack uses OpenCV 2.4 (and you can’t mix OpenCV versions in the same process). So, I publish the Image Message directly, based upon the sample from Nuitrack_gl_sample. Now all my ROS nodes have access to color and depth frames!
Code is checked in at
UPDATE: Added publisher for color pointclouds, so other nodes can now do point cloud processing with PCL, etc.
Thank you so much for keeping this updated. I really appreciate your work and that you share it.
I am trying to build a person following project as well.
If there are multiple people in the frame, how do you keep the following “locked on” to the person you want to follow? How do you designate this person from the outset? (I was thinking maybe via a gesture?)
Thanks!
dshinsel, thanks for working on maintaining this package.
I’ve been running into a runtime issue when using the package. My setup is using a Kinect v2 on ROS Kinetic on Ubuntu 16.04. The nuitrack example code works fine, and this package compiles with catkin_make, but running the rosnode causes a segmentation fault.
This occurs in the tdv::nuitrack::Nuitrack::init("") line, and is reproducible in a more bare-bone ROS node that only runs the Nuitrack::init. Has anyone else encountered this issue?
As far as I can tell, Nuitrack internally designates Skeletons with an id. (See http://download.3divi.com/Nuitrack/doc/structtdv_1_1nuitrack_1_1Skeleton.html) It differentiates between them using facial recognition and keeps track of separate bodies even during occlusion, so you should easily be able to select one person to follow using the Skeleton.id parameter. You could store the first id recognized to be the one to follow, or could use gestures, etc, to change which id to follow.
Thank you for the reply!
Actually no, at least not at this moment. Facial recognition is not used for skeleton tracking, and separate 3d object tracking is used for skeletons, so skeletons IDs couldn’t be used for long-term identification of persons. Facial recognition would definitely improve accuracy and usability of person/skeleton IDs, so this feature is planned for one of the next releases (coming Nuitrack AI version will include this feature).
Is there any way to just identify bodies, not skeletons? I am hoping to make the tracking easier. I notice it is hard for nuitrack to maintain the skeleton identification when part of the person is obscured, like when they go behind an object like a chair.
For my needs I don’t need full skeleton tracking, just person following. Nuitrack so far seems to be the only solution for this for realsense, but it is hard using nuitrack to consistently maintain identification of a person.
Hi GOB,
Please specify what do you mean by “identify”? Do you need to track people or recognize them? If only tracking is required, you can track only the segments of users with Nuitrack (no skeleton tracking). However, this is not suitable if you want to track a specific user. Each segment has an ID but it’s not linked to a specific user. For example, if a person with ID:1 leaves the frame and then enters the frame again, it can be marked as ID:2. But if you want to track just one person who doesn’t leave the frame, segment tracking would be suitable for you.
Thanks for the reply!
I am making a person following robot essentially, so I need to identify a designated person and track that person.
It is possible that the person being tracked would go in and out of the frame.