Hand Tracking Reliability with Realsense


We’re developing an application using your software, and planning to deploy it at various retail locations.

We’ve been using Realsense D415 cameras for development on Unity with descent skeletal tracking results, but often hand tracking (waving, grabbing, etc) gestures seem to glitchy at best, sometimes overly sensitive, sometimes not sensitive at all.

I would like to know if this might be a camera or software issue. We’re considering trying alternatives like the Orbecc Astra, but would like to know your teams input as to what camera might deliver better performance for hand gesture tracking.


1 Like

I would suggest that without coding your own downstream filtering

  • you will not be likely to get commercially workable results out of the box with Unity and Nuitrack
  • Or any other current sensor technology.

Even when Kinect 2 was at the height of its game to get the best results from the data tracked - there was need for downstream filtering to achieve workable commercial results.

The issue is really a case of - sensors are inherently noisy and error prone - nuitrack can only work with the data its given - and when that data is poor - nuitrack struggles.

The option then is - assist nuitrack downstream by coding your own filtering system that works with the data created to produce more commercially viable results.


Hello Westa,

My team has created somewhat satisfactory results detecting waving and other gestures in a controlled environment, however when tested in field the technology doesn’t work for most individuals.

I’d like to know if yourself or someone you know that has achieved commercially viable results are available for consulting to be able to bring our project up to speed.


Whats the status? Have you found anything? We’re walking into the same problem. Data is way to jittery, and we tried some simple filtering but can’t get the expected behaviour. Especially the hand tracker gives us struggles

My team of 3 developers spent 5 months using different cameras and equipment trying to create a reliable prototype that tracked a skeleton and simple off the shelf gestures (swipe, wave, etc). We were really disappointed and had to abandon the project in the end, opting for other types of technology (facial detection, OpenCV, etc).

Nuitrack looks great on paper and on the example videos on the website, but it was an expensive experiment for us. Mind you, we never tried hand tracking.

I wish you all the success we didn’t have!

1 Like

It’s becoming an expensive project for us as well. We’re making progress but Nuitrack doesn’t provide us with the behaviour/reliability we want/need, especially for the price they ask. Skeleton tracking seems to work fine except for the legs and user tracker works fine as well, sadly it does have some jittery data, but we filtered out most of that. The challenge we’re facing now is the hand tracker.

Thanks for the update!

1 Like

I built a hand tracking app in Unity for an Android Nvidia Shield using the Intel Realsense. Nuitrack’s android software is pretty glitchy. If the device is turned off it looses its license and fails in other ways that make it a hard thing to hand off to my client. Can Nuitrack sort through some of the glitches or are you all done with it?

Hi Tim

Please specify what do you mean by “glitchy”? Nuitrack shouldn’t “lose the license” when the device is turned off. What license do you use? If it’s an annual license, perhaps, the date&time settings are reset when the device is turned off and on.