Hand Tracking - C# WPF

Hi there,

I am currently in the process of developing a hands free controller for our software solutions. The hand position controls the mouse cursor etc.

I am finding that this works very well in some cases but then just a few minutes later without making changes it will be really jumpy. The mouse position will jump from one location on the screen to another or jumps around.

I was wondering if I could get some help in editing the nuitrack.config files settings. I think the problem is due to noise from the camera. Looking at the skeleton, he is having problems with the lower arm / hand.

We are using a realsense D415. In the other projects which I have created directly using the sdk I added filters to improve the input image. Perhaps editing the spatial filter to include holeFilling / changing the filterMagnitude / smoothing. RealSense also includes a holeFilter. Can this all be done through editing the config file? If so would it be possible to get an example of how to do this. Nuitrack is definitely the best I have seen but so far this is not good enough to give to a customer. I am afraid that they will get confused and lose their patience.

I also wondered if there was some best practice in positioning the camera for this particular application. Is it better to position the camera high looking down for instance? Or just level with the user.

1 Like

Hi Karl,

Make sure that the distance between a sensor and a user is about 1.5-2 meters and sensor is placed at height of ~1 m.

To reduce the noise, you can edit nuitrack.config:

  • Find the section Realsense2Module > Depth > DownsampleFactor and set it to 2
  • Find the section Realsense2Module > Depth > PostProcessing > SpatialFilter > spatial_iter and set it to 2

Hi Olga,

Thanks for the help. This is not working too badly now.