Tracking skeletons with angled sensor (Intel RealSense)

Nuitrack SDK
1

Hello,

I am trying to track people in front of a big screen. To do so I have a RealSense camera that is positioned on top of the screen (~4 meters high), in the middle, looking down at the area in front of the screen (~35° angle from the horizontal).

With this setup I only get the top part of the skeletons passing by, the leg are not recognized, as shown in the photo below

NuitrackSkeletonAngled
NuitrackSkeletonAngled1031×764 150 KB

If I angle myself to be orthogonal to the sensor (leaning back ~35°), then the legs appear and the whole skeleton is tracked so I believe the sensor angle is the problem.

Is there a way in nuitrack.config to specifiy a sensor angle for the skeleton tracking, or parameters that I could adapt to better track the skeletons (especially the legs) of people in my configuration ?

Thanks in advance.

2

For RealSense sensors, you can try to improve tracking by reducing noise in depth frames. To do this, open the nuitrack.config file and set the parameter Realsense2Module.Depth.PostProcessing.DownsampleFactor to “2”.
I would also like to draw your attention to the fact that your legs are beyond the recommended distance for tracking.

3

Thank you for your answer.
I tried increasing the downsample factor in the realsense module but it did not help.

I also tried getting the sensor closer but the problem remains which make me believe it really is the sensor angle that is causing the issue. As soon as I bend my leg or knees to make my leg orthogonal to the sensor, it is detected fine, even when far away.

4

After different tests, I found that the proximity and the angle between the legs and the floor seem to be the problem. I guess the algorithm does not manage to separate the legs from the background in this configuration.

If I place the camera on the ground, facing upward, even with an angle the whole skeleton is detected fine. In this case there is no floor behind so I guess this helps the foreground extraction and/or the skeletonization algorithm.

I would also like to mention that when there is no background behind me (camera facing upward), or when the camera is horizontal to the floor (no angle), the detection works fine up to more than 5 meters, almost double the recommended distance for tracking.