Nuitrack (2018-06-08) poor skeletal tracking with Intel RealSense D435

Nuitrack SDK Reporting Software Issues
1

I am using:

  • Nuitrack 2018-06-08 on win64
  • Intel Realsense 5.9.11 Firmware

I’ve had poor experiences trying out the Nuitrack skeletal tracking with the Intel RealSense D435 in this latest version as well as previous versions.

It is confusing because Nuitrack seems to work much better on another sensor such as the Orbbec Astra/Persee, but it does not work well with the Intel RealSense D435.

Is this a known problem that will be fixed, or am I doing something wrong? Are other people having similar issues?

See example pictures below.

Sometimes, it works fine:

looks-ok
looks-ok3024×4032 1.08 MB

Often, the skeletal tracking has issues, especially with the lower body (but not exclusively):

lower-body-legs-not-tracked
lower-body-legs-not-tracked3024×4032 1.1 MB

lower-body-legs-not-tracked-2
lower-body-legs-not-tracked-23024×4032 999 KB

another-tracking-issue-example-2-people
another-tracking-issue-example-2-people3024×4032 1.01 MB

Another common case is if I try “squatting”, the lower body fails to be correctly detected:

squatting-loses-leg-tracking-often
squatting-loses-leg-tracking-often3024×4032 1000 KB

Sometimes, the skeletal tracking goes entirely wonky and often loses track of the player entirely.

messed-up-before-losing
messed-up-before-losing3024×4032 1.06 MB

2

This is very similar to the testing results we see with the 435 - the quality of the tracking is well below average and bordering on unusable.

We expected that nuitrack would be using the highest resolution tracking data - but it seems to be stuck at 640x480 - and any changes to the nuitrack.config to attempt to improve on that leads to a completely unworkable result.

Westa

1 Like
3

FYI - have a look in some of the other threads - there is a recent conversation about the d435 using lower resolution raw settings ( there are two hidden values that can be added to the nuitrack.config to change the default raw values to 1280x720 - we have noticed some improvement with these settings under certain settings.

        "RawWidth": 1280,
        "RawHeight": 720,

BUT the most significant improvement seems to come when we use a more powerful computer system. The tracking system seems to have a CPU load cap - when it hits this it seems to start reporting less reliable data.
The same can be seen when reducing the skeletal tracking bodys to one instead of two on slower systems.

Westa

1 Like
4

Thanks for the info.

The screenshots in my post above were taken on a desktop using an Intel Core i7-4790. I wouldn’t have thought to try a more powerful computer.

I’ll try tweaking those settings at some point, though I’m hoping Nuitrack gets improved RealSense support in a future update…

–

Note: this seems to be one of the relevant threads you mentioned.

5

Hi Sonny,

You may want to try adding the “RawHeight” and “RawWidth” parameters to the Realsense section on the config.

Westa

6

BUT - you should also have a read of this explanation on the intel website -

BASICALLY, In short, the Intel RealSense D415 is more than 2x more accurate than the D435

Westa

2 Likes
7

Hello!
Does It mean nuitrack skeleton tracking works well with realsense 415?
Or still works badly with little improvement?
I have only 435 so can’t test it.
Thanks in advance.

8

I guess it really depends on your definition of “works well”.

With perfect lighting conditions and a perfect setup - a d435 can produce results as good or better than a orbbec astra. As the depth image is more stable with considerably less noise - though this may have more to do with intel’s post processing than most other things.

The d415 increases that accuracy and stability for the same scanning area by a factor of 2 - BUT its also a much narrower field of view - which may not be suited to everyone’s needs.

Westa

2 Likes