Realsense D435 can't run Nuitrack samples, but intel viewer works

Nuitrack SDK Reporting Software Issues
1

C:\Program Files\Nuitrack\nuitrack\nuitrack\bin>nuitrack_c11_sample.exe
Start nuitrack c11
Create DepthProvider: Realsense2DepthProvider
terminate called after throwing an instance of ‘tdv::nuitrack::TerminateException’
what(): Realsense2DepthProvider: rs2_pipeline_wait_for_frames(pipe:00000000043E1AD0):
Frame didn’t arrived within 5000
terminate called recursively

I cannot get the nuitrack_c11_sample.exe to work, I get the above output and it shuts down after a few seconds every time (after another popup window with some yellow text shows up in front of it). But the camera works fine with Intel’s RealSense viewer.

  • Windows 10, tried on two different hardware setups same results.
  • Environment variables are set correctly. Have rebooted (many times).
  • Tried installing on different drives. Tried reinstalling.
  • Tried moving install dir to C:\
  • Full permissions granted to install dir.
  • Made sure I have VC++ redist installed.
  • Tried uninstalling & reinstalling RealSense SDK.
  • Firmware is 5.11.6.200 released 3.26.2019 (note intel’s page is screwed up and lists an older version as latest)
  • Have tried both with default windows drivers and the Intel UWP drivers, no difference.
  • USB 3.2 connection, also tried through a USB 3.0 powered hub.
  • Tried downgrading through multiple firmwares to 5.10.13, no difference.

We bought licenses to use Nuitrack with unity, but I’m getting blocked before I can even run the activation tool. I’ve looked through this forum and tried all the things people have posted about issues with this camera, none seem to fit or solve my problem. I see this particular error cited on RealSense github pages but for seemingly different reasons than what I’m seeing. I haven’t seen anyone having this only with Nuitrack.

The evidence I’ve gathered suggests a problem with Nuitrack talking to the driver. It does not seem to be a hardware issue as again it works fine with Intel RealSense viewer on two different computers.

For Nuitrack developers, does this issue seem applicable?

Any ideas? Help!! :slight_smile:

2

Hi Ryan,

Please advise what is your Nuitrack version? It’s displayed at the bottom of the Nuitrack activation tool window.

3

On one computer, where I last tried to move it to the root of C:, it shows as “unknown”.

On the second computer, it shows as 0.27.0.

4

Please run realsense-viewer tool and send us the screenshot with parameters (when the camera works fine).
You can also try to edit nuitrack.config:

  • Find the section "Realsense2Module" > "Depth"

  • Add the following lines:

          "RawWidth": 640, 
      "RawHeight": 480,

And then try to run Nuitrack again.

5

RealSense_ScreenshotDepth
RealSense_ScreenshotDepth2676×1586 898 KB

I’ll try the config change when I have a free second and report back.

6

That config change does seem to allow the sample to run, thank you. Tested on two PCs, and a 3rd Mac mini with bootcamp Win 10, the config change was required on all 3 for samples to not give the original error.

FWIW, on the Mac mini we were able to get a RealSense D435i camera to work without the config change.

7

Do you use the standard cable for connecting RealSense?

8

Yes. (Which is incredibly short)

Update: although I have the camera more or less sending data now, it’s really having trouble recognizing the skeleton reliably, it ends up being really a jumbled mess of points that jumps all around my body. I’ve tried it during daylight and at night in more controlled lighting situations, same results. Any suggestions for fine tuning skeleton recognition?

9

So after spending another day messing around, I found a few other things that helped our particular setup.

  1. Environmental: Shiny’ish curtains were scattering the IR light and confusing the camera as far as Nuitrack was concerned. Removing those from camera view helped a bunch.
  2. nuitrack.config. This file should really be documented somewhere or at least mentioned during install steps. I understand it is “experimental” however it is also essential to tweak some of those parameters, which for example, RealSense viewer exposes through user accessible UI. I found that changing the assorted height/width values to be the native highest res the camera could deliver significantly improved skeleton tracking, in addition to setting DownsampleFactor to 2 to help mitigate IR noise.

Once I figured out how to tweak that file, I was able to get a mostly functional pipeline to a Unity scene.

2 Likes