[Bug] Color and Segment frames not aligned

Hi everyone!

I have trouble using the nuitrack SDK in Unity3D when trying to mix ColorProvider and SegmentProvider frames, as shown here:

Two years ago, I was using an Astra Pro camera with a Nuitrack Pro portable licence and everything was working perfectly. I recently switched to a Stereo S U3 camera, and currently use a trial version, waiting to buy an AI one.

I tried two different things:

  • rotating the camera by 90°. Surprisingly, the image from the segment frame appears to be always below the color frame (so under the user when the camera is set horizontally, and at his left when set vertically)

  • Inspired by this thread: Is nuitrack AI enabled by default if you have a licence for it, I also played with the Depth2ColorRegistration value in nuitrack.config. There is a difference between when the value is set to true instead of false, but not in the right direction: when set to true, the distance between the two images is even higher:

Besides, when using the “Try nuitrack!” button in the licence activation tool without AI, it seems to be correctly superposed.

Did anyone already crossed this problem, or has any idea on how to solve it?


Hello @Dandelion
At the moment, “Depth 2 Color registration” is not supported on this sensor.

Thanks for the answer!

Indeed, I tried the FrameVizualization in Unity and still had the problem. With the nuitrack activation tool, the skeleton is well superposed with the segment, but not with the RGB (which is logical since both skeleton and segment uses depth sensor) as shown is those pictures:


I suppose you don’t know yet when the support will be made?

Best regards.

Yes. Unfortunately, it is difficult to say now when support for this sensor will be added

Thanks again for the answer, I’ll go check on another camera then. Have a nice day!

1 Like

Hi @Dandelion,

I should say that we qualified this issue as a bug (as Stereo S U3 sensor is supported principally), so will try to include resolution for it in delivery cycle as soon as possible. Thank you for bringing attention to this issue. Please stay tuned for resolution.

1 Like

Hi and thank you a lot, I’m saying tuned then!