[Bug] Color and Segment frames not aligned

Nuitrack SDK Unity3D
1

Hi everyone!

I have trouble using the nuitrack SDK in Unity3D when trying to mix ColorProvider and SegmentProvider frames, as shown here:

cam_1
cam_1563×553 277 KB

Two years ago, I was using an Astra Pro camera with a Nuitrack Pro portable licence and everything was working perfectly. I recently switched to a Stereo S U3 camera, and currently use a trial version, waiting to buy an AI one.

I tried two different things:

  • rotating the camera by 90°. Surprisingly, the image from the segment frame appears to be always below the color frame (so under the user when the camera is set horizontally, and at his left when set vertically)

    cam_2
    cam_2424×604 305 KB

  • Inspired by this thread: Is nuitrack AI enabled by default if you have a licence for it, I also played with the Depth2ColorRegistration value in nuitrack.config. There is a difference between when the value is set to true instead of false, but not in the right direction: when set to true, the distance between the two images is even higher:

    depth2col
    depth2col613×627 316 KB

Besides, when using the “Try nuitrack!” button in the licence activation tool without AI, it seems to be correctly superposed.

Did anyone already crossed this problem, or has any idea on how to solve it?

Regards.

3

Hello @Dandelion
At the moment, “Depth 2 Color registration” is not supported on this sensor.

4

Thanks for the answer!

Indeed, I tried the FrameVizualization in Unity and still had the problem. With the nuitrack activation tool, the skeleton is well superposed with the segment, but not with the RGB (which is logical since both skeleton and segment uses depth sensor) as shown is those pictures:

skeleton_1

skeleton
skeleton741×353 188 KB

I suppose you don’t know yet when the support will be made?

Best regards.

5

Yes. Unfortunately, it is difficult to say now when support for this sensor will be added

6

Thanks again for the answer, I’ll go check on another camera then. Have a nice day!

1 Like
7

Hi @Dandelion,

I should say that we qualified this issue as a bug (as Stereo S U3 sensor is supported principally), so will try to include resolution for it in delivery cycle as soon as possible. Thank you for bringing attention to this issue. Please stay tuned for resolution.

1 Like
8

Hi and thank you a lot, I’m saying tuned then!

9