Map color to depth with kinectv2

Nuitrack SDK Unity3D
1

i’m working on mapping depth and color with kinectv2 but i didn’t found anything to do that.
Can you help me. i activate "Depth2ColorRegistration in nuitrack.config but it doesn’t work
thanks

2

Hi Giulio,

Please advise:

  1. What is your Nuitrack version? It’s displayed at the bottom of the Nuitrack activation tool window.
  2. What sample do you run? Try to check depth-to-color mapping using our standard nuitrack_sample.exe (you can switch RGB and depth maps by left mouse click).
  3. To turn on depth-to-color registration, you have to edit nuitrack.config: set Depth2ColorRegistration to true. If this doesn’t help, you can use Nuitrack API. Insert the following line after Nuitrack::init() / Nuitrack.Init() :

C++
Nuitrack::setConfigValue(“DepthProvider.Depth2ColorRegistration”, “true”);

C#
Nuitrack.SetConfigValue(“DepthProvider.Depth2ColorRegistration”, “true”);

3

hi olga, i’m working with unity with 0.27.0 version in the sample it works but in unity 2018.3.12 with point cloud sample doesn’t work
i add in pointcloud.cs this line: nuitrack.Nuitrack.SetConfigValue(“DepthProvider.Depth2ColorRegistration”, “true”);
thanks

4

Are there any errors in Unity log? It’d be great if you could post some screenshots that show the problem.

5

hi olga there is no error in unity but i discovered that color frame is 1920x1080 and not 640x480 as depth frame

6

screenshot
screenshot2560×1800 507 KB

7

Raw RGB resolution of Kinectv2 is Full HD and it’s fixed. Raw depth frame resolution of Kinectv2 is resized to 640x480 for further processing by Nuitrack.

8

yes i know but how can i map color pixel with depth pixel if the frame is different?

9

You can use normalized coordinates (project.x, project.y) to get the corresponding points on the depth and RGB maps.

10

i’m sorry i’m not sure i have to use this function: ConvertProjToRealCoords (int x, int y, UInt16 depth) ?
i found covert function only in depthSensor class but i understand that this function convert depth pixel in real world coordinate and not in color coordinate. Can you explain me better how can i take normalized coordinated?

11
  1. Make sure that Depth2ColorRegistration is set to "true". We checked the registration with Kinect v2 and it works fine in our case (you can check the registration in our nuitrack_sample by left mouse click).
  2. Take the pixel on the depth map with the coordinates (x, y, depth_value). Convert it to normalized coordinates: (x/width, y/height), where width and height mean the size of the depth map. Then, you can get the corresponding pixel on the RGB map this way: (x/width*width_rgb, y/height*height_rgb).
1 Like