Get camera coordinates and world coordinates using PyNuitrack

Nuitrack SDK
1

Hello,

How can I get camera coordinates and world coordinates of each skeleton key point using PyNuitrack?
And the transformation relationship between them?

I’m using Nuitrack to get the position of skeleton key points but I notice that the z-coordinate of the head is about 2.2 m > my height. Is it normal?

Best regards

1 Like
2

Hi @Irisviel

Sorry for the delayed reply.

You can look up how to get skeleton points from this example (draw_skeleton function). Here el.projection would be array with XYZ coordinates, where X and Y normalized projective coordinates and Z is real world coordinate (in millimeters). el.real would be XYZ real world coordinates.

About Z coordinate: here it’s distance from the camera (in other words - depth). Height would be Y coordinate.

Let me know if you need further explanations.

3

HI @Irisviel

Do you have any other questions I can help you with?

4

Hello, when I try to use cv2 to plot the projection coordinate (X, Y) on the RGB image, I notice that there are some shifts, that is, the plotted points are not precisely located in the key points. But there is no such problem when plotting the points on the depth images.

5

Hi @a.bragin,

I notice that PyNuitack only outputs the skeleton data of 2 persons when more than 2 persons move before the camera. How to change the max. number of skeletons?

And will the persons’ ids keep the same when they move in and out of the camera’s view?
Or the persons’ ids can keep the same only when the persons are always in the camera’s view? What will happen when they change their positions?

Thanks a lot.

6

Hi @Irisviel

when I try to use cv2 to plot the projection coordinate (X, Y) on the RGB image, I notice that there are some shifts

You need to enable depth to color alignment.

nuitrack = py_nuitrack.Nuitrack()
nuitrack.init()
// Add this line after init to enable alignment
nuitrack.set_config_value("DepthProvider.Depth2ColorRegistration", "true")

How to change the max. number of skeletons?

Same as above

// add after init
nuitrack.set_config_value("Skeletonization.ActiveUsers", "5")

You can also change config values in nuitrack.config file. It’s located in $NUITRACK_HOME/data/ folder on Linux systems and %NUITRACK_HOME%\data\ folder on Windows systems

And will the persons’ ids keep the same when they move in and out of the camera’s view?

Nuitrack will preserve person’s id if the person move in from the side of sensor view they moved out

What will happen when they change their positions?

Nuitrack will keep same user id while they stay in the sensor’s view. When the user move out of the sensor’s view the can (but not necessarily) loose their id if thet come back from a different side/angle of view.

Recently we’ve added Person Re-identification feature (nuitrack-sdk/Person_Reidentification.md at master · 3DiVi/nuitrack-sdk · GitHub). It tracks same user globally (even if they came back from different side of sensor’s view). It’s not available in python yet, but let us know if you’re interested

7

Hi @a.bragin,

Thanks for your answer.

(1) I’m using an IntelRealsense 455 camera. Does the FPS of Nuitrack always keep the same (30FPS)?

(2) How could I get the timestamp of each frame? I tried to use nuitrack.get_skeleton().timestamp to get it, but I noticed the timestamp is always a same number.

Thanks.

8

Hello @a.bragin,

I’m using the code example of PyNuitrack (nuitrack-sdk/README.MD at master · 3DiVi/nuitrack-sdk · GitHub). But the keypoints drawn by the draw_skeleton function are not in the right place. But when I try to use Nuitrack software directly, they are in the right place.

I can’t figure out the reason. Is there any parameters which are not correctly configured?

Best regards