Clarification on joint.projection on python SDK

Nuitrack SDK
1

Hi!
There’s something wrong (or just different) on the structure documented on Nuitrack: tdv::nuitrack::Joint Struct Reference and what is given by the .get_skeleton() method for the .projection on each joint. The documentation states that ‘proj’ is “Joint position in normalized projective coordinates (x, y from 0.0 to 1.0, z is real).” but the values that are provided are not normalized.

For example:

left_shoulder=Joint(type=PyNuitrack.py_nuitrack.JointType.left_shoulder,
confidence=0.75,
real=array([ 101.467155, -166.7774 , 1245.0547 ], dtype=float32),
projection=array([ 285.46588, 260.4319 , 1245.0547 ], dtype=float32),
orientation=array([[-0.12268531, 0.22730845, -0.96606374],
[ 0.7213615 , -0.64811254, -0.24410596],
[-0.6816054 , -0.7268294 , -0.08445768]], dtype=float32))

Perhaps I’m missing something (is ‘proj’ in the c++ library different than ‘projection’ atribute in python?).

If I try the example from github, the function ‘draw_skeleton’ draws the points not in the same positions that the user in front of the camera has:


def draw_skeleton(image):
    point_color = (59, 164, 0)     
    for skel in data.skeletons:
        for el in skel[1:]:
            x = (round(el.projection[0]), round(el.projection[1]))
            cv2.circle(image, x, 8, point_color, -1)

and, as you can see, the el.projection is not used as a normalized value.

2

Oh! Making a little check, I found that the el.projection value is based on the .get_depth_data() sizes and not on the .get_color_data() , so there’s the explanation on why the points are shifted when drawing. Using a Kinect v2 I get a 512x424 depth image and a 1920x1080 color one.

Anyway, I still don’t understand why these values are not normalized as stated on the docs.

3

Hello @fjoselevich
Sorry for the delay.
We checked it out, it’s a bug, we’ll fix it in the next update.