T-Pose detection

Nuitrack SDK Reporting Software Issues
1

Whats the best practise to detect T-pose?

I read in the documentation;

Do I have to iterate through all the joints and check if all joints except wrist,feet and head are identity?
How do I test for identity, do I have to check the Orient property?

2

VicoVR v1.1 SDK should contain RoomHMD example with T-pose detection and calibration (VicoVR SDK v1.1).

Currently in our demos we do it like this:

  1. we get head, torso, shoulders, elbows and wrists joints from skeletonData. Check their confidence (should be greater then 0.5). And then check if arms joints lie approximately on same line ( Vector3.Angle (handsDiff[0], handsDiff[i]) < angleDiffTreshold (handsDiff[0] here is the difference between wrists, handsDiff[i] - differences between nearby arms joints (leftWrist - leftElbow - leftShoulder - rightShoulder - rightElbow - rightWrist) ) ). If all conditions are met we save positions of those joints.
  2. then for some time (~1 - 2 sec) we check joints confidences and positions (should be approximately in same place as they were).
3

Is the status of these features the same in sdk 1.3+? what is the unit of error between the tracking joint and the real joint? Is it precise enough to calculate angles and planes that will have actual correlation to the physical skeleton with high precision and low position error rates? Would it be best to calculate angles between joints as snapshots where the user was instructed to preform a specific action from a prompt point in the game or could it reliably be called on a live-time basis? the skeletal tracking in the test sensor activity seems to lag somewhat from the physical skeleton which makes me think that a snapshot instead of a live-stream of data calculations based on skeletal position would be best vs a dashboard view for the user of the calculated data in realtime…

4
  1. Is the status of these features the same in sdk 1.3+?

Latest version of SDK is 1.2 at the moment. There is t-pose calibration script and UI visualization example in it.

  1. what is the unit of error between the tracking joint and the real joint?

Not all poses are handled correctly, but in properly handled poses error has order of magnitude of several millimeters, depending on distance to sensor (bigger errors on longer distances).

  1. Is it precise enough to calculate angles and planes that will have actual correlation to the physical skeleton with high precision and low position error rates?

Usually it’s precise enough to estimate angles and planes (at least for use in VR games) when there are no big errors in pose estimation. But it really depends on what precision you are looking for.

  1. Would it be best to calculate angles between joints as snapshots where the user was instructed to preform a specific action from a prompt point in the game or could it reliably be called on a live-time basis? the skeletal tracking in the test sensor activity seems to lag somewhat from the physical skeleton which makes me think that a snapshot instead of a live-stream of data calculations based on skeletal position would be best vs a dashboard view for the user of the calculated data in realtime.

Yes, there is a bit of lag (sensor processes raw data and then transmits it to device), it’s comparable to lag in network games(~50ms). In some of our demos we do calculate velocities / angles on a live-time basis. But again, if it applicable in your case depends on what accuracy and latency you aim for.