Detect unnatural skeletal rotations or positions

Nuitrack SDK Unity3D
1

Hi, is there a way of ascertaining whether a recognized skeleton has limbs or joints in a way that would be considered unnatural? My unity project is similar to the “Animating the avatar using skeleton” example. I basically want to avoid having the model ever move into an unnatural/creepy position, which sometimes happens when the skeleton isn’t properly recognized, and instead have the model move back to a default position. I have tried setting constrains on the limbs of the model with very little success.

2

I’ve tried to rotate the avatar bone Transforms through their “localEulerAngles” rather than their world “rotation”, so that I later can detect when a bone rotation exceeds a specified limit.

Quaternion localRotation = worldRotation * Quaternion.Inverse(parentRotation);

I was pretty sure I could get the local rotation in the above manner, where worldRotation and parentRotation are the calculated potential world rotations for both the bone in question and it’s parent. When I apply the local rotation however, it does not rotate the same way as when I directly set the world rotation.

Here’s the code:

private void AnimateAvatar(nuitrack.Skeleton skeleton)
{
    _newRotations = CalculateNewRotations(skeleton);
    RotateAllJoints(_newRotations);
}

// Converts Nuitrack joint rotations to bone rotations and adds them to a dictionary with JointType as the key
private Dictionary<nuitrack.JointType, Quaternion> CalculateJointRotations(nuitrack.Skeleton skeleton)
{
    var rotations = new Dictionary<nuitrack.JointType, Quaternion>();

    foreach (var riggedJoint in _jointsRigged)
    {
        nuitrack.Joint joint = skeleton.GetJoint(riggedJoint.Key);
        ModelJoint modelJoint = riggedJoint.Value;
        Quaternion jointOrient = Quaternion.Inverse(CalibrationInfo.SensorOrientation) * (joint.ToQuaternionMirrored()) * modelJoint.baseRotOffset;
        rotations.Add(riggedJoint.Key, jointOrient);
    }
    return rotations;
}

// Rotates the character's bones by first converting the calculated 'world' joint rotations to local rotations
private void RotateAllJoints(Dictionary<nuitrack.JointType, Quaternion> newRotations)
{
    foreach (KeyValuePair<nuitrack.JointType, ModelJoint> riggedJoint in _jointsRigged)
    {
        ModelJoint modelJoint = riggedJoint.Value;

        Quaternion worldRotation = newRotations[riggedJoint.Key];
        Quaternion parentRotation = newRotations[modelJoint.parentJointType];
        Quaternion localRotation = worldRotation * Quaternion.Inverse(parentRotation);

        modelJoint.bone.localEulerAngles = localRotation.eulerAngles;
    }
}

Is my way of calculating the local rotation wrong?

3

You need to replace the line Quaternion localRotation = worldRotation * Quaternion.Inverse(parentRotation); with Quaternion localRotation = Quaternion.Inverse(parentRotation) * worldRotation;.
Quaternion is a 4x1 matrix, and the order of matrix multiplication is important (https://docs.unity3d.com/ScriptReference/Quaternion-operator_multiply.html).
You can also replace modelJoint.bone.localEulerAngles = localRotation.eulerAngles; with modelJoint.bone.localRotation= localRotation;.

Additional info:
https://answers.unity.com/questions/765683/when-to-use-quaternion-vs-euler-angles.html
https://docs.unity3d.com/ScriptReference/Quaternion.html

1 Like
4

Thank you so much Iosif!
I had no idea this was the case, but it solved my problem.