2D character assignment and rigging

Nuitrack SDK Unity3D
1

Hello,

I really love the Face Track example (even thought I haven’t got it to work yet - see the other post I just made) and want to do something similar, but I’m so new to this so I’m wondering if you could give me some tips.

Basically, I’m trying to have multiple people controlling different 2D characters, based on gender and age detected from faces. These 2D characters are very simple puppets with distinct joints and they have already been rigged and can be re-rigged if needed, like those in these images:

Annotation%202019-04-03%20041708
Annotation%202019-04-03%20041708542×852 65.8 KB
Annotation%202019-04-03%20041531
Annotation%202019-04-03%20041531680×1007 82.2 KB

Just like the Face Track example, I want these characters assigned based on gender and age, but here the whole body changes, not just the face.

What do you think is the best way to achieve this? From the Unity examples, my impression is that no real rigging of skeletons takes place, lines are drawn based on joints. So what’s the best way to assign 2D body parts based on face data?

2

Hi ysysys,

We have many avatars for reference: 3D skeletons (see the tutorial project Displaying Skeletons on an RGB Image), 3D skeletons (Creating your First Unity Project using Nuitrack SDK and sceneStickman.scene), 3D avatars (Animating the Avatar using Skeleton).

As far as we understand, you just need to change the rotation of joints. In this case, there are two solutions:

  • get the joint orientation from Nuitrack (see the Indirect Mapping section in our Animating the Avatar tutorial);
  • get the positions of two adjacent joints and calculate the correct rotation of a joint in your game.

There’s an example for 2D in SimpleSkeletonAvatar.cs:

connections[i].transform.right = endJoint.transform.position - startJoint.transform.position;

Perhaps you’ll need to change ...transform.right to ...transform.up or to ...transform.forward.

For 3D, you can take a look at our SkeletonAvatar.cs.

So what’s the best way to assign 2D body parts based on face data?

You can get the gender info from our Face Tracker and change the skin gender according to this info (the Mesh field on the SkinnedMeshRenderer component).