Unable to record torso animation and questions

[grifis]

Hello,

I’m using Nuitrack v0.31.0 (licensed), Unity 2019.3 with depth camera RealSense D415.

I can’t find a way to record torso movements using Record Mode = Humanoid. Movements are recorded but the torso remains completely “locked”. Do I miss something?

I’ve also other questions:

• Is it possible to use more than one RealSense to improve detection?
• Do Nuitrack allows facial movements recording in addition to recognition?

Thanks

[iosif.semenyuk]

1. In the Humanoid system, it’s not possible to record the position of a specific joint. As a workaround, we’ve recorded the position of the Root joint instead of Torso in our tutorial.
   In other words, animation in the Humanoid system is presented as values (from 0 to 1) of bending the specific muscles. For example, if we take the Forearm Stretch muscle, “1” means a bended arm and “0” means a straight arm. Actual limitation of angles is set in a specific model (see the Rig tab, if AnimationType = Humanoid). This allows to apply the animation to models of any shape.
   If you’re using Unity 2019, you have to replace the GetClip property in HumanoidRecoder with the following one (since Unity API has been changed).
2. Currently, Nuitrack cannot be used with several sensors in one application. To implement this feature, many issues have to be solved (for example, mixing of data from different sources, etc.)
3. At this stage, we don’t have any examples with face recording. This feature will be implemented in one of the next Nuitrack versions.

[iosif.semenyuk]

To record facial movements, you can try to combine the Animated Emoji and Motion Capture tutorials and record animation with Blend Shapes. You can see a recorder module for Blend Shapes in the attachment (see the comments for reference).

BlendShapesRecoder.cs

[grifis]

thank you, you’ve been very helpful!

I will do some tests and then report the progress

[grifis]

@iosif.semenyuk

In the Humanoid system, it’s not possible to record the position of a specific joint. As a workaround, we’ve recorded the position of the Root joint instead of Torso in our tutorial.
In other words, animation in the Humanoid system is presented as values (from 0 to 1) of bending the specific muscles. For example, if we take the Forearm Stretch muscle, “1” means a bended arm and “0” means a straight arm. Actual limitation of angles is set in a specific model (see the Rig tab, if AnimationType = Humanoid). This allows to apply the animation to models of any shape.

I’m not trying to record specific joint; I need the movement of the torso together with other joints. Humanoid recorderer records the animation as if the character is fixed and will not bend or turn.
I added every type of joints but there is no way to make it works.

I also noticed in your example that the character has a fixed torso: https://raw.githubusercontent.com/3DiVi/nuitrack-sdk/master/doc/img/Umc_16.gif

maybe it’s some setting or is there something else I’m missing?

[iosif.semenyuk]

You should use Record Mode = Generic