For my project, I am trying to create and upload new models to Unreal, then use Nuitracks motion capture to give them real-time body tracking similar to the example avatar in the plugin. I am having quite a bit of difficulty though figuring out how the example avatar is set up so that I can copy this process onto a different model/avatar and achieve the same results. Is it possible to use Nuitrack on different unreal models or is the example avatar the only one that works?
Hello @Slhelus
You can use it with your avatar
Note on point 1 in the second section of this instruction: Do not create a child class, but make a copy and change the parameters for your model in it.
I have been following the process as you mentioned and I am running into an issue with updating some of the blueprint components. When updating the Current Avatar Anim BP, it says that the set function is not compatible with my animation blueprint and is still look for the AB_indirect_animation_blueprint. I dont think that should be happening as I have updated all the variables i could find related to that, namely the “currentAvatarAnimBP” and the “AvatarAnimBP” variables in the side window, and it still wont update. I tried replacing the set function but I cant seem to find the one y’all used. Is there any guidance you could provide on changing the copied blueprint class properly?
Nevermind, I found out how to recreate the variable. You had to drag from the get variable then find it.
Can you list exactly what i need to change for the blueprint to work? I’ve changed everything to my new animation blueprint and skeleton things, but it still wont track during simulation while the Nuitrack example does. Not sure what i havent changed yet but any guidance would be appreciated.
Yes i believe i completed all steps correctly. I figured out the issues with it not tracking (I forgot to bind the skin - unrelated to nuitrack) but now the arms track oddly (bend foward and in when they should be out and upright) and im not sure what the issue is. I can post an example when im back in our lab.
Perhaps the hands just fall out of the sensor’s field of view. Try adding an BP_FrameViewer to the level that visualizes sensor frames.
Ive been comparing the new models tracking with the sample you guys created in live time to make sure it is tracking correctly. I’ve since found a different model in Tpose and when i tracked that it worked as expected.
Is it possible to use nuitrack to animate models in A-Pose or do they have to be T-Posed like the provided plugin?
I think there shouldn’t be a problem with a-pose.
The new version of the plugin will have such a model in A-pose. But the Blueprint itself hasn’t changed much. You can try updating the sample or wait for an update from our side to learn more. (Program Files\Epic Games\UE_5.X\Engine\Plugins\Marketplace\NuitrackPlugin\Content\Samples)
Oh this is perfect thank you. Should I replace the current avatar folder in the nuitrack plugin or could i create a separate folder just for the updated avatar folder? also, is the updated version meant to work with the default manny avatar or an edited version?
Yes, you need to replace the folder. There are two blueprints, manny and the old avatar.
Understood. So I replaced and updated the folder but the avatar blueprint for manny is having compiler issues. The “Get instances Json”, “From string”, and “has field” functions in the get face angles function seem to be unknown. Im not exactly sure what this does but manny still seems to track properly although he does disappear until someone steps in the camera. Is this something still being worked on or am i missing something?