[Announcement] Nuitrack Unreal Engine 5 Plugin

[irakli]

Hi there,

We’re happy to present our real-time body tracking for Unreal Engine 5. It could be used without writing any code, simply connecting Nodes in Blueprints
It’s perfect for developing games , fitness apps and other interactive projects with body tracking.

Get started with examples showing different Nuitrack capabilities:

• SkeletonJoints - check what the camera sees and how the skeleton joints are tracked
• AvatarMannequin - animate a game character using Nuitrack skeleton
• FittingRoom - how to create a virtual fitting room
• GesturesDemo - demonstrates how the gesture recognition works
• HandTracker - how to use hand tracking to interact with the UI

If you’re interested in Skeleton Tracking with Orbbec, RealSense or Kinect sensors (v1, v2, Azure) , you’re welcome to try out.

Unreal Engine 5 Plugin For Real Time Skeleton Tracking

[TAG]

Nuitrack For Unreal Engine 5 (Blueprints) | How to Start

[irakli]

Hello. And here is a new video tutorial that will show how to work with joints and visualize the skeleton

[irakli]

A new version of the beta-plugin has been released:

• Simplified Game Building. Now you don’t need to edit any files, just do the build.
• Simplified Plugin Rebuilding. In README.txt (and first post) added instructions on what to do if Unreal Engine requests a rebuild. Check out the Plugin Rebuilding section
• Removed errors from the output log that appeared at startup
• The size of the plugin has been significantly reduced

New Version Beta Nuitrack UE5 plugin

[irakli]

Hello there, new text tutorial that will show How To Animate Avatar UE:

AnimationBlueprint

1. First of all, open Mannequin_Skeleton_AnimBlueprint

2. Look at what variables are here and pay attention to the initial values (especially the fact that JointsTransform is an array with 13 elements)

3. See how the blueprint animation is configured for this model. If you look closely, the nodes are made up in the shape of a person, two nodes for each arm, two for each leg, and nodes for the body. If you click on one, then in the first field (“Bone To Modify”) you can see which bone it rotate.

4. Then open or create an animation blueprint for your model. And create the same variables as in Mannequin_Skeleton_AnimBlueprint.

   Summary

   

5. And copy the nodes from Mannequin_Skeleton_AnimBlueprint to this one, then connect them according to the sample from that file, do not forget to select the desired, previously mentioned, “Bone To Modify” in each node.
   To make sure that everything is done correctly. Select the Start value, apply and recompile. If the model assumes a strange pose, then the values of the turns of the bones change. Disable the “Start” value back

   Summary

   

BlueprintAvatar

1. Right click on “AvatarMannequin” blueprint. Select “Create a Child Blueprint Class”. Rename it, for example, to “AvatarTest”. And put it on the level. Make sure it is in location 0, 0, 0. And has a rotation of 0, 0, 180.

2. Open “TestAvatar” blueprint for editing. Select “Mannequin” in components. And select your mesh in “Skeletal Mesh Asset”. (T-Shirt just for example)

   Summary

   

3. Open Event Graph and Copy variable “Joints” from Variables in Blueprint “AvatarMannequinBlueprint”. Bitween “Begin Play” and “Parent: BeginPlay” set “Joints” with new “Joints_0” variable. In “Joints_0” in first column stored Nuitrack Joint Types, in second column contains the corresponding names of the bones in the mesh. If necessary, they will need to be changed, just write down the actual values in these fields that can be viewed in the “Skeletal Mesh Asset”

   Summary

   

   
   

   

Try to choose the bones on your mesh as close to these points as possible

Summary

​4. Try to Start Level

[irakli]

Good news everyone
Now you can download the plugin from the store!

Nuitrack - Real-time body tracking in Code Plugins - UE Marketplace