Fittingroom UE5 doesn't look as good as expected [InProgress]

Nuitrack SDK Unreal Engine
3

Hello @30_Samu_03
Thank you for your interest in the plugin. We are preparing improvements regarding this case, we will try to update the plugin as soon as possible

1 Like
4

Thank you for your reply @Stepan.Reuk .
Is it foreseeable when the plugin will receive this update? I want to use Nuitrack for a project that needs to be completed by January 20th.
Is there already a way to make the demo look like it does in the video (maybe Unreal Engine 4 or Unity)?

5

@30_Samu_03
You can test the “Virtual Fitting Room Sample” scene in our Unity-package
https://github.com/3DiVi/nuitrack-sdk/tree/master/Unity3D

6

Good to know, thank you very much! Is it foreseeable when the plugin in Unreal Engine 5 will receive an update?

7

I think in the first quarter of 2025

2 Likes
8

Is there a tutorial for aligning a mixed reality camera view in Unreal Engine 5? We’re launching a project for a client beginning installed testing on Jan 8th and hoping to use Nuitrack for this.

We’re comfortable doing custom development work if there is a description of the necessary steps available, perhaps this could even be contributed back to the plugin. Thanks and happy holidays!

9

There is no such tutorial at the moment, but we will definitely add it in the future.

10

Thanks for your reply! A couple more questions:

  • Are you able to provide a quick bullet point list of steps to align a mixed reality camera, in lieu of a full tutorial?

  • What material is necessary to have the arm occlusion over the shirt as shown in the blueprint video above?

  • I notice that the hips don’t seem to rotate much, causing stretching of the garment. Is this expected and/or part of the pending plugin updates? (any potential solutions would be greatly appreciated!)

1 Like
11

@AC_TK @30_Samu_03 Filling out this short form can greatly speed up the addition of the necessary features and tutorials. (the more detailed the better) https://docs.google.com/forms/d/1_V4iSpwwOQxC6sHEyPenIj7j0uFGU4zjsHW1J4ChP2g







 A couple more questions
  1. This scenario has not been worked out enough yet and it turns out quite a lot of manual steps. But the general recommendations are as follows:
  • You need to compare a real sensor, a virtual camera and a model. The camera on the level must be in the same coordinates as the model (for example, (0,0,0)) and have the same fov as the sensor.
  • the model should use not only the rotations of the joints, but also the positions (Animation Blueprint). In this case, the model will stretch.
  • for 3D, it’s better to use the classic tracking mode (not AI) now

  1. The usual one-sided material. Just make sure that the inside of the model has no polygons pointing inwards and you can look through the model (through a sleeve or other opening in the clothes)

  2. I don’t quite understand, can you send me a video?

12

Happy New Year!

It seems that the BP_AvatarMannequinBlueprint updates both position and rotation via Update Joints Transform, is this correct?

Thank you for getting back on all those points. Knowing that the camera and the model should both be positioned to 0,0,0 and the FOV should match is very helpful. From testing we agree that classic tracking is better than AI for now.

For #3 I was able to replace the skeleton mesh of the clothes directly within the Mannequin of the BP_AvatarMannequinBlueprint to fix this issue. Previously I had replaced the mesh in BP_T-ShirtBlueprint.

13

Hello and Happy New Year!

I am trying to integrate cursors (hand tracking) into UE5 for my program. The cursors have been implemented according to my vision, but I can’t manage to simulate clicking so the cursor can interact with widgets. The clicking gesture is already detected by my hand, but it cannot interact with the widgets. Is there a simple approach to this that I might be missing?

I’ve also tried using hand tracking to move my actual cursor (and it even moves outside of Unreal Engine), but I still can’t simulate a click (e.g., a simple left-click). I would be very grateful if someone could help me with this! :slight_smile:

Thank you in advance!!

14

Hello again,

Does someone have a solution for my problem?

Thank you in advance!

15

Hello @30_Samu_03, have you checked information from this page ? It could provide general guidelines on the implementation of NUI / gesture control through a hand tracking.
Another more complete tutorial is here (it is for Unity, but the same principles applies to UE).

Please let us know if there are uncovered issues / topics in these materials, we’ll try to help.

16

Yes. In order for the model to fit properly on a person, the positions and orientations of the bones must be updated.

17

Hello @30_Samu_03

Did you manage to implement the cursor “click”?

18

Yes kinda, I connected the cursor to every Button to implement the “click”. I couldn’t find a better way, but now it works. Thanks for your help!!! :slight_smile:

1 Like
19

@30_Samu_03 I’m glad to hear it. Do you have any other questions about Nuitrack?

1 Like
20

For now I’m just waiting for Nuitrack’s next update in UE5 and have no more questions. Thank you very much for your help!! :))

1 Like
21

In the new plugin update, there was a small update to the fitting room sample https://www.unrealengine.com/marketplace/en-US/product/nuitrack-real-time-body-tracking

Manual settings have become automatic. And the examples of clothing now stretch (change the length of the bones) depending on the detected person. (metahuman and mixamo skeletons)

22