User Mesh Visualizer problem

[chang]

Dear Nuitrack Support/Team,

I have been using the Kinect V2 for a project, where I utilize a mesh-based approach to visually represent human figures. This setup allows for the depiction of human characters which, upon collision, trigger the start of a game. I am considering switching to Nuitrack for this purpose. Could you please inform me if Nuitrack supports visualizing human figures using a mesh-based approach similar to what I’ve been doing with Kinect V2?

Thank you for your assistance.

Best regards,

[Stepan.Reuk]

Hello @chang

@chang

visualizing human figures using a mesh-based approach

Could you explain in more detail what you mean?
A 3D avatar or a mesh built on the basis of a depth frame (Mesh point cloud, but only a person without a background)

[chang]

Hello,

Thank you for the clarification. I am interested in achieving the effect shown in the second image, which features a detailed mesh model with textures and clothing. Could you provide guidance on how to integrate such a detailed mesh-based human figure into my game? Are there specific formats or processes within Nuitrack that facilitate the incorporation of this type of model for real-time interaction and collision detection?

Best regards,

[Stepan.Reuk]

At the moment, we do not have any relevant ready-made solutions. And it is also difficult to advise something, since there have been no similar requests before.
At the moment, there is a mesh point cloud on AllModulesScene, where you can turn off the background (only segments of people will remain). And there is some interaction with physics in the “AR Nuitrack” tutorial. A mesh point cloud is also drawn there, and colliders are placed based on the skeleton to interact with the environment.
A detailed video on this tutorial: https://www.youtube.com/watch?v=vcAwxuPvTyM

I hope this will help in the development of your project

[chang]

Apologies for the lack of clarity in my previous message. I aim to achieve an effect similar to what is depicted in the image. Could you please confirm if the “AllModelsScene” you mentioned can be placed within a 3D scene?

Thank you for your assistance.

[Stepan.Reuk]

I think yes.

And I even see two options:

• You can add the “UserTrackerVisMesh” prefab to your scene. And use your script to call the method of enabling and displaying the background. Then there will be a 3D model of a person on the stage
• or you can use objects from the SmoothSegmentSample scene. Then you will have a 2D UI man with smoothed edges.

[chang]

I thought of using “UserTrackerVisMesh” but I don’t know how to use it
In addition to “NuitrackScripts”, what other things need to be added?

[Stepan.Reuk]

@chang

1. Drag to the scene prefab UserTrackerVisMesh
   
   
2. Edit UserTrackerVisMesh.cs (enable visualizing and disable background by default)
   
   
3. Set Camera position to (0,0,0) (and rotation)
4. Connect sensor and play scene

P.S. minimal three objects Camera, NuitrackScripts and UserTrackerVisMesh

[chang]

I made it! Thank you so much!!!

[Stepan.Reuk]

Do you have any more questions about Nuitrack?

[chang]

hi, i just wondering if my unity game with intel realsense d435 and nuitrack can be used on android tv?
because i saw the picture below.

[Stepan.Reuk]

AndroidTV is not officially supported currently.
We could consider adding its support if you have a commercial demand for some licensing volume.
If you plan to distribute your application with 100 depth sensor or more - please connect us through an Nuitrack Enterprise feedback form or directly at nuitrack@3divi.com