How to use hand tracking to interact with the UI in Unreal Engine 5.6

Nuitrack SDK Unreal Engine
1

Hi,

I want to use the hand tracker from the template so I can interact with the UI. But I can’t find any Unreal Engine tutorials.
Can you tell me how I can interact with UI? Or Or you can share the file in which all this is done.

Your team has provided numerous video tutorials and online help resources for Unity, but there is currently no similar support available for Unreal Engine.

Please upload video tutorials for Unreal Engine on your YouTube channel so that more people can learn by watching the tutorials.

Thanks

Sandeep

Hand2DCursor (1)

2

Hello @Kool

At the moment, we do not have a tutorial on interacting with the UI for Unreal Engine, but it is planned in the near future.

What types of interaction would you like to use for your project (button clicks, hover and hold, drag or something else)?

3

hi,

Thanks for reply.

I need all type of interaction like button clicks, hover, hold and drag.
If you can’t provide a tutorial.
Can you give me the Unreal Engine 5 file that contains all these interactions so I can study the interaction blueprint system from that file?

This will help me a lot

I know tutorials take a lot of time, but if Unreal Engine also had tutorials like Unity engine, a lot of people could do a lot with this plugin.

Thanks in advance.

Sandeep

4

Thanks for your interest, the Unreal plugin appeared much later than the Unity package, so there are fewer tutorials. We are trying to close this gap, probably the next one will be in terms of interaction with the UI.

So far, there are no files on how cursors interact with the UI. Only separate cursors at the ”HandTracker“ level and UI at the ”Virtual Fitting Room" level. I’ll do some research this week, and if I can make a file, I’ll definitely send it to you.

5

Thank you so much for taking my problem so seriously.

I am eagerly waiting for this file. And I’m confident you can do it.
I hope my problem gets solved as soon as possible.
Finger crossed. :crossed_fingers:

6

The first test implementation appeared.

So far, this is not as common an implementation as Unity. You will need to implement the logic for each element separately.

Windows cursor moves using the right hand position. It only works with one button.

It is recommended to study the logic of operation

  1. Replace the file https://drive.google.com/file/d/1rIiPgRqIK7qwwnUdW7BTSo5Cfa8H0kYu/view?usp=sharing
  2. And start the Hand Tracker Level

image
image1797×710 199 KB

7

Thank you so much. I’ll try to apply some logic to this..:grinning_face:

8

@Kool did you manage to set up interaction with the UI?

9

Yes, it is working very well. I tried some logic and I am using it in my project also. Thank you so much.

1 Like