Hopefully someone can point me in the right direction as I don’t have the equipment always available to me to test. But when I do use D435, Unity and the trial version of nuitracker for now.
I followed the basic tutorial nuitrack-sdk/Unity_Basic.md at master · 3DiVi/nuitrack-sdk · GitHub and got it all set up. Since I only want to use hand tracking as the interaction I removed the other joints. I then spawned a bunch of falling cubes (Z axis = 0) but now I have no way to interact with them because the hand have a different deth (Z axis value).
What would be the easiest way to get only the (invisible) hands to interact with objects that are on a different Z axis or to rephrase how would I make the Z axis of the hands equal to the Z axis of the playing field/scene in Unity.
If I understood correctly, you just need to set the desired Z for the hand objects so that it coincides with the game objects. Something like this (This is a modified line from the basic tutorial. needed_depth - this is the value where your hands should be on the Z axis):
Vector3 pos = joint.Position
CreatedJoint[q].transform.position = new Vector3(pos.x, pos.y, needed_depth);
Thank you @Stephen.Reuk that is exactly what I wanted. I guess it’s not that hard if you know where to look to make the changes. I’ll have to dive into the code to understand it, but I like the flexibility and the potential it has for implementing different ideas.
I have 2 more questions if you or anyone else can chime in.
It’s listed that the tracker supports up to 6 people and I saw an example where they connect the skeleton joints with lines (2 people on the screen). I take it that I could use the same/similar approach if I wanted to do this type of hand tracking for up to 6 people?
The second question is about interaction with objects. I want to keep it simple since it will be open to public. If I wanted to “grab” an object only by coming into contact with it I could probably set the objects position and rotation as the child to the hand joint (meaning it will move with the hand). However, how would I go about releasing the “grabbed” object in that case? If there is a simpler or more error prone solution I’m all ears.
Yes. Since you are using the joints of the skeleton, you can use 6 pairs of hands. (just keep in mind that the more users, the more difficult it is to track them, they may not fit in the field of view of some sensors and overlap each other)
It’s hard to suggest anything specific. I think here you need to come up with something based on the gameplay of your game. Perhaps some special zones on the stage will come up, where the object will be automatically “unhooked” or by a timer. Here you need to experiment
I know that the room/space and the camera used are the main concerns, but thank you for pointing out the overlapping.
And thank you for giving me some ideas on how to go about implementing a release mechanic for a simplified grabbing interaction.
I don’t have an exact idea of what the game/interaction with the objects will be yet, but it’s good to know what’s possible to do and what the limitations are before starting a project.
Hello @Stepan.Reuk as a matter of fact I do have a quick question if you know the answer to. Is jump detection also supported for up to 6 people? I’ll be making an endless runner type of game but since I can’t test this by myself I do have to know so that I can plan the game accordingly.
Nuitrack does not have a separate module for determining jumps. There are several ways to determine the jump to Unity. One of the options can be viewed here:
I think this should work for 6 users as well. But experiments are needed
