Make object follow hand

Nuitrack SDK Unity3D
1

How would I go and make a 3D or 2D object follow my hands using the hand tracker?
I tried using both the real and proj positions received from the handtracker. De proj positions which are between 0 and 1 are very odd and don’t seem to work right. The real positions work but not if you change your real life Z position. So walking closer or further away from the camera.

I’m using this code:

            Vector3 transformPosition2 = new Vector3(handCoupleData.RightHand.RealPosition.x,handCoupleData.RightHand.RealPosition.y, 200f);

            Vector3 cubeWorldPos = Camera.main.WorldToScreenPoint(transformPosition2);
            _manager.RightCube.gameObject.transform.position = transformPosition2;

I’m using my own class (handcoupledata) for the data but it returns the exact same data as the one received from unity since I just put it into my own class in the update event for hands. So that’s correct.

We only want to move the object on the X & Y. The Z needs to be fixed. We also tried converting the proj position to world, and the real position to screen. Any help? We used the RGB camera tutorial to visualize the camera on screen, so we can check if they’re actually on the hands position.

2

Anyone can help me? It’s a really odd problem

3

Hi Patrick,

Proj coordinates of hands in HandTracker are intended to be used as cursors. Hand movements are tracked in a limited virtual space, i.e. a user doesn’t have to move in front of a sensor to reach some point on the screen but just needs to slightly move his hand. Please take a look at our Creating an Interactive Multi-Touch Gallery for reference.

Perhaps, it’d be more convenient for you to use proj coordinates of hands not from HandTracker but from SkeletonTracker instead. You can find more info about this in our Displaying Skeletons on an RGB Image tutorial.

4

Thakyou, seems logical. But very weird at first since we expected to just follow it 1 by 1. Skeletontracker seems to be a bit heavy for using only hands. so we will check how the gameplay is with the real position of the hand tracker. The way you say it, it makes sense and maybe it works great. Thanks for clearing it up :slight_smile: