HandTracker not working (Intel Realsense D415)

Nuitrack SDK Unity3D
1

I want to place a camera in front of a glass door. When a user approaches the door, I’d like the camera to detect the person’s face (and get some info from that person such as age group, gender etc…) . Then following a specific hand signal from that user, I want the door to open.

The hand signal would be something like a hand/arm swipe ( https://www.dropbox.com/s/j0jb6yt95grafj4/Designer-drawing-2-Documenting-an-arm-swipe-interaction-by-drawing.png?dl=0 )

The HandTracker demo that comes with the unity package doesn’t seem to work at all, in that it is not picking up my hands and I cannot seem to interact with the gallery.
Now I’ve read on this forum that Hand tracking seems to be fairly unreliable with Intel cameras, is that true? could that be the cause? all of the other tutorials are working great.

I’m asking because (coming back to the beginning of this post), maybe I should select other joints, such as elbows and wrists which are perhaps more reliable for the purpose of picking up a “swipe left” gesture…

I understand that this post contains 2 separate questions: why is the Handtracker tutorial not working? and should I rely on the Hand tracker at all in the first place?

Would be helpful if you could point me in the right direction.

2
  1. No idea, maybe some references in scene are lost. Maybe try to follow the tutorial from scratch.
  2. Also no idea, we used to spawn a invisible skeleton for games that only use hands. But we’re going to try to convert this to hand tracker soon. We tried a bit and the hand data does seem verry shaky to say the least. a lot of glitching etc. Just try it out yourself.
3

Thanks Pklokgueters…

Unfortunately, no luck even when building it from the start. What do you mean by references being lost in the scene?

4

Also with regards to the Handtracker, have you thought about using wrist as a joint instead?

5

Yes, thats what we did before this, but that requires skeletontracker to be enabled when only hands are needed. Thus unnesecary performance loss. also i meant references in the scene which might prevent the handtracker to spawn things/use objects.

6

Sure. Unfortunately I’m still very much stuck.

7

Could this somehow be linked to the nuitrack.config file I wonder?

8

Shoudln’t be. We need more information, your code, setup, what you’re trying etc

10

Hi Giovanni,

  1. The Gallery tutorial should work fine. Please check that you ticked Hand Tracker Module in Nuitrack Manager. Do you see any errors displayed in the Unity log?
  2. As we understand, you need the “swipe” gesture from the Gesture Recognizer Module (you don’t even have to use the Hand Tracker Module to use gestures):
NuitrackManager.onNewGesture += NuitrackManager_onNewGesture;

private void NuitrackManager_onNewGesture(nuitrack.Gesture gesture)
{                
     if (gesture.Type == nuitrack.GestureType.GestureSwipeLeft)
     {
     }

     if (gesture.Type == nuitrack.GestureType.GestureSwipeRight)
     {
     }

     if (gesture.Type == nuitrack.GestureType.GestureSwipeUp)
     {
     }

     //etc
}

Or you can use the skeleton data from the Skeleton Tracker Module to create your own gestures. You can see the example in our Nuitrack SDK Unity package: see the script GalleryControl.cs. Also don’t forget to tick the Gesture Recognizer Module in Nuitrack Manager.