Nuitrack and Unity for Oculus Quest

Nuitrack SDK Unity3D
1

Hi everyone,
I’m trying to develop a project in Unity using Nuitrack skeleton tracking with Intel RealSense Depth Camera D435i, when I press play in the Unity editor the skeleton detection is activated, but as soon as I build the project on my Oculus headset I see the avatar motionless, as if it doesn’t activated skeleton tracking.
I’ve also tried with other projects tutorial that i found online and i have always the same problem.
For my project it’s important to see the animation of the avatar while wearing the headset, to feel completely immersed in the scene.

Hope that someone could help me,
Thanks

2

Hello @g14
Have you completed this tutorial?
Two years ago, it’s worked with Oculus Quest
https://github.com/3DiVi/nuitrack-sdk/blob/master/doc/Unity_Zombies_Oculus.md

3

Hi Stepan,

Yes i have tried also this tutorial. When i press play in the Unity Editor the Realsense starts the tracking of the skeleton and everything seems to work, but when i wear the headset after the build, i don’t see the movements detected from the camera, All i see is the surrounding environment but everything is stopped.

4

Any ideas to help me, please?

5

@g14
I will try to investigate this issue. If you can send a video of what it looks like in VR, would it be great?

6

Hi stepan,

I can’t send you the video, but i can explain you more details about what i can see:
i have an avatar in my unity scene, when i press play in the unity editor my realsense camera activates and starts to recorder the movements that i do. After this i build my project on my oculus connected via Quest link with che pc. When i open the project on my oculus i can see the scene and all the game objects, but the realsense doesn’t activate, so the skeleton tracking doesn’t work.

I’ve tried installing the Nuitrack.apk on my Oculus and connecting the realsense camera with the Oculus via cable. When i try Nuitrack i can see what the camera records, but in the unity project it doesn’t work

7

@g14

Everything seems to be working fine.
Tested on:
Unity 2022.3.15 OpenXR
Nuitrack 0.37.12
Realsense L515

If you create an empty scene in Unity (without Nuitrack), and run it, will VR via link work fine?

Nuitrack.apk is not needed in this case

8

Hi Stepan,
without building the project on my headset, just pressing the run button, the game view in the editor become black and also on my headset i see everything black.
Until now i have always tested all my projects doing the build of the projects on the oculus.
I don’t know why i can’t see anything when i run the unity editor, but when i build the projects on the Oculus Quest i can see everything clearly.

9

This is strange. Do I understand correctly that this problem does not depend on “Nuitrack”?

10

i think so,
but i’m tryng to create an apk for the oculus, so everything should work when i build the project on the headset, not only when i run in the unity editor

11

@g14

I don’t quite understand how the apk and Oculus Link are connected (it’s only for Standalone (?)). I think that if you make a Standalone build, everything will work well. And it seems that the presence/absence of Nuitrack in the project does not affect the launch in the editor.

Do you have any more questions about Nuitrack?

12

When i make the build of the project on my oculus and i open it i see my avatar without movements. i don’t understand how i have to connect the intel real sense to the oculus to activate it when i open the project. i’ve tried also connecting the intelrealsense to the oculus via usb-c, but it doesn’t work, it is like the camera does not turn on.

The problem seems to be the connection between oculus and camera with nuitrack.

13

are you trying to build an apk?

14

I am doing the build of the project from Unity Editor, it creates an apk of my project in the Oculus

15

Unfortunately, it won’t work that way.
It should work if you build a standalone build (.exe), connect VR via Oculus Link, and run the game on PC.

16

So if i want to have a game on Oculus, made with Unity, using just Oculus and a cam with nuitrack it does’t work?
I always need the pc with Unity connected with the headset?

17

Your application must be running on the same device that the sensor is connected to. This is the easiest way.
In this case, your standalone application will stream via Oculus Link (via Wi-Fi or cable) to your Quest