Hi there, I’m using Nuitrack in Unity for body tracking in a VR app. I wanted to know what settings I can configure to get the lowest possible latency?
-Which camera would provide the lowest latency?
-What Settings in the Unity SDK/nuitrack sdk could be configured to reduce latency?
-Any interop optimization that can be done between the camera/SDK/and unity communications?
We are targeting <50ms end-to-end latency in order to achieve immersive body tracking in VR.
So we would need to optimize anything that we can throughout the pipeline.
What is your target platform? If it’s Android, you can use one of our products: VicoVR or TVico.
VicoVR allows you to receive SkeletalTracker data via Bluetooth / WiFi with a delay of 33ms (30 FPS) on Android.
Hi Iosif,
We are targeting the Oculus Quest, which runs android. Unfortunately, the 33ms you describe is the latency between each frame, but does not describe the total motion-to-photon latency - the time it takes the camera to acquire the depth map, transfer over to pc, any type of frame buffering, skeleton solving, or transfer latencies between each layer.
For example, using the realsense D415 @ 60fps on the Nuitrack Zombie Oculus Rift Demo, shows that it takes 7 frames (116 ms) for the human depth map on the TV to change after the real life person has moved. For comparison, the oculus quest and rift controllers have about 50 ms (3 frames) of motion-to-photon latency.
We suspect that certain depth cameras may have lower latencies. Not sure why TVIco would have a lower latency unless this has been specifically optimized. Please advise on this issue.