New upcoming SDK

[Shogenn1]

Hello there,
I was wandering if we could get some informations about the promising upcoming SDK
Will it be aired in May?
Will it significantly improve the tracking?
Processing time will still be reasonnable with deep learning on RGB image?
(I tried a bit Open Pose and it was way too much resource-consuming)

Thank you for your hard work !

[Dmitow]

it is too early to definitely comment about new AI version)
target is end of 1H2019, not May though.
It is got a great potential but at the beginning it is unlikely quality of tracking will be better than the current Nuitrack version.
as for processing time idea that combination of depth-sensor data and AI can lead to dramatically smaller networks needed for the same quality outcome.

[vpenades]

Hi, I would like to know if, for the new SDK, a number of issues will be fixed, in particular, I’m interested in these:

C# Wrapper Garbage Collector performance

I’ve noticed that when running NuiTrack with the C# wrapper, the GC is running almost continuosly; see VS memory diagnostic window below. This happens even with all the most basic examples. Tracking the memory objects being created and disposed, I noticed that new DepthFrame, ColorFrame, UserFrame objects are created, including their internal Byte[] Data arrays. but they’re not reused in the next frames. This may be convenient for naive, simple or short lived applications but for applications…

Kinect2 wrong depth frame size

@jel.cos Indeed, depth frame resolution is resized to 640x480 for further processing by Nuitrack. You don’t have to change anything in nuitrack.config because Nuitrack processes the depth frame in this resolution (640x480). @vpenades Nuitrack uses the OpenNI2 driver for OpenNI-compatible cameras (including Kinect), which causes noise as shown in your screenshot.

and this:

Sensor intrinsics and correlation

ColorSensor doesn’t include information about depth, so converting the real coordinates to projective coordinates is impossible without the depth data from DepthSensor. You can align a depth image to color image and use the converting methods from DepthSensor.

[olga.kuzminykh]

Hi Vpenades,

We’re working on the first two issues. As for the third one, you can actually get the frame size and FOV using Nuitrack API (check GetOutputMode and OutPutMode). In one of the next releases, you’ll be able to get the name of the used sensor.