Different RGB Resolution for Viewing and for Sensing

Nuitrack SDK Unity3D
1

Hi,

We are using the Nuitrack Unity SDK and currently running with an Orbbec Astra Plus.
Using it as is the skeletal tracking works flawlessly, however when we increase the RGB resolution the tracking’s accuracy reduces significantly to the point it does no track at all.
When using it at 640x480 it works best, increasing it to 1280x720 it fluctuate sometimes and at 1920x1080 the tracking only works sometime.
While we want tracking to be as best it can we also want to view the highest RGB frame the camera can output, and the middle ground (1280x720) is not working for our partners.

This was tested both in Windows x64 and in Andoid ARM64.

Is there a way to separate the RGB frames to a different resolution for tracking and different resolution for viewing the frames?

Perhaps we can output the highest available, and then feed into Nuitrack a downsized frame (we can downsize using opencv)?

*Also worth noting that due to us using Android we are not using Nuitrack AI, which might fix that.

Thank you
Ofek

2

Hi @ofek

Are you only changing the rgb resolution? Does the depth remain the default?

3

Hi irakli,

Yes only the RGB resolution. Depth remains at the default, which I think is 640x480.

Thank you
Ofek

4

Okay, we’ll check on our side and get back to you.

5

We were not able to reproduce the problem on our side. Could you please make a record of the problem (instructions) and send it to us (e.g. via a link to gdrive in private messages or to the support mail).

Before recording add the following fields for OrbbecSDKDepthProviderModule to the config(NUITRACK_HOME/data/nuitrack.config) to change the resolution

"OrbbecSDKDepthProviderModule": {
        "Depth": {
            "ProcessWidth": 640,
            "ProcessHeight": 480,
            "AutoExposure": true
        },
        "RGB": {
            "ProcessWidth": 1920,
            "ProcessHeight": 1080,
            "AutoExposure": true
        }
    },
6