Nuitrack Knowledge Base


#1

Support for Devices and Platforms

Do you support other Linux distributions besides Debian/Ubuntu?

At the moment, there is official support only for Debian/Ubuntu distributions. You can try to install Nuitrack by unpacking the .deb archive and running the postinst script.

Do you have any samples that utilize Java or even Xamarin.Android since the API has .NET bindings?

We have a simple Xamarin.Android sample.
It has no GUI, it prints log messages to an Android console only. However, it shows how to use Xamarin.Android to build apps for VicoVR.

Can I make an Unreal Engine project with Nuitrack on Windows?

At this moment, Nuitrack SDK supports Unreal Engine projects for Android only.
You can find a tutorial about Nuitrack usage in Unreal Engine here.
Unfortunately, projects for Windows are not supported yet.

Does your SDK work with Windows 10 UWP applications?

Current Nuitrack SDK uses native С++ libraries built for Windows x86/x86_64 architecture. Besides it has some 3rd-party dependencies (drivers for depth cameras) which are available only for a desktop platform.

Can I run Nuitrack on Ipad?

Unfortunately, Nuitrack doesn’t run on an Ipad at the moment.

Installing and Uninstalling Nuitrack

How to install Nuitrack on my PC?

Please, take a look at Installation Instructions in our online documentation.

How to uninstall Nuitrack on Linux?

To uninstall Nuitrack, use the following command:
sudo dpkg -r nuitrack

Questions about Nuitrack License

I cannot enter the secret key because the Activation button is greyed out!

Make sure that you have installed the msi packages from the downloaded Nuitrack archive and Microsoft Visual C++ Redistributable for Visual Studio (steps 2 and 3 from our Installation Instructions. You need these packages installed on your computer to resolve missing dependencies for the Nuitrack module.

After that, run Nuitrack.exe tool and press ‘Compatibility Test’. You should see the window with the depth map. After a few seconds, the window will close and you’ll be able to enter your activation key.

After finishing the Compatibility Test, you should see a file “e24b…” in the %NUITRACK_HOME%/data folder. If it’s not created, please, check the access permissions for %NUITRACK_HOME%/data folder.

If you use RealSense, please check that it’s connected via USB 3.0 and camera firmware version is 5.8.15 or higher.

I bought the license but didn’t receive the secret key.

The secret key and instructions on license activation are sent automatically to your email after you purchased the license. Please, check your ‘Spam’ folder. If it doesn’t help, please contact info@3divi.com and we will duplicate the activation key for you as soon as possible.

I get the error: “Network Error (99) : Connection timed out”.

Please check your network connection as it is required for Nuitrack activation (your device should have access to http://download.3divi.com/).

I updated the software and now I cannot apply my license.

You have to reactivate Nuitrack using the same depth sensor that you used for the first activation. You don’t have to purchase a new license, you can use your activation key.

How can I reactivate the license?

The reactivation process of Nuitrack on a different PC with the same sensor is as follows:

  1. Plug in the sensor and run Nuitrack activation tool;
  2. Click “Compatibility Test”;
  3. Wait until the test is finished and enter the license key;
  4. Click “Get Available Licenses”. The tool will show that you have “0” licenses BUT this means that you have 0 NEW licenses; your old licenses can still be used;
  5. Click “Activate”.

Please note that you have to use the same sensor (with the same serial number) that you used when activating Nuitrack for the first time.

I enter the license key but receive an error “failed: wrong client key”.

Please try to copy your key from the Activation letter (check your email) and paste it to the “Client key” field. Please note that register matters.

I purchased too many licenses! Can I get them back? Can I deactivate them?

For testing or temporary use, we provide customers with Nuitrack Free Trial

In case you have already activated the sensor, you can not deactivate it as this procedure is not provided. To activate a new sensor, you need to purchase another license.

When registering the order, you can specify the same email address and 1 license will be added to your activation key automatically.

What is the difference between Nuitrack Trial and Pro? Can I use Nuitrack for development?

Nuitrack Trial is for demo purpose only and may not be used for development of applications (Project Content) for commercial use.

Nuitrack Pro may be used by anyone who pays the applicable subscription and / or unit license fees to develop, distribute, sell and offer for sale Project Content. Nuitrack Pro license is required for each copy of Project Content.

To remove the time limit, you need to upgrade the Nuitrack Trial to Nuitrack Pro (via license purchase).

You can download Nuitrack Trial here.

SDK and examples are available here.

Online documentation is available here

I get LicenseNotAcquiredException in my app, though standard Nuitrack work fine.

Make sure that you don’t specify the path to nuitrack.config file in the nuitrack::Init() function. If you specify the path, the file would contain incorrect path to the license.json file. The license.json should be located at the initial folder (when it was created).

When I try to run the nuitrack_license_tool in Ubuntu I always get "Please run with sudo" and when I run with sudo I get "Please run with "sudo -E"

Did you log out after Nuitrack installation? Please take a look at the 3rd step here.

Nuitrack Features and Functionality

What are minimum specifications for Nuitrack?

There are no strict requirements for hardware performance in Nuitrack. RAM usage is below 200 Mb. There is only one restriction for desktop processors - they should support AVX instructions set.

Does Nuitrack provide coordinate mapping between color frames and depth frames?

For Orbbec, please,

  1. Open the file %NUITRACK_HOME%/bin/OpenNI2/Drivers/orbbec.ini
  2. Uncomment the line ;Registration=1 in the [Depth] section (delete “;”)

For ASUS Xtion2, please,

  1. Open the file %NUITRACK_HOME%/bin/OpenNI2/Drivers/SenDuck.ini
  2. Add a line "Registration=1" to the [Depth] section. Also, add the section [Image] and a line "Resolution".

As a result, you should see the following code:

[Driver]
;Nite support mode. 0 - off (default), 1- On
;NiteMode=1

[Depth]
; 0 - QVGA, 1 - VGA, 4 - QQVGA. Default: Arm - 4, other platforms - 0
Resolution=1
Registration=1

[Image]
Resolution=1

For RealSense, please,

  1. Open the file data/nuitrack.config
  2. Find the section “Realsense2Module” and set “Depth2ColorRegistration” to “true”.
Can I select one of connected sensors?

This feature is in our road map. Currently, Nuitrack starts with one of connected supported sensors: Nuitrack loops over supported cameras and starts with the first detected camera.

Can I use Nuitrack with multiple sensors?

Unfortunately, at the time Nuitrack does not support connection of multiple sensors. Nuitrack works with the first supported sensor connected to a PC.

Is it possible to get the IR data using your SDK?

Unfortunately, it’s not possible to get the IR data using Nuitrack SDK, only depth and RGB data.

Can I use OpenCV 3.4 with Nuitrack?

Current Nuitrack version is built with OpenCV 2.4. Upgrading to OpenCV 3.4 version is in our road map, however, it will take a lot of time to implement it.

How can I set up tracking of multiple users?

You can track up to 6 skeletons by two ways:

  1. Using Nuitrack API: take a look at SkeletonTracker::setNumActiveUsers (C++) or nuitrack.SkeletonTracker.SetNumActiveUsers (C#) method in Nuitrack online documentation

  2. Editing nuitrack.config file: in the section “Skeletonization”, set “ActiveUsers” to the desired value (up to 6 skeletons)

What is the difference between Hand Tracker Module and "JOINT_RIGHT_HAND" from Skeleton Tracker Module?

HandTracker returns 2D (screen) coordinates of a cursor, 3D positions and grab state. This is an add-in module above the skeleton (it uses positions of skeleton’s hands). In this case, 3D position of a hand is the center of mass of the point cloud of a hand.

How can I enable RGB stream for Orbbec Astra Pro?

To enable RGB stream for Astra Pro, you should set "CameraID" option in the nuitrack.config file (usually it’s 0, but if it doesn’t work, try 1, 2, …).

How can I change RGB resolution for Orbbec Astra Pro?

Please try the following option:

  1. Open nuitrack.config file (from %NUITRACK_HOME%/nuitrack/data);
  2. Find the section "AstraProPerseeDepthProvider";
  3. Add the following lines:
    "RGB": {
    "Width": ---,
    "Height": ---,
    "FPS": --
    }

Set the values that you need.

How can I change RGB resolution for ASUS Xtion2?

You can do that either editing nuitrack.config file:

  1. Open nuitrack.config file (/usr/etc/data/nuitrack for Ubuntu and %NUITRACK_HOME%/data for Windows);
  2. Find the section
    "OpenNIModule": {
    "FileRecord": ""
    }
  3. Add the following lines after the '"FileRecord": ""' line:
    ,"RGB": {
    "Width": <width>,
    "Height": <height>
    }
    where <width>, <height> is one of supported resolutions.

or using Nuitrack API:
After Init(), call…

C++:
Nuitrack::setConfigValue(“OpenNIModule.Depth.Width”, “640”);
Nuitrack::setConfigValue(“OpenNIModule.Depth.Height”, “480”);

С#:
Nuitrack.SetConfigValue(“OpenNIModule.Depth.Width”, “640”);
Nuitrack.SetConfigValue(“OpenNIModule.Depth.Height”, “480”);

How can I change depth resolution for Orbbec Astra?

To change resolution for Orbbec Astra, open the file "/usr/local/lib/nuitrack/OpenNI2/Drivers/orbbec.ini" (for Linux) or "NUITRACK_HOME/bin/OpenNI2/Drivers/orbbec.ini" (for Windows) in a text editor and set the Resolution value to 0 in the [Depth] section.

How can I change depth resolution for RealSense?

Please, open %NUITRACK_HOME%/data/nuitrack.config file in a text editor. Add "RawWidth", "RawHeight", "FPS" options to the “Realsense2Module.Depth” section, for example:

"Realsense2Module": {
"Depth": {
"RawWidth": 640,
"RawHeight": 480,
"FPS": 30,
"Preset": 5,
"ProcessWidth": 640,
"ProcessHeight": 480,
"ProcessMaxDepth": 5000,
"LaserPower": 1.0
},
...
}

RawWidth=848, RawHeight=480, FPS=30 are default values. You can try to set other compatible values in accordance with RealSense SDK.
"ProcessWidth", "ProcessHeight" is the resolution which raw input stream will be resized into (it will be upscaled/downsampled and cropped to fit aspect ratio if needed). Please note that changing these parameters may lead to performance issues.

How can I disable the RGB stream?

To disable the RGB stream:

  1. Open %NUITRACK_HOME%/data/nuitrack.config in a text editor;
  2. In the "DepthProvider" section, find the “RGB” parameter and set it to "Off".
How can I select a different rotation angle?

You can set the rotation angle in the nuitrack.config file. Open the %NUITRACK_HOME%/data/nuitrack.config in a text editor, find "DepthProvider.RotateAngle" option and then set one of possible values: 0 (default value), 90, 180, 270.

What is the difference between Nuitrack SDK and Astra SDK?

Astra SDK is based on Nuitrack SDK (it uses skeleton tracking from Nuitrack). New features are implemented first in Nuitrack SDK and then in Astra SDK (with a possible delay). Besides, all new features not connected to skeleton tracking (for example, face tracking) are not implemented in Astra SDK.

Can I use the pre-recorded video as input with Nuitrack?

You can use the data recorded in .oni format using NiViewer tool from OpenNI2 SDK. This usage presupposes that the depth sensor should be OpenNI2 compatible, for example, Orbbec Astra cameras.

To run Nuitrack with .oni file, open %NUITRACK_HOME%/data/nuitrack.config file in a text editor and set the FULL path to the .oni file in the "OpenNIModule.FileRecord" section. Please note that Nuitrack Pro license is not compatible with this mode.

How can I limit CPU usage by Nuitrack?

"CPU.MaxLoad" parameter in nuitrack.config file limits the CPU usage by Nuitrack libraries. By default, this value is 0.67. You can modify this parameter, however, the decreased value may lead to frame skips.

Issues while Setting Up and Running Nuitrack

When I run Nuitrack samples, I get the following error: Start nuitrack c11 WARNING: Can not load library module:C:\nuitrack/middleware/NuitrackModule.dll ERROR: Empty factory for DepthProvider terminate called after throwing an instance of 'tdv::nuitrack::TerminateException' what(): Empty factory for DepthProvider

Usually, this issue is caused by unresolved dependencies.

Please check that you’ve installed:

  1. Microsoft Visual C++ Redistributable (for Windows)
  2. OpenNI 1.5 from Nuitrack archive (<install-folder>\OpenNI-Win32-1.5.7-Dev.msi, <install-folder>\OpenNI-Win64-1.5.7-Dev.msi for 64-bit).

Nuitrack won’t work without these components.

I use “Nuitrack.NuitrackMode.DEBUG” for Nuitrack initialization but the sample does not run!

Please do not use "Nuitrack.NuitrackMode.DEBUG" as it is only used for Wi-Fi connection between PC and a sensor.

Can I hide the “error messages” displayed at Nuitrack start?

We’ve already hidden some of these warnings in our latest release. However, we cannot hide some messages because they come not from Nuitrack, but from sensor drivers (such as "Can't find video camera!", etc.).

nuitrack_ni_sample doesn’t work with my RealSense!

nuitrack_ni_sample shouldn’t work with RealSense cameras as it works only with the sensors with OpenNI support.

I measure Skeleton Tracking FPS but the results are not stable!

We recommend you to disable RGB stream or downgrade the resolution of the RGB stream to qVGA for more stable FPS measurement.

When I run nuitrack_c11_sample.exe. I get these errors: ERROR: AstraProPerseeDepthProvider can't create RGB Stream (VideoCapture device ID is not valid). DepthProviderManager: AstraProPerseeDepthProvider can't create RGB Stream (VideoCapture device ID is not valid). Can't find video camera! Can't find Depth camera! ERROR: Couldn't open device ( DeviceOpen using default: no devices found). DepthProviderManager: Couldn't open device ( DeviceOpen using default: no devices found). ERROR: Can not create OpenNI depth generator (OpenNI Status: Can't create any node of the requested type!). DepthProviderManager: Can not create OpenNI depth generator (OpenNI Status: Can't create any node of the requested type!). Create DepthProvider: Realsense2DepthProvider. What did I do wrong?

These error messages are displayed for diagnostic purposes only. If you see the line "Create DepthProvider: Realsense2DepthProvider", that means that Nuitrack works fine. The console output is displayed only when a user is detected.

I encounter IndexOutOfRangeException!

Try to:
Open the file %NUITRACK_HOME%/bin/OpenNI2/Drivers/SenDuck.ini
Add the section [Image] and a line “Resolution”:
[Image]
Resolution=1

When I run the sample, I get an error: _"System.DllNotFoundException: 'Unable to load DLL 'libnuitrack': The specified module could not be found. (Exception from HRESULT:0x8007007E)'"_

This error is caused by mixing 32-bit and 64-bit libraries. Please make sure you’re using the 64-bit version of Nuitrack and a 64-bit compiler. Also set "Target platform" to x64 in your project’s properties.

I cannot open Nuitrack sample as a root user.

We provide automatic settings for a non-root user. As for a root user, you have to set environment variables manually. Please, execute these commands as a root user:

export NUITRACK_HOME=/usr/etc/nuitrack
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/lib/nuitrack

Sample SDK projects cannot execute without libnuitrack.dll. Where is libnuitrack.dll?

You have to install the Nuitrack runtime component. Please, follow the installation instructions.

Issues with Nuitrack and Unity

I cannot build my Unity project with Nuitrack!

Make sure that you installed Nuitrack correctly following or Installation Instructions.

Also make sure that you have installed the msi packages from the downloaded Nuitrack archive. You need these packages installed on your computer to resolve missing dependencies for the Nuitrack module.

Try to run Basic Unity Sample from Nuitrack unitypackage to check whether Nuitrack works fine or not: import NuitrackSDK.unitypackage to your Unity project and run “BasicSkelet” from the "Scenes" folder.

I get “DllNotFoundException: libnuitrack” error!

Make sure that NUITRACK_HOME and PATH environment variables are set correctly and for all users. You have to restart Unity after setting these variables.
In Unity, select “Nuitrack SDK/Nuitrack/NuitrackAssembly/nuitrack.net.dll” and tick "Any Platform" in the “Inspector” tab.

When I run Nuitrack samples, my console log is filled with errors stating that "The type or namespace name `nuitrack' could not be found. Are you missing an assembly reference?".

Please, try to select Nuitrack SDK/Nuitrack/NuitrackAssembly/nuitrack.net.dll (in Unity) and tick "Any Platform" in the “Inspector” tab. This issue may occur in case this value is unticked.

I'm getting this error when starting the Unity demo scenes: Nuitrack native libraries initialization error: INIT_NUITRACK_NOT_SUPPORTED

This error occurs at start of a Unity project with Nuitrack, however, it does not indicate any particular error. So don’t worry, this is a regular case and everything should work fine.

Questions about Nuitrack and TVico

Can I transfer depth data, user masks and skeleton data using TVico and PC?

For Linux and Windows desktops, there is a limitation in the current Nuitrack version: only skeleton data can be transferred via wireless connection.

If you connect TVico to an Android device, you can transfer all data (depth, user masks, skeleton).

Can I connect TVico via Ethernet or Wi-Fi?

You can use either Ethernet or Wi-Fi for connection. For more information about connecting TVico to an Android device/PC, take a look at our TVico USer Guide.

What is the difference between TVico and Orbbec Persee?

TVico is an interactive Android computer developed jointly by Orbbec and 3DiVI Inc. It is a Persee camera supplied with Nuitrack middleware and Nuitrack perpetual license. You can use TVico to transmit depth data, user masks and skeleton data to other devices using Wi-Fi. Android, Windows and Linux platforms are supported.

Nuitrack Documentation

Where can I find Nuitrack documentation?

You can find Nuitrack online documentation here.

Is there documentation for nuitrack.config file? I want to change all the parameters!

Unfortunately, there is no documentation for nuitrack.config file at the moment as it is not intended to be edited in most cases. It includes internal parameters for modules and experimental features.


#2

#10