Need help about nuitrack sdk device api for activation

So due to automation purpose we created a nuitrack licence activation tool, modified from this little project:

here are part of the source code we did (with extra debug informations):

int main(int argc, char* argv[]) {
	int errorCode = EXIT_SUCCESS;

	//nuitrack_device_activation -l license
	try {
		cxxopts::Options options("NuitrackActivation", "One line description of MyProgram");
		options.allow_unrecognised_options().add_options()
		("l, license", "license Key", cxxopts::value<std::string>(), "");

		std::string license;
		auto result = options.parse(argc, argv);
		if(result.count("l"))
		{
			license = result["l"].as<std::string>();
		}
		std::cout<<"license = "<< license<<std::endl;

		Nuitrack::init();

		const auto device = selectDevice();

		activateDevice(device, license);
	}
	catch (const LicenseNotAcquiredException& e)
	{
		std::cerr << "LicenseNotAcquired exception (ExceptionType: " << e.type() << ')' << std::endl;
		errorCode = EXIT_FAILURE;
	}
	catch (const Exception& e)
	{
		std::cerr << "Exception: " << e.what() << std::endl;
		errorCode = EXIT_FAILURE;
	}

	Nuitrack::release();
	return errorCode;
}

NuitrackDevice::Ptr selectDevice() {
		std::cout<<"selectDevice started "<< std::endl;
		std::vector<NuitrackDevice::Ptr> devices = Nuitrack::getDeviceList();
		std::cout<<"device grabbed "<< std::endl;

		if (devices.empty())
			throw Exception("No devices found.");

		console::Table<4> table({"Index", "Name", "Serial Number", "License"});
		std::cout<<"table header created "<< std::endl;
		for (std::size_t i = 0; i < devices.size(); i++)
		{
			const auto& device = devices[i];
			table.addRow({
				std::to_string(i),
				device->getInfo(DeviceInfoType::DEVICE_NAME),
				device->getInfo(DeviceInfoType::SERIAL_NUMBER),
				toString(device->getActivationStatus())
			});
		}
		std::cout << "\nAvailable devices: (0 will be used by default)" << std::endl;
		table.printTable();

		int devIndex = 0;

		return devices[devIndex];
}

void activateDevice(NuitrackDevice::Ptr device, std::string manualLicenseKey) {
		bool isActivated = device->getActivationStatus() != ActivationStatus::NONE;

		if (isActivated)
		{
			std::cout << "Device already activated" << std::endl;
		}

		if (!isActivated) {
			device->activate(manualLicenseKey);
			std::cout << "Activation status: " << toString(device->getActivationStatus()) << std::endl;
		}
}

(Not all source code are pasted here)

We want to run it on Linux (Ubuntu) for device activation, however, on some of our devices it works, but some doesn’t, it quits at the code: Nuitrack::getDeviceList, and the log “device grabbed” doesn’t appear. Can someone tells us what’s going on here?

More information:
Device 1 (working): Ubuntu 18.04.6, with GNOME UI, nuitrack 0.35.12, Kinect for xbox one camera
Device 2 (not working): Ubuntu 20.04, without UI (just TTY), nuitrack 0.35.12, Kinect for xbox one camera.
Device 3 (working): Ubuntu 20.04, with GNOME UI, nuitrack 0.35.12, Kinect for xbox one camera.

Also on device 2, license can be activated through nuitrack ui, however, without GNOME, running nuitrack_device_api_sample gives the same fail result.
On Device 3, we tested in tty, it has the same error behaviour, the device list doesn’t print. We tried to enable xserver, but with same fail result.

Update:

we can confirm that it’s caused by not running the program with GNOME, because we reproduced this issue with tty. However, we do need to know what exactly is missing, because on final product, we won’t have Desktop UI running.

Our other developer is debugging it and says nuitrack_GetDeviceList() is causing trouble, because he can’t access stacktrace after this line, and he can’t debug it because it’s extern without any debug symbols.

Hi @dongyiCai_retrobrain,

I will try to reproduce this issue.
Please answer the following questions:

  • Do you use the same sensor with all devices?
  • Does nuitrack_sample work on device 1 and device 3?
  • Was USB 3 used for all devices?
  • Are devices 2 and 3 the same device?
  • Do you use the same sensor with all devices?

No, all devices have different sensors. We also test with deactivated license with a method that delete license.json in one folder (can’t remember where exactly)

  • Does nuitrack_sample work on device 1 and device 3?

Yes, both nuitrack_device_api_sample and nuitrack UI itself are running on device 1 and 3, within GNOME Desktop, and Terminal opened.
Update: I didn’t realize you mean the test camera window, yes it works on both device 1 and 3 within GNOME Desktop. They’re units for application development and we also test this with production application. Nuitrack skeleton tracking are working fine.

  • Was USB 3 used for all devices?

Yes, they’re all USB 3. We’re using NUC 10 and 11, and both of them don’t have USB 2.0 externally. And we’re connecting the camera directly on the box without any usb dongles or something.

  • Are devices 2 and 3 the same device?

No device 2 and 3 are not the same unit.

Hi @dongyiCai_retrobrain,

  • We have reproduced the issue in tty environment and could confirm that nuitrack wouldn’t work in specific cases.
  • Currently such environments aren’t tested/supported normally during mainstream release preparation.
  • Main purpose of DeviceAPI is to provide ability to activate license automatically (inside end-user application), currently we didn’t intend to fully cover/support server-like environments.
  • We have added this issue into backlog, it should be resolved during future development (but it’s hard to tell exact timeline, because prioritization is mostly shaped by paid projects).

Thank you for bringing attention to this issue.

Hi @dongyiCai_retrobrain,

We tested and recorded a video demonstrating how nuitrack_device_api_sample works in tty-only mode on Ubuntu 20.04.
We recommend installing nuitrack v0.35.15.
Could you please confirm that nuitrack 0.35.15 also works for you in tty-only mode?

Sorry for late response.

I still can’t get it working, I updated my nuitrack to 0.36 on Ubuntu 20.04, and also fresh cloned your nuitrack-sdk-master from github. After compile nuitrack_device_api_sample, I still have the same issue on the built binary on tty.

This is the result in Terminal in Gnome

This is the result in tty:

I tried restart, remove and reinstall nuitrack package, still doesn’t work. Also, I’m testing the official nuitrack_device_api_sample code, not the modified one by us.
PS: if I execute nuitrack_device_api_sample directly, which is installed with nuitrack package, same thing happens. So for me this issue is not fixed.

I removed nuitrack, installed 0.35.15, restarted nuc after both uninstall and install, checked out 0.35.15 branch, compiled the nuitrack device api sample. Same issue happens, it works in Gnome Terminal, but didn’t work in tty

Hi @dongyiCai_retrobrain, as you possibly know we reproduced this issue with Kinect 2 sensor (headless activation should work ok for other non-Kinect sensors).
In the meantime - could you please tell what API do you use currently with Kinect 2 for tty-only (headless) environments?

We are asking because we faced indirect dependency on opengl through libfreenect driver (which makes it incompatible out-of-box with headless environments).

Hi @dongyiCai_retrobrain, we have released fix for headless activation of Kinect v2 on Linux.
It is readily available at GitHub - 3DiVi/nuitrack-sdk: Nuitrack™ is a 3D tracking middleware developed by 3DiVi Inc.. You are welcome to try it out on your side.
Please let us know how it works for you.

Hi, sorry for the late reply, I don’t get notification for replies so I didn’t know this thread is answered.

I’m testing 0.36.3 right now and it seems finally working now with default device api! I’m doing further test and try to integrate it with our system. We’ll let you know with further progress.

OK so for now I noticed 2 further issues,

  • After activation, there are 2 error messages. We can ignore them, but I’m not sure if it will bring further problem. See screenshot.

  • Sometimes the device activation will just hang (or freeze) before device prints. It happens to me twice, not always reproducable. If I cancel with ctrl+c and rerun it will work. Restart PC doesn’t really reproduce this problem. Maybe you can tell us what’s going on? Screenshot here:

My colleague tested my modified code (activate license without selecting device, skip “run Nuitrack”), and on 1 occasion, the program printed this:

list_video_devices: Error open dir pathCan't find video camera!
list_video_devices: Error open dir pathCan't find Depth camera!
Cannot identify '/dev/video0': 2, No such file or directory
Cannot identify '/dev/video0': 2, No such file or directory
Segmentation fault

And quit. There is no device list printing in between. As before, this is not reproduceable and it works on second try.
Do you know what could cause the issue?

Hi @dongyiCai_retrobrain,

Sorry for a delayed reply.

After activation, there are 2 error messages. We can ignore them, but I’m not sure if it will bring further problem

Thank you for bringing this issue to our attention, we will fix this in future updates

Sometimes the device activation will just hang (or freeze) before device prints. It happens to me twice, not always reproducable.

How often does this problem recur? Do you use USB 3?

My colleague tested my modified code (activate license without selecting device, skip “run Nuitrack”), and on 1 occasion,

What do you mean by "activate license without selecting device"?
To activate any sensor, you need to get a list of sensors, select the one you need, even if only one sensor is connected, and activate it.

Which of the problems currently has the highest priority for you?

How often does this problem recur? Do you use USB 3?

Yes we’re using USB 3.0 with kinect. And it occurs around 10% to 20% on my side, but we also found similar issue in actual game with nuitrack C# sdk. So this is not just happening in this device api’s case.

To activate any sensor, you need to get a list of sensors, select the one you need, even if only one sensor is connected, and activate it.

We have requirement that nuitrack needs to be activated before our product sent to client, and we need fully automatic device setup process. One of the setup step is activating nuitrack. The purpose of all of these, is to make it fully automated. The modified code will automatically select the first connected device, and activate it with given (through execution argument) license key.

Which of the problems currently has the highest priority for you?

The hang and freeze has the most priority for now, because it isn’t just affecting me, it’s affecting our product too, which sometimes also hangs when nuitrack is initializing.

One more thing: I’ll no longer be here after 2 or 3 weeks, so another colleague will take over the communication after that point. Also our CTO have contact with you. As developers we’ll probably still use this thread for device api related issue communication.

Hi @dongyiCai_retrobrain
Sorry for a delayed reply.

but we also found similar issue in actual game with nuitrack C# sdk. So this is not just happening in this device api’s case.

on which platform is this happening? (linux or windows)

Linux, Ubuntu 20.04 LTS, to be specific

Hi @dongyiCai_retrobrain,

We fixed the hanging problem in the new version 0.36.6 https://github.com/3DiVi/nuitrack-sdk.
Please check and confirm from your side that this problem is not reproducible.

Hello,

Thanks for the update, I did a few tests with newer version of nuitrack, and it doesn’t happen to me anymore. I already informed the team of my progress, and they will also do some further integration and testing on their side.

I’ll no longer be available starting the day after tomorrow. So further communication will be handled by someone else.