Python SDK crashing at load_modules

KooroshRH · February 28, 2024, 8:30pm

Hi,

I first tried the compatibility and working process of an IntelRealSense D455 with the Windows application and everything worked perfectly from activating the license to running the demo. But when I tried connecting to the same device using Python SDK, the code (the same as provided in the installation doc) suddenly crashed at the load_modules line without any error. It seems the code stops at that point.

gvr · March 5, 2024, 10:35am

Hello, @KooroshRH.

Apologies for the late reply.

We were unable to reproduce your problem. (we have tested it on the latest release v0.37.12.). Could you, please, provide us with the version of Nuitrack you used and the snippet of code? Also, we want to tell you that connection issues (bad cable or USB port) are common for RealSense sensors. In addition, we would recommend you to try to launch the latest version (link) if you haven’t already.

We’re sorry you have encountered such an issue. We’re looking forward to feedback.

KooroshRH · March 7, 2024, 2:36pm

Hi @gvr,

Thanks for your response.

I used the exact code explained here. I can’t find any cause because everything works well with the latest Windows application. The load_modules function doesn’t throw any exception I can debug.

gvr · March 14, 2024, 12:19pm

Hi, @KooroshRH.

Sorry for the late reply.

Could you, please, tell us about your actions step by step? Also, is your link referring to this code? (it’s quite tricky to get there). If not, could you, please, provide us with the exact code snippet? (via link, file, etc.)

Thank you for your interest in Nuitrack. We are eagerly waiting for the feedback.

KooroshRH · March 14, 2024, 6:56pm

Hi @gvr,

First, I connect my RealSense camera to the computer and run the Nuitrack desktop application. It works perfectly without any problem, but when I close the application and run the code from the link you sent (it’s the exact link I put in my previous response), it crashes at load_modules function without throwing any exceptions, so I can’t debug it.

Thanks

gvr · March 16, 2024, 8:58am

Hey, @KooroshRH.

After considerable testing, we can’t reproduce your issue. Let’s dig a little bit deeper.
Could you, please, describe your workflow as minutely as possible? It would be wonderful if you could include screenshots of console output, code (or a file), and the ‘NUITRACK_HOME’ and ‘LD_LIBRARY_PATH’ environmental variable settings. Also, please provide us with the actions leading to the crash (it would be nice if it’s presented in some kind of list). The info we ask for will help us solve your issue as soon as possible.

Thanks for your contribution. We’re looking forward to your response.

KooroshRH · March 18, 2024, 8:19pm

Hi @gvr,

Thanks for replying and helping to fix the issue.
I’m using Windows and installed the Python package using the wheels you provided in the GitHub repo (py_nuitrack_windows_python3.9-0.1.0-py3-none-any).

This is the code I’m trying to run completely:
(I can’t upload images due to a restriction, please find this image in this drive folder)

And this is the outputs:
(I can’t upload images due to a restriction, please find this image in this drive folder)

More precisely, I always connected the RealSense camera using a USB cable to my laptop, then tried it with the Nuitrack desktop application to see if everything was fine or not. Finally, I will run this code using Python 3.9.
For env variables, I have NUITRACK_HOME which is C:\Program Files\Nuitrack\nuitrack\nuitrack, but I don’t have LD_LIBRARY_PATH. Maybe this can be the problem.
Looking forward to hearing from you soon.

Thanks,
Koorosh

gvr · March 19, 2024, 2:12pm

Hey, @KooroshRH.

but I don’t have LD_LIBRARY_PATH

Yes, this is because it is the Path actually. (I typed it wrongly). If you attach it, that will be fantastic.

Thank you for providing the info, we will take it into further testing and inform you about results as soon as possible.

gvr · April 5, 2024, 5:53am

Hey, @KooroshRH.

Apologies for the late reply. The forum was down for some time.

We used the additional info for thorough testing, but we were still unable to replicate your problem. It seems we’re still having some differences between setups. As a solution we can suggest you a remote session (via anydesk, for example) to identify the problems. Please, tell us if it suits you. If so, we’ll arrange a meeting.

KooroshRH · April 5, 2024, 9:03pm

Hi @gvr,

That’s great, and I’m available for a meeting next week.
Let me know how we can arrange the meeting in detail.

gvr · April 8, 2024, 11:39am

Hello, @KooroshRH.

Here’s the calendar (link) with the available time. Choose the one that suits you best.

We’re eagerly awaiting your response.

gvr · May 7, 2024, 7:35am

Hello, @KooroshRH.

Apologies for the such a long delay.

Is this issue still relevant to you? If so, we have prepared the debug package.
Please reproduce the issue with the above package installed and send us the log.

We’re looking forward to feedback.

KooroshRH · May 13, 2024, 1:48pm

Hi @gvr,

Thanks for your message; It seems the package is password-protected, and I don’t know it. It would be awesome if you could let me know it.

KooroshRH · May 14, 2024, 5:02pm

Hi @gvr,

I tested the new debug package and it was working perfectly. Thanks for your help to fix this problem. I think we can close this thread now, but I have some questions about the python module:

How can I change the AI or Classical mode of the model for extracting joints?
Is it possible for the model to extract joints only using the depth stream of the real-sense camera while the RGB stream is turned off because of privacy issues?
Is it possible to run the model on a .bag file recorded from a real-sense camera instead of a live connection? Because I want to do a data collection and run different models for joint extraction after the data collection is done.
Is there any documentation about the outputs of the model, for example the exact specifications of the “real” dimensions for each joint?

Thanks,
Koorosh

gvr · May 15, 2024, 9:26am

Hello, @KooroshRH.

You can set a config option “Skeletonization.Type” to “CNN_HPE” in order to use AI skeletonization.
It is possible to turn off the RGB stream output by setting config option “DepthProvider.RGB” to “Off”.
Yes! It is possible to run realsense bag files through providing config with “Realsense2Module.FileRecord: /path/to/the/file.bag”. Make sure you have your sensor connected to the PC, otherwise Nuitrack won’t run (current behaviour).
Regarding outputs, here’s the documentation for the skeleton system nuitrack-sdk/doc/Overview.md at master · 3DiVi/nuitrack-sdk · GitHub, and joint struct reference Nuitrack: nuitrack.Joint Struct Reference. Also, if you need more information about Nuitrack, you can check out this page on github.

Feel free to reach out to us if you have any other questions.