As you can see on the picture all the joints align pretty good except of ankles.
I did number of tests with various distances, lights, rooms, the result was almost the same.
Is it known problem?
Can anybody suggest the solution?
Which sensor are you using for your testing?
Westa
Hi @Westa,
The sensor is regular Astra.
Previously I used the original Astra SDK where foots detection was working better. But the overall accuracy was not so good.
Frustratingly - with the astra sensors this is something we have seen in the past. At the distance you are running - based on the image you posted - its a challenge sometimes for the astra sensor to report reliable depth disparity between the feet and the floor. This may in part in your setup be due to the position of the light directly over the person and the person appearing to wear dark clothing and shoes in the image.
As discussed in the nuitrack setup - light color clothing tends to product better results - especially where there is good contrast between the foreground body and the background. Better front lighting can also sometimes improve the sensor accuracy around contact points.
One of the issues is that - if you examine the depth data around the feet - you will likely be seeing some depth errors and possible holes( distance reported as 0) these often appear as halos around the body. These errors make if very difficult for nuitrack to determine exactly where the feet end and the floor begins.
Westa
@Westa thanks for your suggestion, I already tried different setups with Nuitrack, but it did not help.
To find a root of the problem, the camera or the SDK, I compared Nuitrack results to Astra SDK.
I have used the same camera and same environment conditions (also same shoes and jeans), so
In regards of foots detection the result of AstraSDK looks better.
Hopefully the coming release will fix that issue
Hi Alex,
Please advise what sample do you use?
From the first picture that you’ve posted it seems that you’ve changed some parameters in nuitrack.config file that could lead to such incorrect detection of a skeleton. Please use the default settings in nuitrack.config file - skeleton joints should be detected correctly.
Hi @olga.kuzminykh,
Please advise what sample do you use?
The sample is my own, it maps the body data and the color frame data together on the screen. It also recording the body data to the file. The goal is to evaluate Nuitrack accuracy vs Astra SDK.
From the first picture that you’ve posted it seems that you’ve changed some parameters in nuitrack.config file that could lead to such incorrect detection of a skeleton. Please use the default settings in nuitrack.config file - skeleton joints should be detected correctly.
I am using the default config file. The only difference may be in x86 / x64 bitnes versions, but they have the same config files.
“Hip” joints should be the same length with default nuitrack.config file settings. However, in the first picture joints are not similar in length.
Please try to run one of our samples from Nuitrack SDK to compare the results.
I tried the nuitrack_csharp_sample.
The depth view looks good, but the color view is not.
On the color view the body looks shifted, although the window size and the frames resolutions are the same.
Maybe the difference in FOV of the cameras? I checked the GetOutputMode(), it shows the same HFOV.
You have to turn on depth-to-color registration:
- Open the file
%NUITRACK_HOME%/bin/OpenNI2/Drivers/orbbec.ini - Uncomment the line
;Registration=1in the[Depth]section (delete “;”)
However, we cannot guarantee that the result will be perfect because this depends largely on the sensor that you use.
Hello @olga.kuzminykh,
The change of Registration=1 in the [Depth] did not help.
Actually any changes in %NUITRACK_HOME%/bin/OpenNI2/Drivers/orbbec.ini don’t affect the output, I have uncommented and changed mirror, crop, other params but nothing changed in the resulting image, looks like that file is not used.
Do I need to update something in config file or other place to path to the orbbec.ini?
P.S. the NUITRACK_HOME and PATH to bin already defined in environment variables
fwiw - frustratingly we have found that orbbec sensors seem to be somewhat unreliable in terms of rbg to depth registration - depending on the exact build and frankly sometimes the day it was built it seems - your results in terms of registration may vary widely.
We have seen examples of devices running side by side that produce widely varying results.
I would start by testing the sensor with the viewer provided by Orbbec to compare the depth map and rgb alignment.
If the output frames do not align with Registration=1 turned on in the path below the viewer sample - then you should maybe talk to orbbec about if your sensor needs to be factory re-aligned.
Westa
Please try to turn on depth-to-color registration using nuitrack.config file: find the section “DepthProvider” and set “Depth2ColorRegistration” to “true”.
I set “Depth2ColorRegistration” to “true”, but still no effect.
And orbbec.ini changes don’t affect the output.
Just to remind I am using regular Astra’s, when config refers to AstraPro and Persee, could it be the reason?
Here is the results from two different Astra’s, same body and same application. Second camera body looks shifted left. Can it be color frame cropping problem?
From our experience, some Orbbec Astra cameras are non-calibrated. You can find information about this on Orbbec Community. This can cause different results when using different Orbbec cameras with Nuitrack.
Thanks again for your response, I have already contacted to Orbbec in separate thread.
Something still not clear for me. As it looks Nuitrack has an ability to improve the deth2Color registration independently from the device. Why its not works in my case?
Until I am waiting for your response two things happened
- Orbbec confirmed the cameras were calibrated.
- I have compared Astra SDK vs Nuitrack SDK registration performance. Based on the result I can confidently assert there is no calibration problem with the devices, the problem is with Nuitrack.
See results from two different Astras below.
Please help to make Depth2ColorRegistration working or advice what else am I doing wrong?
What should be “CameraID” in “AstraProPerseeDepthProvider” for regular Astra? Currently it equals 1.
Camera 1 ---------- Astra SDK
Camera 1 ------------- Nuitrack
====================================================================
Camera 2 ------------ Astra SDK
Camera 2 -------------- Nuitrack
EDIT: here is the console output I am getting every time the application runs:
ERROR: AstraProPerseeDepthProvider can’t create RGB Stream (Can’t open VideoCapture device)
DepthProviderManager: AstraProPerseeDepthProvider can’t create RGB Stream (Can’t open VideoCapture device)
Can’t find video camera!
Can’t find Depth camera!
Create DepthProvider: OpenNI2DepthProvider
What version of Nuitrack do you use? According to the console output, it seems that you’re using the version not higher than v0.23.1. The messages that you’ve mentioned are not displayed in the console output with the latest version of Nuitrack (v0.24.0). These messages as well as changing the values of properties in AstraProPerseeDepthProvider section of nuitrack.config doesn’t not affect the regular Orbbec Astra.
Please note that this option:
was introduced in v0.23.3 of Nuitrack. If your version is lower than 0.23.3, it won’t work.
We’ve just checked depth-to-color registration with two Orbbec Astra cameras. Registration worked correctly for both sensors.
You can also check the registration using nuitrack_sample.exe from Nuitrack Runtime (bin folder) (starting from v0.23.3). In this sample, you can switch between RGB and Depth modes.
Thanks, version upgrade worked for me.