I am developing an application for Unity with 3 RealSense sensors connected to the same PC. The idea is to perform skeleton tracking with each camera for one user. The first tests were done with a single sensor and everything was fine. However, when using all three, I noticed a very significant drop in FPS.
In the profiler, I was able to see that the issue was related to the script NuitrackManager.cs. After reviewing the script’s code, I noticed that there is a function that waits/synchronizes between sensors: Nuitrack.WaitUpdate. I replaced it with Nuitrack.Update, and everything works fine.
I would like to know whether what I have done could have any implications for the stability of the application, or if it could cause any kind of error or problem.
Nuitrack.Update() - Initiate Nuitrack update. Request new data from all created Nuitrack modules. No data synchronization is performed
Nuitrack.WaitUpdate(module) - Initiate Nuitrack module update and wait for it. Similar to the Nuitrack::Update, but waits until all the required callbacks (event handlers) are called.
Therefore, you can observe a drop in FPS when using WaitUpdate. In fact, it blocks the main Unity thread until the skeleton module issues a new skeleton (in the case of NuitrackManager.cs). And if the camera is running at 30 FPS, your Unity app will also be limited to 30 FPS.
When using Update, the skeleton may not match on a specific frame, for example, with a depth or RGB frame.
Here you need to see if you really need such an exact match of the skeleton with the frames in your project, it is probably better to use Update (especially if only skeletons are needed)