Context
I work for a choreographer as a consultant and we are tracking a dancer on a typical indoor scene with an Intel Realsense D435 camera and use Nuitrack AI for skeletal detection. Yet, we have acceptable performance in terms of time (14 to 20 skeleton FPS) and of precision, sometimes getting tracking issues. We are testing different light and camera settings to see if we can improve the tracking. However, the range is not great, and we often lose track of the dancer’s members when she turns around or make specific moves. Let’s call it a “work in progress”.
Next things to try
- We would like our system to work in darkness, as the show is about darkness… However, I know Nuitrack AI works with the RGB images. I read here that we can try to turn off the RGB sensor, as we will do next. Also, we can light the scene with as much infrared light as needed, no problem. Do you have any advice for us?
- We would like to attempt improving the range, stability, reliability and precision of the tracking by adding multiple cameras, each linked to its own Nuitrack process, and all on the same computer, by implementing a system where every process sends its skeleton data to a central process which averages (with a confidence based weighting), time smoothes (within a certain time window), and rejects outliers (with too low confidence). We are thinking of buying the new Intel Realsense D455 and/or Intel LiDAR L515, in some quantity (1, 2, 3 or more…?). I also need to think about calibrating the camera positions in the scene to get coherent coordinate and orientation data. Do you have any advice or already working systems to share with with me?
Any other ideas?
Before we get into buying all this stuff and engaging in many hours of development, we’d love to have some advice about if it is doable. Thanks for sharing your thoughts! I can also give some feedback about our experience as it follows to anyone interested.