How to convert NuiTrack RGB image to OpenCV format in Unity?

Hello @olga.kuzminykh

I want to use RGB image processing stuff with OpenCV. how to convert the RGB image that I’m receiving from NuiTrack to OpenCV format?

It will be helpful if anyone can provide a short template.


Hi Prasanna,

Check out this sample code:

_colorSensor = ColorSensor::create();
// Bind to event new frame
_colorSensor->connectOnNewFrame(std::bind(&NuitrackGLSample::onNewRGBFrame, this, std::placeholders::_1));
// handler
void NuitrackGLSample::onNewRGBFrame(RGBFrame::Ptr frame)
     cv::Mat rgbMap = cv::Mat(frame->getRows(), frame->getCols(), CV_8UC3,
     (void *)frame->getData()).clone(); // rgbMap is OpenCV Mat with data from RGBFrame

This code is modification of nuitrack_gl_sample.

1 Like

Thanks for the reply.

Will the same method work in Unity?

I mean in c#?

I want to convert NuiTrack depth and RGB image to OpenCV formats so that I can do some image processing.

Later I can convert back to either NuiTrack or Unity texture format for rendering.