How to convert NuiTrack RGB image to OpenCV format in Unity?

[prasanna.routray]

Hello @olga.kuzminykh

I want to use RGB image processing stuff with OpenCV. how to convert the RGB image that I’m receiving from NuiTrack to OpenCV format?

It will be helpful if anyone can provide a short template.

Prasanna

[olga.kuzminykh]

Hi Prasanna,

Check out this sample code:

_colorSensor = ColorSensor::create();
// Bind to event new frame
_colorSensor->connectOnNewFrame(std::bind(&NuitrackGLSample::onNewRGBFrame, this, std::placeholders::_1));
...
// handler
void NuitrackGLSample::onNewRGBFrame(RGBFrame::Ptr frame)
{
     cv::Mat rgbMap = cv::Mat(frame->getRows(), frame->getCols(), CV_8UC3,
     (void *)frame->getData()).clone(); // rgbMap is OpenCV Mat with data from RGBFrame
}

This code is modification of nuitrack_gl_sample.

[prasanna.routray]

Thanks for the reply.

Will the same method work in Unity?

I mean in c#?

I want to convert NuiTrack depth and RGB image to OpenCV formats so that I can do some image processing.

Later I can convert back to either NuiTrack or Unity texture format for rendering.

[minux]

Dear Prasanna, I have found your request. Were you able to accomplish anything? I would also like to do RGB image processing stuff with OpenCV for Unity. Can you give me some advice? Thank you very much, Michele

[Stepan.Reuk]

Hello @minux
I don’t think anyone will answer. I find it difficult to answer specifically about the OpenCV format, but with the Unity SDK, you can get a frame in the format of various Unity textures. You can also get a raw array with the data of these frames.

https://github.com/3DiVi/nuitrack-sdk/blob/master/Unity3D/doc/Development.md

[Stepan.Reuk]

@minux do you have any more questions about this issue?