How to access image buffer with OpenCV

Nuitrack SDK
1

Looking at “NuitrackGLSample.cpp”, I see how to get RGB and Depth pixels by walking through the Nuitrack data structures.
I’d like to work directly with OpenCV, and it looks like you use OpenCV internally?

Is it possible to directly cast or convert the frame buffers to OpenCV?

2

Hi Dave,

There is no way to cast the frame buffers directly to OpenCV, however, you can use the following solution:

void onNewDepthFrame(DepthFrame::Ptr frame)
{
	cv::Mat depthFrame = cv::Mat(frame->getRows(), frame->getCols(), CV_16UC1, frame->getData()).clone();
	...
}
1 Like
3

Thanks! this is very helpful!
Does NuiTrack link with OpenCV? If so, what version (I seem to be having some sort of version conflict…)

4

Actually, I figured out that Nuitrack uses OpenCV 2.4: /usr/local/lib/nuitrack/libopencv_core.so.2.4.8.

thanks!

1 Like
5

FYI, I ended up not using OpenCV, as I needed to use OpenCV 3.0 for ROS Kinetic transport. Instead, I created published the Image Message directly using the sample from Nuitrack_gl_sample. I’ll post a new forum message for anyone interested in ROS.

6

Good Evening, I am trying to convert the frame to an opencv matrix using your suggestion. However, I am running into the following error during compilation:

error: no matching function for call to ‘cv::Mat::Mat(int, int, int, const DataType*)’
depthFrame = cv::Mat(frame->getRows(), frame->getCols(), CV_16UC1, frame->getData()).clone();

I am doing this in the function onNewDepthFrame(DepthFrame::Ptr frame) in one of the nuitrack examples. I have already modified the included cmake to use opencv and fixed a versioning issue where I was trying to use opencv 2.4.13 while nuitrack was expecting 2.4.8. I am successfully declaring cv::Mat depthFrame; on the previous line without any errors when the line in question is commented out.

Any help or suggestions you can provide would be greatly appreciated.

Thank you

7

I ran into a lot of issues with OpenCV versioning, and finally just used a brute-force approach and copied the bits from Nuitrack to a ROS point cloud message.

On the upside, I am able to publish not just a depth cloud, but a colorized depth cloud.

I welcome any idea/suggestions!!!

8

Hi Joey,

Try to cast a pointer to void*:

depthFrame = cv::Mat(frame->getRows(), frame->getCols(), CV_16UC1, (void*)frame->getData()).clone();