Segfault in SkeletonTracker::create() when using PCL visualization

Brett · February 14, 2019, 9:43am

As mentioned in the title, I get a segfault in the SkeletonTracker::create() method when using the PCL library for visualization. I’m using version 0.25.0 installed via nuitrack-ubuntu-amd64.deb. The following code (src/main.cpp) demonstrates the issue.

#include <iostream>
#include <thread>
#include <nuitrack/Nuitrack.h>
#include <pcl/visualization/pcl_visualizer.h>

void on_skeleton_update(tdv::nuitrack::SkeletonData::Ptr user_skeletons) { std::cout << "Skeleton update\n"; }
void on_user_update(tdv::nuitrack::UserFrame::Ptr frame) { std::cout << "User update\n"; }
void on_new_depth_frame(tdv::nuitrack::DepthFrame::Ptr frame) { std::cout << "New depth frame\n"; }
void on_new_RGB_frame(tdv::nuitrack::RGBFrame::Ptr frame) { std::cout << "New RGB frame\n"; }

int main(int argc, char *argv[])
{
	using namespace std::literals;
	using namespace tdv::nuitrack;

	std::cout << "Initialising PCL components\n";
	pcl::visualization::PCLVisualizer viewer{"3D Viewer"};

	std::cout << "Initialising Nuitrack components\n";
	Nuitrack::init();
	std::cout << "Creating depth sensor\n";
	auto depth_sensor = DepthSensor::create();
	depth_sensor->connectOnNewFrame([&](DepthFrame::Ptr frame) { on_new_depth_frame(frame); });
	std::cout << "Creating colour sensor\n";
	auto colour_sensor = ColorSensor::create();
	colour_sensor->connectOnNewFrame([&](RGBFrame::Ptr frame) { on_new_RGB_frame(frame); });
	auto mode = colour_sensor->getOutputMode();
	std::cout << "Mode: " << mode.xres << "x" << mode.yres << "\n";
	std::cout << "Creating user tracker\n";
	auto user_tracker = UserTracker::create();
	user_tracker->connectOnUpdate([&](UserFrame::Ptr frame) { on_user_update(frame); });
	std::cout << "Creating skeleton tracker\n";
	auto skeleton_tracker = SkeletonTracker::create(); // <--- Segfault occurs here!
	// skeleton_tracker->setNumActiveUsers(1); // Only track a single skeleton
	skeleton_tracker->connectOnUpdate([&](SkeletonData::Ptr user_skeletons) { on_skeleton_update(user_skeletons); });
	std::cout << "Running Nuitrack\n";
	Nuitrack::run();

	std::this_thread::sleep_for(5s);

	std::cout << "Shutting down.\n";
	Nuitrack::release();
	return 0;
}

Without the line pcl::visualization::PCLVisualizer viewer{"3D Viewer"};, it runs as expected.

Here is the CMakeLists.txt file:

cmake_minimum_required(VERSION 3.9)

set(CMAKE_CXX_STANDARD 17)
set(CMAKE_CXX_STANDARD_REQUIRED ON)
set(CMAKE_CXX_EXTENSIONS OFF)

set(PROJECT_NAME delme)
project(${PROJECT_NAME})
find_package(Boost COMPONENTS system REQUIRED)
find_package(Threads REQUIRED)
find_package(PCL REQUIRED COMPONENTS common visualization)

add_definitions(-fPIC -Wall -Wextra -Werror -pedantic -Wno-unused-parameter ${PCL_DEFINITIONS})

include_directories(${PCL_INCLUDE_DIRS})

set(NUITRACK_LIBRARY_PATH /usr/local/lib/nuitrack)
set(LIBS stdc++fs nuitrack ${Boost_SYSTEM_LIBRARY} Threads::Threads ${PCL_LIBRARIES} usb-1.0)
link_directories(${NUITRACK_LIBRARY_PATH} ${PCL_LIBRARY_DIRS})

set(SOURCES
    src/main.cpp)

add_executable(${PROJECT_NAME} ${SOURCES})
target_link_libraries(${PROJECT_NAME} ${LIBS})

And some output from gdb:

$ gdb --ex=r ./delme
GNU gdb (Ubuntu 8.2-0ubuntu1) 8.2
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./delme...done.
Starting program: /home/brett/tmp/build/delme 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Initialising PCL components
Initialising Nuitrack components
Creating depth sensor
[New Thread 0x7fffe3dc8700 (LWP 802)]
UVCVideoCapture::open ERROR: can't find device
[Thread 0x7fffe3dc8700 (LWP 802) exited]
[New Thread 0x7fffe3dc8700 (LWP 803)]
[Thread 0x7fffe3dc8700 (LWP 803) exited]
[New Thread 0x7fffe3dc8700 (LWP 804)]
[New Thread 0x7fffe2ea5700 (LWP 805)]
[New Thread 0x7fffe2444700 (LWP 806)]
[New Thread 0x7fffe15a7700 (LWP 807)]
[New Thread 0x7fffe0da6700 (LWP 809)]
[New Thread 0x7fffdbfff700 (LWP 810)]
[Thread 0x7fffdbfff700 (LWP 810) exited]
[New Thread 0x7fffdbfff700 (LWP 811)]
[New Thread 0x7fffdb7fe700 (LWP 812)]
[New Thread 0x7fffdaffd700 (LWP 813)]
[New Thread 0x7fffda7fc700 (LWP 814)]
[Thread 0x7fffdaffd700 (LWP 813) exited]
[Thread 0x7fffdb7fe700 (LWP 812) exited]
[Thread 0x7fffda7fc700 (LWP 814) exited]
[New Thread 0x7fffda7fc700 (LWP 815)]
[New Thread 0x7fffdb7fe700 (LWP 816)]
[New Thread 0x7fffdaffd700 (LWP 817)]
[Thread 0x7fffdb7fe700 (LWP 816) exited]
[Thread 0x7fffda7fc700 (LWP 815) exited]
[Thread 0x7fffdaffd700 (LWP 817) exited]
[Thread 0x7fffdbfff700 (LWP 811) exited]
[New Thread 0x7fffdbfff700 (LWP 818)]
[New Thread 0x7fffdaffd700 (LWP 819)]
[New Thread 0x7fffda7fc700 (LWP 820)]
Create DepthProvider: Realsense2DepthProvider
Creating colour sensor
Mode: 1280x720
Creating user tracker
[New Thread 0x7fffdb7fe700 (LWP 821)]
[New Thread 0x7fffd9ffb700 (LWP 822)]
[New Thread 0x7fffd97fa700 (LWP 823)]
[New Thread 0x7fffd8ff9700 (LWP 824)]
[New Thread 0x7fffcffff700 (LWP 825)]
Creating skeleton tracker

Thread 1 "delme" received signal SIGSEGV, Segmentation fault.
0x00007fffe84c26bd in ?? () from /usr/local/etc/nuitrack/middleware/libNuitrackModule.so
(gdb) bt
#0  0x00007fffe84c26bd in  () at /usr/local/etc/nuitrack/middleware/libNuitrackModule.so
#1  0x00007fffe80602d6 in  () at /usr/local/etc/nuitrack/middleware/libNuitrackModule.so
#2  0x00007fffe805b6c2 in  () at /usr/local/etc/nuitrack/middleware/libNuitrackModule.so
#3  0x00007fffe805ebb0 in  () at /usr/local/etc/nuitrack/middleware/libNuitrackModule.so
#4  0x00007fffe805b6c2 in  () at /usr/local/etc/nuitrack/middleware/libNuitrackModule.so
#5  0x00007fffe8080057 in  () at /usr/local/etc/nuitrack/middleware/libNuitrackModule.so
#6  0x00007fffe8068edd in  () at /usr/local/etc/nuitrack/middleware/libNuitrackModule.so
#7  0x00007ffff7d66406 in std::shared_ptr<tdv::nuitrack::middleware::Skeletonization> tdv::nuitrack::middleware::Core::createModule<tdv::nuitrack::middleware::Skeletonization>() ()
    at /usr/local/lib/nuitrack/libnuitrack.so
#8  0x00007ffff7d66a95 in std::shared_ptr<tdv::nuitrack::middleware::Skeletonization> tdv::nuitrack::middleware::Core::getModule<tdv::nuitrack::middleware::Skeletonization>(unsigned long) ()
    at /usr/local/lib/nuitrack/libnuitrack.so
#9  0x00007ffff7d63e01 in  () at /usr/local/lib/nuitrack/libnuitrack.so
#10 0x00007ffff7d6bd7d in nuitrack_CreateSkeletonTracker () at /usr/local/lib/nuitrack/libnuitrack.so
#11 0x0000555555564c27 in tdv::nuitrack::SkeletonTracker::create() () at /usr/local/include/nuitrack/modules/SkeletonTracker.h:83
#12 0x0000555555560ce0 in main(int, char**) (argc=1, argv=0x7fffffffd898) at /home/brett/tmp/src/main.cpp:33
(gdb) select-frame 12
(gdb) info locals
viewer = {_vptr.PCLVisualizer = 0x7ffff7cbfae0 <vtable for pcl::visualization::PCLVisualizer+16>, interactor_ = {<vtkSmartPointerBase> = {Object = 0x5555559d1f40}, <No data fields>}, 
  update_fps_ = {<vtkSmartPointerBase> = {Object = 0x555555619570}, <No data fields>}, stopped_ = false, timer_id_ = 2, exit_main_loop_timer_callback_ = {<vtkSmartPointerBase> = {
      Object = 0x555555db08d0}, <No data fields>}, exit_callback_ = {<vtkSmartPointerBase> = {Object = 0x555555db0940}, <No data fields>}, rens_ = {<vtkSmartPointerBase> = {
      Object = 0x5555556195b0}, <No data fields>}, win_ = {<vtkSmartPointerBase> = {Object = 0x555555619610}, <No data fields>}, style_ = {<vtkSmartPointerBase> = {
      Object = 0x55555561a460}, <No data fields>}, cloud_actor_map_ = {px = 0x5555556246a0, pn = {pi_ = 0x5555556246e0}}, shape_actor_map_ = {px = 0x555555624700, pn = {
      pi_ = 0x555555624740}}, coordinate_actor_map_ = {px = 0x555555624760, pn = {pi_ = 0x5555556247a0}}, axes_widget_ = {<vtkSmartPointerBase> = {Object = 0x0}, <No data fields>}, 
  camera_set_ = false, camera_file_loaded_ = false, use_vbos_ = false}
depth_sensor = std::shared_ptr<tdv::nuitrack::DepthSensor> (use count 1, weak count 0) = {get() = 0x555555f0bb60}
colour_sensor = std::shared_ptr<tdv::nuitrack::ColorSensor> (use count 1, weak count 0) = {get() = 0x555555f0bbb0}
mode = {fps = 30, xres = 1280, yres = 720, hfov = 1.20948672}
user_tracker = std::shared_ptr<tdv::nuitrack::UserTracker> (use count 1, weak count 0) = {get() = 0x555555ee05d0}
skeleton_tracker = std::shared_ptr<tdv::nuitrack::SkeletonTracker> (use count 1610612821, weak count 1431658915) = {get() = 0x0}
(gdb) select-frame 11
(gdb) info locals
pimpl = 0x555555f0a230
errorCode = tdv::nuitrack::OK

olga.kuzminykh · February 19, 2019, 7:13am

Hi Brett,

We were able to reproduce this issue. It seems that there is a conflict when resolving the symbols of the tinyxml library when using PCL. As a temporary solution, you can try the following:

Add the directive #include <tinyxml.h> to your code.
Add auto dummyXmlDoc = std::make_shared<TiXmlDocument>(); after the using namespace directives.
Build your project.

We’ll investigate this issue in more details and fix it in the next Nuitrack release.

Thank you for reporting this bug!

Brett · February 19, 2019, 10:29am

Hi Olga,

Thanks for your reply! Unfortunately, the temporary solution you suggested didn’t prevent the segfault from occurring at the same point on my local machine. Let me know if I can provide any further information that might be helpful.

Best regards,
Brett.