Sensor Not Recognized In Unity App

Hi,

I am trying to deploy the simplest unity application onto android. After deploying I cannot see anything from the sensor, and the log says it didn’t recognize the sensor. I checked every setting from other user posted issues on this forum but was unable to get the sensor to work.

The application currently only contains a scene, Nuitrack Manager and the most basic RGB camera lookout in the scene.

  • Target Device: Rockchip RK3588S
  • Architecture: ARM64
  • Android Version: 13
  • Minimum Set Android API: Android 9
  • Unitrack SDK: Nuitrack_arm64.apk
  • Nuitrack version: Nuitrack v0.37.24
  • Unity version: 2022.3.38f1
  • Connected Sensor: Orbbec Astra Plus (AI License)

I also activated the sensor from the Nuitrack APK and can see both the RGB feed and the depth feed. I tried to run the unity app with the Auto Activate License field and without it, without any changes in the app.

When I run the application the only thing I can see is the unity background (the 3d scene) and overlaying it is a white screen without output.

Below is the stack trace from the android app.

0001/01/01 00:00:00.000 -1 -1 Info  --------- beginning of main
2024/10/01 08:55:25.738 3415 3415 Info Zygote seccomp disabled by setenforce 0
2024/10/01 08:55:25.749 3415 3415 Debug CompatibilityChangeReporter Compat change id reported: 171979766; UID 10094; state: ENABLED
2024/10/01 08:55:25.755 3415 3415 Verbose GraphicsEnvironment ANGLE Developer option for 'com.unity.template.urpsample' set to: 'default'
2024/10/01 08:55:25.756 3415 3415 Verbose GraphicsEnvironment ANGLE GameManagerService for com.unity.template.urpsample: false
2024/10/01 08:55:25.756 3415 3415 Verbose GraphicsEnvironment Neither updatable production driver nor prerelease driver is supported.
2024/10/01 08:55:25.756 3415 3415 Debug NetworkSecurityConfig No Network Security Config specified, using platform default
2024/10/01 08:55:25.756 3415 3415 Debug NetworkSecurityConfig No Network Security Config specified, using platform default
2024/10/01 08:55:25.800 3415 3415 Info IL2CPP JNI_OnLoad
2024/10/01 08:55:25.809 3415 3415 Debug CompatibilityChangeReporter Compat change id reported: 237531167; UID 10094; state: DISABLED
2024/10/01 08:55:25.825 3415 3415 Error SurfaceSyncer Failed to find sync for id=0
2024/10/01 08:55:25.863 3415 3433 Info Unity MemoryManager: Using 'Dynamic Heap' Allocator.
2024/10/01 08:55:25.872 3415 3433 Info Unity SystemInfo CPU = ARM64 FP ASIMD AES, Cores = 8, Memory = 7665mb
2024/10/01 08:55:25.872 3415 3433 Info Unity SystemInfo ARM big.LITTLE configuration: 4 big (mask: 0xf0), 4 little (mask: 0xf)
2024/10/01 08:55:25.872 3415 3433 Info Unity ApplicationInfo com.unity.template.urpsample version 0.1
2024/10/01 08:55:25.872 3415 3433 Info Unity Built from '2022.3/staging' branch, Version '2022.3.38f1 (c5d5a7410213)', Build type 'Release', Scripting Backend 'il2cpp', CPU 'arm64-v8a', Stripping 'Enabled'
2024/10/01 08:55:26.222 3415 3433 Info Unity Company Name: Specularo
2024/10/01 08:55:26.222 3415 3433 Info Unity Product Name: VTO
2024/10/01 08:55:26.220 3415 3415 Info UnityMain type=1400 audit(0.0:1933): avc: denied { open } for path="/dev/__properties__/u:object_r:vendor_default_prop:s0" dev="tmpfs" ino=305 scontext=u:r:untrusted_app:s0:c94,c256,c512,c768 tcontext=u:object_r:vendor_default_prop:s0 tclass=file permissive=1 app=com.unity.template.urpsample
2024/10/01 08:55:26.224 3415 3433 Info mali_so [File] : hardware/arm/maliT760/driver/product/base/src/mali_base_kbase.c; [Line] : 1392; [Func] : base_context_deal_with_version_affairs_rk_ext;
2024/10/01 08:55:26.224 3415 3433 Info mali_so arm_release_ver of this mali_so is 'g18p0-01eac0', rk_so_ver is '2 '.
2024/10/01 08:55:26.224 3415 3433 Info mali_so [File] : hardware/arm/maliT760/driver/product/base/src/mali_base_kbase.c; [Line] : 1408; [Func] : base_context_deal_with_version_affairs_rk_ext;
2024/10/01 08:55:26.224 3415 3433 Info mali_so arm_release_vers are match. to set the full mali_ver 'g18p0-01eac0-2' as value of 'sys.gmali.version'.
2024/10/01 08:55:26.244 3415 3433 Debug Unity  GL_EXT_debug_marker GL_ARM_rgba8 GL_ARM_mali_shader_binary GL_OES_depth24 GL_OES_depth_texture GL_OES_depth_texture_cube_map GL_OES_packed_depth_stencil GL_OES_rgb8_rgba8 GL_EXT_read_format_bgra GL_OES_compressed_paletted_texture GL_OES_compressed_ETC1_RGB8_texture GL_OES_standard_derivatives GL_OES_EGL_image GL_OES_EGL_image_external GL_OES_EGL_image_external_essl3 GL_OES_EGL_sync GL_OES_texture_npot GL_OES_vertex_half_float GL_OES_required_internalformat GL_OES_vertex_array_object GL_OES_mapbuffer GL_EXT_texture_format_BGRA8888 GL_EXT_texture_rg GL_EXT_texture_type_2_10_10_10_REV GL_OES_fbo_render_mipmap GL_OES_element_index_uint GL_EXT_shadow_samplers GL_OES_texture_compression_astc GL_KHR_texture_compression_astc_ldr GL_KHR_texture_compression_astc_hdr GL_KHR_texture_compression_astc_sliced_3d GL_EXT_texture_compression_astc_decode_mode GL_EXT_texture_compression_astc_decode_mode_rgb9e5 GL_KHR_debug GL_EXT_occlusion_query_boolean GL_EXT_disjoint_timer_query GL_EXT_blend_minmax GL_EXT_discard_framebuffer
2024/10/01 08:55:26.244 3415 3433 Debug Unity  GL_OES_get_program_binary GL_OES_texture_3D GL_EXT_texture_storage GL_EXT_multisampled_render_to_texture GL_EXT_multisampled_render_to_texture2 GL_OES_surfaceless_context GL_OES_texture_stencil8 GL_EXT_shader_pixel_local_storage GL_ARM_shader_framebuffer_fetch GL_ARM_shader_framebuffer_fetch_depth_stencil GL_ARM_mali_program_binary GL_EXT_sRGB GL_EXT_sRGB_write_control GL_EXT_texture_sRGB_decode GL_EXT_texture_sRGB_R8 GL_EXT_texture_sRGB_RG8 GL_KHR_blend_equation_advanced GL_KHR_blend_equation_advanced_coherent GL_OES_texture_storage_multisample_2d_array GL_OES_shader_image_atomic GL_EXT_robustness GL_EXT_draw_buffers_indexed GL_OES_draw_buffers_indexed GL_EXT_texture_border_clamp GL_OES_texture_border_clamp GL_EXT_texture_cube_map_array GL_OES_texture_cube_map_array GL_OES_sample_variables GL_OES_sample_shading GL_OES_shader_multisample_interpolation GL_EXT_shader_io_blocks GL_OES_shader_io_blocks GL_EXT_tessellation_shader GL_OES_tessellation_shader GL_EXT_primitive_bounding_box GL_OES_primitive_bounding_
2024/10/01 08:55:26.244 3415 3433 Debug Unity box GL_EXT_geometry_shader GL_OES_geometry_shader GL_ANDROID_extension_pack_es31a GL_EXT_gpu_shader5 GL_OES_gpu_shader5 GL_EXT_texture_buffer GL_OES_texture_buffer GL_EXT_copy_image GL_OES_copy_image GL_EXT_shader_non_constant_global_initializers GL_EXT_color_buffer_half_float GL_EXT_unpack_subimage GL_EXT_color_buffer_float GL_EXT_float_blend GL_EXT_YUV_target GL_OVR_multiview GL_OVR_multiview2 GL_OVR_multiview_multisampled_render_to_texture GL_KHR_robustness GL_KHR_robust_buffer_access_behavior GL_EXT_draw_elements_base_vertex GL_OES_draw_elements_base_vertex GL_EXT_protected_textures GL_EXT_buffer_storage GL_EXT_external_buffer GL_EXT_EGL_image_array GL_EXT_clear_texture GL_EXT_texture_filter_anisotropic GL_OES_texture_float_linear GL_ARM_texture_unnormalized_coordinates GL_EXT_shader_framebuffer_fetch GL_EXT_clip_control GL_EXT_polygon_offset_clamp
2024/10/01 08:55:26.249 3415 3476 Debug Choreographer Attaching thread to JVM for AChoreographer
2024/10/01 08:55:26.267 3415 3479 Warn AudioTrack Use of stream types is deprecated for operations other than volume control
2024/10/01 08:55:26.267 3415 3479 Warn AudioTrack See the documentation of AudioTrack() for what to use instead with android.media.AudioAttributes to qualify your playback use case
2024/10/01 08:55:26.427 3415 3433 Info IL2CPP Locale en-US
2024/10/01 08:55:26.620 3415 3433 Verbose MediaRouter Audio routes updated: AudioRoutesInfo{ type=HDMI }, a2dp=false
2024/10/01 08:55:26.621 3415 3433 Verbose MediaRouter Selecting route: RouteInfo{ name=HDMI, description=null, status=null, category=RouteCategory{ name=System types=ROUTE_TYPE_LIVE_AUDIO ROUTE_TYPE_LIVE_VIDEO  groupable=false }, supportedTypes=ROUTE_TYPE_LIVE_AUDIO ROUTE_TYPE_LIVE_VIDEO , presentationDisplay=null }
2024/10/01 08:55:26.621 3415 3433 Info MediaRouter Skip setBluetoothA2dpOn(): types=1, isPlaybackActive()=false, BT route=null
2024/10/01 08:55:28.839 3415 3433 Info Unity CurrentUserTracker mode was disabled for UIAvatar(Clone)
2024/10/01 08:55:28.839 3415 3433 Info Unity NuitrackSDK.TrackedUser:set_UserID(Int32)
2024/10/01 08:55:28.839 3415 3433 Info Unity NuitrackSDK.Avatar.SkeletonsUI:Start()
2024/10/01 08:55:28.839 3415 3433 Info Unity 
2024/10/01 08:55:28.840 3415 3433 Info Unity CurrentUserTracker mode was disabled for UIAvatar(Clone)
2024/10/01 08:55:28.840 3415 3433 Info Unity NuitrackSDK.TrackedUser:set_UserID(Int32)
2024/10/01 08:55:28.840 3415 3433 Info Unity NuitrackSDK.Avatar.SkeletonsUI:Start()
2024/10/01 08:55:28.840 3415 3433 Info Unity 
2024/10/01 08:55:28.842 3415 3433 Error Unity ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.
2024/10/01 08:55:28.842 3415 3433 Error Unity Parameter name: index
2024/10/01 08:55:28.842 3415 3433 Error Unity   at System.Collections.Generic.List`1[T].get_Item (System.Int32 index) [0x00000] in <00000000000000000000000000000000>:0 
2024/10/01 08:55:28.842 3415 3433 Error Unity   at NuitrackSDK.Avatar.SkeletonsUI.Start () [0x00000] in <00000000000000000000000000000000>:0 
2024/10/01 08:55:28.842 3415 3433 Error Unity 
2024/10/01 08:55:28.853 3415 3433 Error Unity ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.
2024/10/01 08:55:28.853 3415 3433 Error Unity Parameter name: index
2024/10/01 08:55:28.853 3415 3433 Error Unity   at System.Collections.Generic.List`1[T].get_Item (System.Int32 index) [0x00000] in <00000000000000000000000000000000>:0 
2024/10/01 08:55:28.853 3415 3433 Error Unity   at NuitrackSDK.Avatar.SkeletonsUI.Update () [0x00000] in <00000000000000000000000000000000>:0 
2024/10/01 08:55:28.853 3415 3433 Error Unity 
2024/10/01 08:55:28.853 3415 3433 Info Unity InitNuitrackLibraries() starts.
2024/10/01 08:55:28.853 3415 3433 Info Unity NuitrackSDK.Loader.NuitrackLoader:InitNuitrackLibraries()
2024/10/01 08:55:28.853 3415 3433 Info Unity NuitrackManager:Init()
2024/10/01 08:55:28.853 3415 3433 Info Unity <AndroidInit>d__114:MoveNext()
2024/10/01 08:55:28.853 3415 3433 Info Unity UnityEngine.SetupCoroutine:InvokeMoveNext(IEnumerator, IntPtr)
2024/10/01 08:55:28.853 3415 3433 Info Unity 
2024/10/01 08:55:28.856 3415 3433 Info Unity Nuitrack callback: onInitFailure
2024/10/01 08:55:28.856 3415 3433 Info Unity NuitrackSDK.Loader.NuitrackCallback:onInitFailure(Int32)
2024/10/01 08:55:28.856 3415 3433 Info Unity System.Reflection.RuntimeMethodInfo:Invoke(Object, BindingFlags, Binder, Object[], CultureInfo)
2024/10/01 08:55:28.856 3415 3433 Info Unity UnityEngine.AndroidJavaProxy:Invoke(String, Object[])
2024/10/01 08:55:28.856 3415 3433 Info Unity UnityEngine.AndroidJavaProxy:Invoke(String, IntPtr)
2024/10/01 08:55:28.856 3415 3433 Info Unity UnityEngine._AndroidJNIHelper:InvokeJavaProxyMethod(AndroidJavaProxy, IntPtr, IntPtr)
2024/10/01 08:55:28.856 3415 3433 Info Unity 
2024/10/01 08:55:28.952 3415 3433 Warn Unity Additional Lights Cookie Format (GrayscaleHigh) is not supported by the platform. Falling back to 32-bit format (RGBA8 UNorm)
2024/10/01 08:55:28.952 3415 3433 Warn Unity UnityEngine.Rendering.Universal.UniversalRenderPipelineAsset:get_additionalLightsCookieFormat()
2024/10/01 08:55:28.952 3415 3433 Warn Unity UnityEngine.Rendering.Universal.UniversalRenderer:.ctor(UniversalRendererData)
2024/10/01 08:55:28.952 3415 3433 Warn Unity UnityEngine.Rendering.Universal.UniversalRendererData:Create()
2024/10/01 08:55:28.952 3415 3433 Warn Unity UnityEngine.Rendering.Universal.UniversalRenderPipelineAsset:CreateRenderers()
2024/10/01 08:55:28.952 3415 3433 Warn Unity UnityEngine.Rendering.Universal.UniversalRenderPipelineAsset:CreatePipeline()
2024/10/01 08:55:28.952 3415 3433 Warn Unity UnityEngine.Rendering.RenderPipelineAsset:InternalCreatePipeline()
2024/10/01 08:55:28.952 3415 3433 Warn Unity UnityEngine.Rendering.RenderPipelineManager:PrepareRenderPipeline(RenderPipelineAsset)
2024/10/01 08:55:28.952 3415 3433 Warn Unity UnityEngine.Rendering.RenderPipelineManager:DoRenderLoop_Internal(RenderPipelineAsset, IntPtr, Object)
2024/10/01 08:55:28.952 3415 3433 Warn Unity 
2024/10/01 08:55:28.959 3415 3433 Warn Unity Additional Lights Cookie Format (GrayscaleHigh) is not supported by the platform. Falling back to 32-bit format (RGBA8 UNorm)
2024/10/01 08:55:28.959 3415 3433 Warn Unity UnityEngine.Rendering.Universal.UniversalRenderPipelineAsset:get_additionalLightsCookieFormat()
2024/10/01 08:55:28.959 3415 3433 Warn Unity UnityEngine.Rendering.Universal.UniversalRenderer:.ctor(UniversalRendererData)
2024/10/01 08:55:28.959 3415 3433 Warn Unity UnityEngine.Rendering.Universal.UniversalRendererData:Create()
2024/10/01 08:55:28.959 3415 3433 Warn Unity UnityEngine.Rendering.Universal.UniversalRenderPipelineAsset:CreateRenderers()
2024/10/01 08:55:28.959 3415 3433 Warn Unity UnityEngine.Rendering.Universal.UniversalRenderPipelineAsset:CreatePipeline()
2024/10/01 08:55:28.959 3415 3433 Warn Unity UnityEngine.Rendering.RenderPipelineAsset:InternalCreatePipeline()
2024/10/01 08:55:28.959 3415 3433 Warn Unity UnityEngine.Rendering.RenderPipelineManager:PrepareRenderPipeline(RenderPipelineAsset)
2024/10/01 08:55:28.959 3415 3433 Warn Unity UnityEngine.Rendering.RenderPipelineManager:DoRenderLoop_Internal(RenderPipelineAsset, IntPtr, Object)
2024/10/01 08:55:28.959 3415 3433 Warn Unity 
2024/10/01 08:55:28.961 3415 3433 Warn Unity Additional Lights Cookie Format (GrayscaleHigh) is not supported by the platform. Falling back to 32-bit format (RGBA8 UNorm)
2024/10/01 08:55:28.961 3415 3433 Warn Unity UnityEngine.Rendering.Universal.UniversalRenderPipelineAsset:get_additionalLightsCookieFormat()
2024/10/01 08:55:28.961 3415 3433 Warn Unity UnityEngine.Rendering.Universal.UniversalRenderer:.ctor(UniversalRendererData)
2024/10/01 08:55:28.961 3415 3433 Warn Unity UnityEngine.Rendering.Universal.UniversalRendererData:Create()
2024/10/01 08:55:28.961 3415 3433 Warn Unity UnityEngine.Rendering.Universal.UniversalRenderPipelineAsset:CreateRenderers()
2024/10/01 08:55:28.961 3415 3433 Warn Unity UnityEngine.Rendering.Universal.UniversalRenderPipelineAsset:CreatePipeline()
2024/10/01 08:55:28.961 3415 3433 Warn Unity UnityEngine.Rendering.RenderPipelineAsset:InternalCreatePipeline()
2024/10/01 08:55:28.961 3415 3433 Warn Unity UnityEngine.Rendering.RenderPipelineManager:PrepareRenderPipeline(RenderPipelineAsset)
2024/10/01 08:55:28.961 3415 3433 Warn Unity UnityEngine.Rendering.RenderPipelineManager:DoRenderLoop_Internal(RenderPipelineAsset, IntPtr, Object)
2024/10/01 08:55:28.961 3415 3433 Warn Unity 
2024/10/01 08:55:29.004 3415 3433 Error Unity ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.
2024/10/01 08:55:29.004 3415 3433 Error Unity Parameter name: index
2024/10/01 08:55:29.004 3415 3433 Error Unity   at System.Collections.Generic.List`1[T].get_Item (System.Int32 index) [0x00000] in <00000000000000000000000000000000>:0 
2024/10/01 08:55:29.004 3415 3433 Error Unity   at NuitrackSDK.Avatar.SkeletonsUI.Update () [0x00000] in <00000000000000000000000000000000>:0 
2024/10/01 08:55:29.004 3415 3433 Error Unity 
2024/10/01 08:55:29.022 3415 3433 Error Unity ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.
2024/10/01 08:55:29.022 3415 3433 Error Unity Parameter name: index
2024/10/01 08:55:29.022 3415 3433 Error Unity   at System.Collections.Generic.List`1[T].get_Item (System.Int32 index) [0x00000] in <00000000000000000000000000000000>:0 
2024/10/01 08:55:29.022 3415 3433 Error Unity   at NuitrackSDK.Avatar.SkeletonsUI.Update () [0x00000] in <00000000000000000000000000000000>:0 
2024/10/01 08:55:29.022 3415 3433 Error Unity 
2024/10/01 08:55:29.040 3415 3433 Error Unity ArgumentOutOfRangeException: Index was out of range. Must be non-negative and less than the size of the collection.
2024/10/01 08:55:29.040 3415 3433 Error Unity Parameter name: index

Update: After removing both the NuitrackSDK from unity, and the nuitrack_arm64.apk; and installing them again (on the android app the sensor was activated and nuitrack packages were installed), and on unity I used another version from June (on from github releases). I could see another error message from the adb log, which was that the project was not able to run libnuitrack.so (it couldn’t find it), and whatever I done the app was not able to run the library, even after adding it manually to the app or copying the library directly to android.

@ofek Answered in PM