Changing part of texture according to depth

1

I’m new to using nuitrack and was wondering if it is possible to do something like this

Ive gotten to changing a color of a quad based on its depth value but am facing the issue of not knowing how to tell unity that certain areas are lower than another and to change the area accordingly

2

Hello @zenzx
I think it will be useful
How to use Depth: https://github.com/3DiVi/nuitrack-sdk/blob/master/doc/Unity_Point_Cloud.md
How to paint it in the desired color: How do i change the color of depth visualization

3

So ive managed to get to this point but now i am facing a new issue where the color stays after leaving the depth zone as shown in the video, is there a way to remove the coloring if the player leaves a certain area to not cause overlaps?

4

Heres a better visualization of the goal i have in mind, right now it only works with a single color but i want it to do the same but with different colors on different layers

5

This can be done by normalizing the depth for each pixel and matching this value with the gradients of your colors.

If you use the Point Cloud tutorial as a starting point, then you need to do the following:

  1. Change the texture format for depth Texture to ARGB32.

    depthTexture = new Texture2D(cols, rows, TextureFormat.ARGB32, false);

  2. Add fields for gradient and maximum recognizable depth.

    [SerializeField] Gradient depthGradient;
[SerializeField] float maxDepthSensor = 8f;

  3. Use the conversion of the depth value to color via a gradient.

    depthColors[pointIndex] = depthGradient.Evaluate(depthFrame[i, j] / (maxDepthSensor * 1000));

  4. Adjust the gradient to your liking.

image
image1626×643 293 KB

However, conversions to Point Cloud are not optimized and will heavily load your system. We will develop the most productive solution and offer it to you later.

6

Thanks alot for the solution it worked perfectly! One more thing to add would be instead of the display being pixels is it doable for solid colors instead to make the object and background be a smoother look

7

Yes, of course, you can change the gradient colors and make a smooth transition as you require.

8

We have prepared an optimized CPU solution for you.

Your question prompted us to improve the depth map display module and soon, methods for obtaining a depth map with arbitrary coloring will become part of NuitrackSDK.

Thank you for using NuitrackSDK!

1 Like
9

As for the smoothness, what I was thinking of was instead of it being pixels adjusted to color could it be a solid color instead

10

This can be easily done by switching the gradient mode to Fixed.

image

If you need one color of different tones, then it can also be adjusted using a gradient by setting the necessary color keys. This is a very flexible system, experiment.

1 Like
11

As for this package I am currently using unity version 2019.4.29f1 and it says the package was built for a newer version is there a way to get it to work for my version or do I have to update?

I use 2019.4.29f1 as its more stable with less bugs

12

Check out this package built for Unity 2019.

NuitrackSDK latest version is aimed at Unity 2020.3.1f1 and higher. Using earlier versions may cause errors and unstable operation.
We use this version as the most stable and tested for NuitrackSDK.
We recommend that you use 2020.3.1f1.