ColorFrame Blue Output

Nuitrack SDK Unity3D
1

In Unity if two scripts get Texture2D from the ColorFrame during the NuitrackManager.OnColorUpdate event the texture becomes blue.

An example of what the image looks like:
Capture_2

To replicate, make a new scene with a NuitrackManager and add two Color Frame Canvas prefabs.

How can I avoid this?

2

Hi ed-spatialsauce,

We’ve reproduced your issue. There are two possible solutions.

Solution 1 (recommended):

Just do not receive two textures at the same time. Receive one texture and use it. This solution is effective and simple.

Example:

using UnityEngine;
using UnityEngine.UI;

public class DrawColorFrame : MonoBehaviour
{
    [SerializeField] RawImage background, background2;

    void Start()
    {
        NuitrackManager.onColorUpdate += DrawColor;
    }

    void DrawColor(nuitrack.ColorFrame frame)
    {
        background.texture = frame.ToTexture2D();
        background2.texture = background.texture;
    }
}

Solution 2:

Change the ToTexture2D method. Initially, Nuitrack passes ColorFrame in the BGR format instead of RGB. That’s why ToTexture2D also reverses B and R. As a result, the first script receives a color frame and swaps R and B to make an RGB image. The second script receives the same frame, swaps R and B one more time and as a result you see a blue image.

In DrawColorFrame.cs this is the line:
background.texture = frame.ToTexture2D();

In this case, you don’t have to change the same frame, just copy it and change the copy to RGB in NuitrackUtils.cs (see the method ToTexture2D for ColorFrame). You have to add one line for copying the array (see a comment in the script below):

    public static Texture2D ToTexture2D(this nuitrack.ColorFrame frame)
    {
        byte[] sourceData = new byte[frame.Data.Length]; //update this line
        frame.Data.CopyTo(sourceData, 0); //add a new line
        for (int i = 0; i < sourceData.Length; i += 3)
        {
            byte temp = sourceData[i];
            sourceData[i] = sourceData[i + 2];
            sourceData[i + 2] = temp;
        }

        Texture2D rgbTexture = new Texture2D(frame.Cols, frame.Rows, TextureFormat.RGB24, false);
        rgbTexture.LoadRawTextureData(sourceData);
        rgbTexture.Apply();

        Resources.UnloadUnusedAssets();

        return rgbTexture;
    }
1 Like