Multiple users control individual player characters

Multi person detection test
I have tested the Intel D435i camera using the skeleton example and it picked up 3 people (out of 6 present). Since the space where this wall interaction will be installed is not as spacious the 3 person limit is good enough.

The game interaction
I currently have a simple endless runner that works for 1 person. I have made it so the person can move left/right, jump and crouch a bit and the character they control will also move as they do.

Since the controls now work I wanted to tackle the multi user part of the game.

In the image, on the left side in the hierarchy, you can see a bunch of players, I tested how the split screen would look like. The Player is a gameObject with a Camera and a 3D player character model. It also has a PlayerMove script attached to it which takes in a reference to the 3D player character model so I can switch between playing the jump/crouch/run animations. I’m using a sphere as a joint (currently purple but will be invisible in the end).

This is the PlayerMove script

public class PlayerMove : MonoBehaviour
    [SerializeField] float moveSpeed = 3f;
    [SerializeField] float strafeSpeed = 4f;

    public static bool canMove = true;
    public bool isJumping = false;
    public bool comingDown = false;
    public bool isRolling = false;

    public GameObject playerObject; //reference to the player so we can play the animations

    public nuitrack.JointType[] typeJoint;
    GameObject[] CreatedJoint;
    public GameObject PrefabJoint;

    private float setMovementInY = 0.32f;
    private float moveDirection = 0;

    private float startYaxisPosition;
    private bool userDetected = false;

    void Start()
        CreatedJoint = new GameObject[typeJoint.Length];
        for (int q = 0; q < typeJoint.Length; q++)
            CreatedJoint[q] = Instantiate(PrefabJoint);
            CreatedJoint[q].transform.localScale = new Vector3(0.55f, 0.275f, 0.55f);
            CreatedJoint[q].transform.localPosition = new Vector3(0, 0, 0);

    void FixedUpdate()
        transform.Translate(Vector3.forward * moveSpeed * Time.deltaTime, Space.World);

        if (canMove){ //canMove is not used yet, will be used when in the menu screens
            if (NuitrackManager.Users.Current != null && NuitrackManager.Users.Current.Skeleton != null){
                for (int q = 0; q < typeJoint.Length; q++){
                    UserData.SkeletonData.Joint joint = NuitrackManager.Users.Current.Skeleton.GetJoint(typeJoint[q]);

                    if (!userDetected){
                        userDetected = true;
                        startYaxisPosition = joint.Position.y;

                    //moveDirection - multiplying the joint position and dividing by how fast we move
                    moveDirection = joint.Position.x * LevelBoundry.rightSide / strafeSpeed;  

                    //we can move left/right only when we are inside the LevelBoundry values, say -10 and 10
                    if (transform.position.x >= LevelBoundry.leftSide && transform.position.x <= LevelBoundry.rightSide){
                        transform.Translate(moveDirection / strafeSpeed, 0, 0);
                    //if we overshoot the left boundry set the value to LeftBoundry value
                    if (transform.position.x <= LevelBoundry.leftSide){
                        transform.position = new Vector3(LevelBoundry.leftSide, setMovementInY, transform.position.z);
                    //if we overshoot the right boundry set the value to RightBoundry value
                    if (transform.position.x >= LevelBoundry.rightSide){
                        transform.position = new Vector3(LevelBoundry.rightSide, setMovementInY, transform.position.z);

                    //Jumping implementation. Torso (joint) needs to make a difference of 0.2f to be detected as a jump. Also prevent rolling during a jump
                    if (joint.Position.y - startYaxisPosition > 0.2f && !isJumping){
                        if (!isJumping && !isRolling){
                            isJumping = true;
                            playerObject.GetComponent<Animator>().Play("Running Forward Flip");
                    //Rolling/crouching implementation. Torso (joint) needs to make a difference of negative 0.2f to be detected as a roll/crouch. Also prevent jumping during a roll
                    if (joint.Position.y - startYaxisPosition < -0.2f && !isRolling){
                        if (!isJumping && !isRolling){
                            isRolling = true;
                            playerObject.GetComponent<Animator>().Play("Sprinting Forward Roll");

            if(comingDown == false){
                transform.Translate(Vector3.up * Time.deltaTime * 3, Space.World);
                transform.Translate(Vector3.up * Time.deltaTime * -3, Space.World);

        if (isRolling){
            if (comingDown == false){
                transform.Translate(Vector3.up * Time.deltaTime * -0.5f, Space.World);
                transform.Translate(Vector3.up * Time.deltaTime * 0.5f, Space.World);

    IEnumerator JumpRollSequence(){
        yield return new WaitForSeconds(0.45f);
        comingDown = true;
        yield return new WaitForSeconds(0.45f);
        isJumping = false;
        isRolling = false;
        comingDown = false;
        playerObject.GetComponent<Animator>().Play("Slow Run");

However, I don’t want to get ahead of myself so I need some guidance on how I could implement multiple users that would be locked to a Player character until the game ends. But first how could I determine who from the people that the camera sees is player 1,2,3? I could add a menu before the game start that would require from participants to push a button to indicate they want to play, could the user ID be “locked in” to one of the player characters in this way?

I tried the SkeletonAvatar example and the script works as you can easily set the User ID. However, at runtime, each SkeletonAvatar script makes a SkeletonRoot game object that consists of joint prefabs and connection lines (not what I want and this can be seen in the image hierarchy and game view).

Since the SkeletonAvatar inherits from BaseAvatar which Inherits from TrackedUser I made the PlayerMove inherit from TrackedUser instead of MonoBehaviour like in the script above. This gave me the User ID which I could change on each Player inside the editor. However, I was still controlling both player characters. Since I was testing alone I at least expected the player character with User ID set to 2, to not move(only run straight), so what am I missing?

Hello @NUIty
It is correct that you inherit from TrackedUser class. >

“Current” in NuitrackManager.Users.Current.Skeleton… This means that the data will be received from a single user (most likely it is a user with id = 0)
Try replacing with this ControllerUser.Skeleton…

So I managed to detect multiple people but I don’t know if there is any convenient way to link a person with a player character based on their position relative to the camera. As player characters 1/2/3 were controlled by person 2/1/3

Is there by any chance any “hack” for a single person to test multi-person detection?

I also solo tested split screen for 2 people. I don’t know what the trigger was for this to occur, but during the game I was initially player 1 and then I was controlling player 2.

My last question is for matching person position relative to the camera. For instance if I have a game where users have to move left/right they would need to face the camera head on, this is not a problem for a single player but multiple people can’t occupy the same space.

At the moment my workaround is to get their X joint reference position at start, but if they later change the position from the initial reference position the controls don’t work as intended. Is there an easier way to do this? Otherwise I was thinking of changing the reference X position every 3-5 seconds.

Experiments are needed here, but it seems to me a good way is to determine 2-3 points (depending on the number of players). And, for example, if a person is standing closer to the leftmost point, then he controls the leftmost character on the stage, if in the center, the central one, and if on the right, then the right one. To determine how close a person is to a virtual point, you can track the position of the “waist” joint. If the skeleton is closest to this to some point, then the avatar switches to work with this skeleton. In the same way, you can test the work, if one person passes from left to right, then the characters in turn should switch to working with his skeleton.
Movement can be done in this way, if the player is standing at this point, then the character runs straight, if to the left or to the right, then the character shifts to the side.

Thank you for a possible solution, I will see if I can make something work.

1 Like