TCP connection HoloLens2/ PC

Question

I would like to create a TCP connection between a HoloLens2 (Client) device and my PC (W11, server). I already tested this app on another HoloLens2, the connection is working, I'm just printing something in power-shell when the connection is happening, and on the HoloLens side I have a TextMesh printing the current state of the connection. The problem is when I'm changing device (I try this on another HoloLens2), in this case the connection is not happening. I tried already several debugging options (firewall, switch port, SLL in WDP). Still nothing. Some one has any idea of possible options IN the Hololens parameters that could block the connection? I would suggest that the problem is coming from the device as the code is working with one and also in Unity playMode (even if I do not consider this last fact as a sufficent condition). For what I know it is impossible to ping HoloLens to verify how available it is, can someone confirm that?

Thank you in advance!

Here below the client code (inside Unity attached to an empty Mesh):

using System;
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System.Net;
using System.Net.Sockets;
using System.Threading;
using System.Text;

public class TcpTestClient : MonoBehaviour
{
    private TcpClient tcpClient;
    private NetworkStream sendStream;
    private NetworkStream receiveStream;
    private Thread clientListeningThread;

    private int counter = 0;

    public Int32 connectPort = 13000;               // address and port to connect to:
    public string connectAddress ="your ip"

    String responseData = String.Empty;              // String to store the response ASCII representation.
    bool _connectionOn = false;
  
    public bool GetIfConnectionWorks()
    {
        return _connectionOn;
    }

    void decodeServerMessage(String responseData)
    {
        switch(responseData)
        {
            case "U":
            case "u":
                Debug.Log("I ENTER NEAR INCREMENT IF +");
                break;
            case "D":
            case "d":
                Debug.Log("I ENTER NEAR INCREMENT IF -");
                break;
            case "S":
            case "s":
                break;
            case "G":
            case "g":
              break;
            default:
                break;
        }
    }

    // Start is called before the first frame update
    void Start()
    {
        tcpClient = new TcpClient(connectAddress, connectPort);

        if (tcpClient != null)
        {
            Debug.Log("Constructor succeeded!");
            _connectionOn = true;
        }

        sendStream = tcpClient.GetStream();

        string msg = DateTime.Now.ToString("hh:mm:ss.ffff");
        
        SendMessageToServer(msg);
       // GetComponent<CreateLog>().WriteLog();

        //clientListeningThread = new Thread(new ThreadStart(Listen));
        //clientListeningThread.IsBackground = true;
        //clientListeningThread.Start();
    }

    public void FixedUpdate()
    {
        counter++;
        if(counter == 100)
        {
            
        }
    }

    public void Close()
    {
        // Close everything.
        sendStream.Close();
        receiveStream.Close();
        tcpClient.Close();
    }

    private void Listen()
    {
        try
        {
            Byte[] incomingBytes = new byte[1024];

            while (true)
            {
                // Get a client stream for reading and writing.
                receiveStream = tcpClient.GetStream();

                int msgLen;

                while((msgLen = receiveStream.Read(incomingBytes, 0, incomingBytes.Length)) != 0)
                {
                    var data = new byte[msgLen];
                    Array.Copy(incomingBytes, 0, data, 0, msgLen);
                    string serverMsg = Encoding.ASCII.GetString(data);
                    //Debug.Log("Server msg received: " + serverMsg);

                    // check type 
                    decodeServerMessage(serverMsg);
                }
            }
        }
        catch (SocketException socketException)
        {
            Debug.Log("Socket exception: " + socketException);
        }
    }

    public void SendMessageToServer(String message)
    {
        try
        {
            // Translate the passed message into ASCII and store it as a Byte array.
            Byte[] data = System.Text.Encoding.ASCII.GetBytes(message);

            // Send the message to the connected TcpServer.
            sendStream.Write(data, 0, data.Length);
            //Debug.Log("Sent"); Debug.Log(message);
        }
        catch (ArgumentNullException e)
        {
            Console.WriteLine("ArgumentNullException: {0}", e);
        }
        catch (SocketException e)
        {
            Console.WriteLine("SocketException: {0}", e);
        }
    }
}

Then the core part of the Server code.

using System;
using System.Net;
using System.Net.Sockets;
using System.Threading;
/**
* TcpServerApp
* This server listens for clients and prints a screen monitor through the Server Monitor class
* The messages to be received must be three float numbers sent in a string. 
* The client is the app "distance" on the Hololens 2. 
*/
namespace tcpserverapp
{
    class MyTcpListener
    {
        // Variables used to check when user types a character in the monitor
        static private bool commandChanged = false;
        static private string command;
        static private int pinkIncrement = 0;
        static private int blueIncrement = 0;

        static private bool CheckIncrementation(string command)
        {
            switch (command)
            {
                case "u":
                case "U":
                    pinkIncrement = pinkIncrement + 5;
                    commandChanged = true;
                    break;
                case "s":
                case "S":
                    blueIncrement = blueIncrement + 5;
                    commandChanged = true;
                    break;
                case "d":
                case "D":
                    pinkIncrement = pinkIncrement - 5;
                    commandChanged = true;
                    break;
                case "g":
                case "G":
                    blueIncrement = blueIncrement - 5;
                    commandChanged = true;
                    break;
                default:
                    commandChanged = false;
                    break;
            }

            return commandChanged;
        }

        /**
         * StartKeyboard Listener
         * Listens to keyboard inputs.
         * Called in parallel, run in a separate thread:
         * **/
        static private void StartKeyboardListener(TcpClient client)
        {
            NetworkStream sendStream = client.GetStream();

            while (true)
            {
                command = System.Console.ReadKey(true).KeyChar.ToString(); 
                if(CheckIncrementation(command))
                {
                    SendMessage(sendStream);
                }
            }
        }

        static private void SendMessage(NetworkStream stream)
        {
            if (commandChanged)
            {
                command = command.ToUpper(); // Process the data sent by the keyboard
                byte[] commandMsg = System.Text.Encoding.ASCII.GetBytes(command);
                stream.Write(commandMsg, 0, commandMsg.Length);

                commandChanged = false;
            }
        }

        /**
         * Main
         * Instantiate connection and manages monitor and messages in and out
         **/
        static void Main()
        {
            TcpListener server = null;
            ServerMonitor serverMonitor = new ServerMonitor();

            try
            {

                // Set the TcpListener address and port
                Int32 port = 13000;
                IPAddress localAddr = IPAddress.Parse("your ip adress"); 

                server = new TcpListener(localAddr, port);          // Instantiate the Tcp listener
                server.Start();                                     // Start listening

                Console.Write("Server set up! ");
                Console.Write("This is the local address: " + localAddr +" and this the port: "+ port);

                // Buffer for reading data
                Byte[] bytes = new Byte[256];
                String data = null;

                // Listening loop.
                while (true)
                {
                    Console.Write("\nWaiting for a connection... ");

                    TcpClient client = server.AcceptTcpClient();    // Could also use server.AcceptSocket() here.
                    
                    Console.WriteLine("Connected!\n");
                   
                    serverMonitor.SetMenu();                        // Set the distance menu

                    data = null;                                    // Reset the incoming data buffer to null

                    NetworkStream rcvStream = client.GetStream();   // Get a stream object for reading

                    Thread t = new Thread(() => StartKeyboardListener(client));
                    t.Start();

                    // Connection message reading loop
                    int i;
                    while ((i = rcvStream.Read(bytes, 0, bytes.Length)) != 0)
                    {
                        // Now we start the process whatever msg we receive
                        data = System.Text.Encoding.ASCII.GetString(bytes, 0, i); // Translate data bytes to a ASCII string.
                        
                        serverMonitor.startProcess("Start Process: "+data);
                    }

                    // Shutdown Server Monitor, end Client connection
                    // and terminate keyboard listening Thread 
                    serverMonitor.CloseMonitor();
                    client.Close();
                    try
                    {
                        t.Abort(); // It always raises a ThreadAbortedException
                    }
                    catch(ThreadAbortException e)
                    {
                        Console.WriteLine("Thread aborted: {0}", e);
                    }
                }
            }
            catch (SocketException e)
            {
                Console.WriteLine("SocketException: {0}", e);
            }
            finally
            {
                // Stop listening for new clients.
                server.Stop();
            }

            Console.WriteLine("\nHit enter to continue...");
            Console.Read();
        }
    }
} 

ServerMonitor class:

using System;
using System.Globalization;
using System.Linq;
using System.Diagnostics;
using Xunit;

namespace tcpserverapp
{
    /** Server Monitor Class 
     * creates a small monitor to visualize the distance of the hand from the objects while the connection is on
     * **/
    class ServerMonitor
    {
        private bool firstRefresh = true;
        private int counter = 2; // refresh counter to avoid flickers

        public void SetMenu()
        {
            Console.WriteLine("*************** Motor Imagery Application ***************");
            Console.WriteLine("*                                                       *");
            Console.WriteLine("\r* Arrived messages:                                      *");
            
            Console.CursorVisible = false; // hide cursor to avoid flickers c#
        }

        /**
         * ResetCursor to reset the cursor postition so that it does not rewrite any important value, 
         * but only the three lines from the menu. 
         * **/
        private void ResetsCursor()
        {
            if (!firstRefresh)
                Console.SetCursorPosition(0, Console.CursorTop - 12); //This line resets the cursor to the top of the menu
            else
            {
                Console.SetCursorPosition(0, Console.CursorTop - 12);
                firstRefresh = false;
            }
        }

        public void startProcess(string message)
        {

            /**ProcessStartInfo startInfo = new ProcessStartInfo("Powershell.exe");        // name of the process to start
            string PathToCounterFile = "C:\\Users\\abruschi\\Desktop\\TimerSynch\\timestamp_powershell.ps1";    // arg
            startInfo.FileName = "Powershell.exe";
            startInfo.Arguments = PathToCounterFile;

            Process.Start(startInfo);**/
            string timeserver = DateTime.Now.ToString("hh:mm:ss.ffff");
            
            Console.WriteLine(message + " time server:  "+timeserver);
        }

        public void updateHandDistance(string _handPos, string _fromPink, string _fromBlue, int _pinkIncrement, int _blueIncrement)
        {
            if (counter >= 2)
            {
                ResetsCursor();

                var ci = (CultureInfo)CultureInfo.CurrentCulture.Clone(); // Change the decimal separator - default separator is based on CultureInfo and it's "."
                ci.NumberFormat.NumberDecimalSeparator = ","; // For Unity tests: "." For HoloLens: ","

                int count1 = _handPos.Count(f => f == ',');
                int count2 = _fromPink.Count(f => f == ',');
                int count3 = _fromBlue.Count(f => f == ',');

                //Console.WriteLine(count1 + count2 + count3); Console.Write("\n");

                if ((count1 + count2 + count3) == 3)
                {
                    //float pink = float.Parse(_fromPink, ci);
                    //float hand = float.Parse(_handPos, ci); // parse 
                    //float blue = float.Parse(_fromBlue, ci);

                    //if (Math.Round(pink, 1) == 0){ pinkObjectReached++; }
                    //if (Math.Round(blue, 1) == 0){ blueObjectReached++; }

                    Console.WriteLine("\r* Hand Position:       {0}", _handPos); // Math.Round(hand, 1));
                    Console.WriteLine("* Distance From Pink:  {0}", _fromPink); // Math.Round(pink, 1));
                    Console.WriteLine("* Distance From Blue:  {0}", _fromBlue); // Math.Round(blue, 1));

                    Console.SetCursorPosition(0, Console.CursorTop + 7);

                    Console.WriteLine("* Pink Increment:  {0} ", _pinkIncrement);
                    Console.WriteLine("* Blue Increment:  {0} ", _blueIncrement);
                }
                else
                {
                    Console.SetCursorPosition(0, Console.CursorTop + 12); // To adjust the cursor when it gets in the loop again
                }

                counter = 0; 
            }
            else
            {
                counter++;
            }  
        } 

        /** 
         * CloseMonitor is just a temporary closing-window line, to make the monitor 
         * view cleaner
         * */
        public void CloseMonitor()
        {
            Console.WriteLine("**************************************************");
        }
    }
}

Answer 1

I have implemented both the client and server with the codes you have provided. Both works fine for me. The issue doesn’t occur in my testing environment.

However, it is worth mentioning that you need to configure necessary permissions for a UWP app. The common deployment process is to build first in Unity. Before the build, please navigate to [Unity] -> [Edit] -> [Project Settings] -> [Player] - > [Publish Settings] and enable “InternetClient” in Capabilities section. After that, in Visual Studio, you can also confirm this setting by double click the appxmanifest file in the solution.(should be named Package.appxmanifest).

Please also refer to https://docs.microsoft.com/en-us/windows/uwp/packaging/app-capability-declarations