WebSockets in C#: How do I read a data frame when the bytes are not what they're expected to be?

Question

I'm learning how the websocket protocol works and using C# to listen and receive some websocket data from a client. I'm using Postman to act as my remote websocket client to connect and send data to my server I'm developing. I've made it past the handshake portion and trying to decode the data frames that Postman is sending to my server. I've read what the data frames are suppose to look like here according to this guide by MDN, but Postman is send data with a byte arrangement I was not expecting.

[2022-05-10 14:56:52] [DEBUG] GET / HTTP/1.1
Sec-WebSocket-Version: 13
Sec-WebSocket-Key: cx/wF1vJpkOWmRzFuwaUFA==
Connection: Upgrade
Upgrade: websocket
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits
Host: 127.0.0.1:9050


[2022-05-10 14:56:52] [DEBUG] New websocket request.
[2022-05-10 14:56:52] [DEBUG] Received key: cx/wF1vJpkOWmRzFuwaUFA==
[2022-05-10 14:56:52] [DEBUG] Generated key: L+c/Aeow/W36QkJThZHlh7oSLFE=
[2022-05-10 14:56:52] [DEBUG] Sending response headers:
HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: L+c/Aeow/W36QkJThZHlh7oSLFE=


[2022-05-10 14:56:52] [DEBUG] Client connected.
[2022-05-10 14:56:55] [DEBUG] Client data received: {129, 142, 135, 123, 155, 16, 229, 23, 250, 120, 167, 25, 247, 113, 239, 91, 249, 124, 230, 19}
[2022-05-10 14:56:55] [DEBUG] ���{���x��q�[�|�
[2022-05-10 14:56:55] [DEBUG] Opcode: {155, 16, 229, 23}
[2022-05-10 14:56:55] [DEBUG] Opcode: 9B-10-E5-17
[2022-05-10 14:56:55] [DEBUG] Payload length: {120, 167, 25, 247, 113, 239, 91}
[2022-05-10 14:56:55] [DEBUG] payload length: 4145653624
[2022-05-10 14:56:55] [DEBUG] Client data received: {129, 142, 148, 26, 24, 129, 246, 118, 121, 233, 180, 120, 116, 224, 252, 58, 122, 237, 245, 114}
[2022-05-10 14:56:55] [DEBUG] �����vy�xt��:z��r
[2022-05-10 14:56:55] [DEBUG] Opcode: {24, 129, 246, 118}
[2022-05-10 14:56:55] [DEBUG] Opcode: 18-81-F6-76
[2022-05-10 14:56:55] [DEBUG] Payload length: {233, 180, 120, 116, 224, 252, 58}
[2022-05-10 14:56:55] [DEBUG] payload length: 1954067689
[2022-05-10 14:56:56] [DEBUG] Client data received: {129, 142, 182, 194, 166, 6, 212, 174, 199, 110, 150, 160, 202, 103, 222, 226, 196, 106, 215, 170}
[2022-05-10 14:56:56] [DEBUG] ���¦Ԯ�n���g���jת
[2022-05-10 14:56:56] [DEBUG] Opcode: {166, 6, 212, 174}
[2022-05-10 14:56:56] [DEBUG] Opcode: A6-06-D4-AE
[2022-05-10 14:56:56] [DEBUG] Payload length: {110, 150, 160, 202, 103, 222, 226}
[2022-05-10 14:56:56] [DEBUG] payload length: 3399521902

My console output above. You see it does the handshake, client stays connected and sends 3 data frame all containing the same text message: "blah blah blah" (as seen below in Postman).

What I don't understand is this byte arrangement that Postman sends. According to MDN, the first byte should be FIN and RSV1-3. The first byte in the first data frame is {129, 142, 135, 123}. If I'm understanding this protocol right, the last 3 bits in that arragement should be 0 unless Postman is using the reserved bits for something I don't know of. What's weird is the first two bits are always the same, but the rest is always different. In fact the whole byte arragement is different except for the first two-- which makes no sense to me given that each of these 3 data frames should be exactly the same.

The Opcode isn't what I expected it to be looking at the hex values.

And then I have no idea what to make of the payload length.

Am I sequencing these bytes correctly?

void DecodeWebsocketMessage (ref SS.TcpClient client, ref List<byte> bytes) {
   var fin  = bytes[0];
   // var rsv1 = bytes[1];
   // var rsv2 = bytes[2];
   // var rsv3 = bytes[3];
   var opcode = bytes.GetRange(4, 4);
   var mask = bytes[8];
   var payloadLen = bytes.GetRange(9, 7);

   Log.Debug("Opcode: ", Log.StringifyCollection(opcode));
   Log.Debug("Opcode: ", System.BitConverter.ToString(opcode.ToArray()));
   Log.Debug("Payload length: ", Log.StringifyCollection(payloadLen));
   Log.Debug("payload length: ", System.BitConverter.ToUInt32(payloadLen.ToArray()));
}

---

EDIT 2022 05 12 1404
I'm still not understanding how to interpret these bit fields. The guides say "byte 0 is fin, rsv, and opcode". Ok I got that. Then "byte 1 is mask and payload length". Ok I got that too, but in the data I'm getting the mask bit doesn't appear to be set.

 129[1000 0001], 142[0111 0001]
 ^^^             ^^^ ^
 ^^^             ^^^ This is the mask bit.  
 ^^^             ^^^
 This is byte 0. This is byte 1.

So on byte 1, I see the first bit is 0, which means unmasked data, which means the payload isn't XOR encoded, right? So I just read everything starting at byte 2 as plain UTF8, right? But I get garabled ASCII chars when I do, so it is still masked.

Am I not interpreting byte 1 correctly?

The programming calculator in Windows says that decimal value 142 has a binary representation of 1000 1110, which is not the same as 0111 0001 seen above in my print out. I thought maybe I wasn't converting bytes to bits correctly, but no I am. My log output shows that element [8] in a Collections.BitArray (the first bit of byte 1) is indeed False, so I'm not sure what I'm missing here if that bit is suppose to be set. I guess maybe Postman doesn't set the mask bit?

When I put in a binary value of 0111 0001 into the programming calculator in Windows, it does show a decimal value of 113 which is the payload length I was able to derive. So I'm guessing I am intrepreting this right. I"m assuming there is a mask key and I'm going to ignore that mask bit for now for the sake of understanding the next procedure.

---

EDIT 2022 05 13 0241
Ok so it seems like Collections.BitArray is doing some weird things to my bit order. It's reversing the bit order of every single byte, and I didn't notice this until I tested teh binary value of each byte by hand using the programming calculator. I didn't see this at first because the binary for 129 is a perfect mirror 1000 0001 and totally threw me off.

What's weird is that the whole Bit Array isn't reversed. Each individual set of 8 bits is reversed which is super weird to me. Can anyone else confirm the behaviour of Collections.BitArray? Is it suppose to be doing this? Is there some byte/bit preparation I'm not doing correctly?

Answer 1

Yeah... I feel pretty dumb for trying to treat bytes as though they're bits. Thanks to @MKR, I looked at what I was doing with bytes a lot more carefully.

This output makes a lot more sense now.

[2022-05-11 23:01:47] [DEBUG] Client connected.
[2022-05-11 23:01:53] [DEBUG] Client data received: {129, 142, 5, 186, 106, 148, 103, 214, 11, 252, 37, 216, 6, 245, 109, 154, 8, 248, 100, 210}
[2022-05-11 23:01:53] [DEBUG] ???j?g??%??m??d?
[2022-05-11 23:01:53] [DEBUG] Decoding bits:
{129[1000 0001], 142[0111 0001], 5[1010 0000], 186[0101 1101], 106[0101 0110], 148[0010 1001], 103[1110 0110], 214[0110 1011], 11[1101 0000], 252[0011 1111], 37[1010 0100], 216[0001 1011], 6[0110 0000], 245[1010 1111], 109[1011 0110], 154[0101 1001], 8[0001 0000], 248[0001 1111], 100[0010 0110], 210[0100 1011]}
[2022-05-11 23:01:53] [DEBUG] fin: True
[2022-05-11 23:01:53] [DEBUG] opcode: 1
[2022-05-11 23:01:53] [DEBUG] mask: False
[2022-05-11 23:01:53] [DEBUG] payloadLen: 113