Binary Protocols and Inspecting Protocols

[00:00:00]
>> We've now seen a couple of examples of different text-based protocols, like we saw HTTP, SMTP, and IRC. And we were able to just type everything out by hand, because these protocols are designed for that. They're designed to be easy to implement, just by generating strings. What's really nice about text-based protocols too, is that they're really easy to inspect as they travel over the wire.

[00:00:28]
So this means that if you can somehow capture the information that's going over the wire, like going over the Wi-Fi, going over the Ethernet, then you can more easily debug things. You can also just poke around and see what's happening. With binary protocols, this is harder because you have to have like a decoder that's gonna require something like, special logic that knows how to parse that particular protocol.

[00:01:00]
A good example of binary protocol is SSH. So if you try to connect with netcat over SSH and you can just type something like hello, then it's just gonna tell you, no, you're not speaking this protocol correctly, go away. [LAUGH] Which is what it does here. Although with SSH, the first bit right here is actually a bit of text, but then everything else past that point is not.

[00:01:26]
So this actually discloses a lot of information, like about what version of open SSH you're running, and maybe not the best idea for every protocol to do that, but that's how it is. Also what operating system you're running. So to use these kinds of protocols, we have to use custom programs like the SSH command, or mosh, or something like that, that knows how to speak that combiner or protocol.

[00:01:51]
>> Wireshark.
>> Yeah. However, if we want to know what's going on, not in SSH because it's an encrypted protocol, but in other kinds of protocols that are unencrypted including some binary protocols, we can use something like Wireshark, or I like tcpdump is very fantastic for writing little shell scripts.

[00:02:16]
There's also TShark, which is command line version of Wireshark. That's quite good, and you can sort of inspect what's traveling over the wire on the network, and it's lots of fun. It's also a useful debugging tool sometimes. If you're running a program, it's like doing stuff, it's phoning home, doing all kinds of things that you maybe don't want it to do or you need to know what it's doing, then this is sometimes the best way to do that.

[00:02:44]
Oops, so if you have tcpdump, crap, I don't have that installed. I do have TShark though. If you have tcpdump installed, on Linux at least you can run this command, tcpdump -x, which will show a hex representation of all of the data that's going over the network. So if I do tcpdump -x, I see all of this wonderful information that's just traveling over the Wi-Fi here.

[00:03:20]
You'll notice that a lot of this is actually kind of in plain text. Like if I scroll up, I see I have a bunch of HTTP traffic up there, anyways. So how tcpdump is formatted on the left, you have the hexadecimal representation, and this column is the byte offset.

[00:03:45]
And then you have the ASCII representation. So a lot of packets have the source and destination host just in an ASCII over the wire. You can also filter. So if we only want to see port 80, we can run these commands. And 80 is HTTP, which is unencrypted.

[00:04:09]
So we should be able to see a lot of fun stuff. So I can generate an HTTP response by opening up cURL and requesting some data from my website. And here we can see if I scroll up a bit, indeed here is just raw HTML coming down the pipe, and we can also see the request.

[00:04:34]
This is also true if you're on the same Wi-Fi, even if it's encrypted by someone you'll be able to, unless you're using one of the sophisticated WPA-Enterprise E versions, your computer will be able to see everything that's not encrypted over the wire. So use HTTPS and use encryption, unless you want people to be reading what you're doing on the web, at a coffee shop, or whatever.

[00:05:08]
So some fun stuff to try and of course, you'll see that the browser, or cURL, or whatever is sending all of this information in plain text. So you can see GET, and that's a space slash space HTTP, when that one user agent cURL hosts subset.net, etc. This is also a fun thing to try.

[00:05:34]
I forget what capital A does, but probably it's something cool. Yeah, this is just another way of formatting the data, so you can read the HTML a little more clearly. You can see that this is also using transfer encoding chunked, with these byte lengths which are in hex.

[00:06:02]
Okay, so if you want to read more about these different kinds of text protocols, I think the best way is to just dive into these files called RFCs and just read them. All right, you don't have to read them like front to end or anything, but just skimming them is oftentimes a great way to learn about new little bits of trivia that maybe you didn't know about.

[00:06:23]
I'm sure if I read these, I would learn stuff that I don't know right now, cuz I haven't read these in a long time. So here are some links for that. And that's all that I have for the networking section. So thanks, and play around with all of this stuff.

[00:06:41]
Hopefully, it was helpful.