Speeding Up Web Sockets

[00:00:00]
>> So we already talked about how node has a ton of amazing stuff. We've now created headroom cuz obviously we could remove add in end, or remove this update function. Actually, before we do that, one thing we forgot to do is let's just look at the performance chart, and see are we actually, is update actually being changed now?

[00:00:22]
Because we didn't really actually measure it again to see how much are we shrinking update now. Because before we shrink it a bunch, maybe we're not shrinking it as much now, I don't know. So we'll wait for this thing to get running, I forgot to do this part.

[00:00:36]
I think it's important to kind of look at this. Let it run, let it run, let it run, let it run. Fantastic, yeah, we shrink update by a decent amount. Last time it was 25% now, it's 15%, so it seems maybe we could do better with update, and then we could potentially see some wins, but i just don't think that's where we need to be focusing on.

[00:01:13]
What I do see is we still have we have a little bit of garbage collection not a lot, we do have on message, we do have stream read, we do have right, we do have consume. We do have start loop, we do have some other socket on data, a bunch of these tiny functions all put together that are all causing some amounts of CPU.

[00:01:30]
And not only that, but if we look at the last time we do anything with memory, you'll notice that from here to here is all WebSockets. Can we make WebSockets better? Yes, we can make WebSockets better, we're gonna make WebSockets better, but we're not gonna change WebSockets, we're gonna continue to use WebSockets.

[00:01:54]
We're just not gonna keep on using our current WebSockets, because our current WebSockets is WS. The one that's recommended all over the place, have you used WS?
>> Yeah,
>> Yes, you have, everyone's used WS at some point, very convenient, very naughty, very slow, and causes a lot of memory, shockingly not like that great.

[00:02:14]
And so what we're gonna do is we're actually gonna use the sockets that band uses, we're gonna go use a native set of socket that does all the framing, and requesting in C plus plus, which means that every part of it from the network all the way to the frame can be constructed out of band of JavaScript.

[00:02:31]
And then when that's done, just the frame can be handed into JavaScript, so we can just do processing at that point. And that's a very different paradigm than what's happening right now, cuz right now, we're getting handed the raw data. We're then going through it in JavaScript doing all this piping, and readable, and all that stuff that node uses to kind of generate out these packets.

[00:02:49]
So it's a very different experience. And so I think this would be a good win because again, I'm not even using this side of things. I'm kind of using intuition at this point, and a little bit of looking at the graphs. One, I see that we are seeing some level of WebSocket showing up, not a lot, but I am seeing that we're creating a ton here.

[00:03:09]
And so that gives me a little bit of pause. Like, okay, this might be doing a lot more than we realize. And maybe the performance chart isn't showing us everything that it's supposed to. We're not doing like a deep analysis tool, which would be something more like perf, where we can actually see every last little bit of everything processing.

[00:03:26]
So maybe we're missing some data, I'm not exactly sure what's happening. We have this unusually high amount of idle, despite the fact that when I add more connections, our program slows down. So that can't make any sort of sense. So maybe we're missing some data, but it seems like the memory side of things is a really great indicator.

[00:03:42]
So we're gonna go with our gut, which our gut says JavaScript is slow, and then go with the memory chart, which says, this JavaScript is using a lot of memory, sounds good? Sound good. I've definitely skipped out on my presentation a little bit. So we've been doing hotspot optimizations.

[00:04:02]
So we've assessed, yeah, so the best JavaScript is not JavaScript. That's usually how I think about it, is that if we could just stop writing JavaScript, and I mean that literally, we would actually be you'd just be faster, you'd write something different, right? I mean, the best thing you can do for performance is not just don't do anything, don't that's your fastest program is the program that does nothing.

[00:04:23]
And so if we don't have that as an option, then using a language that's fast is always a really good intermediate. So here we go, uNetworking WebSockets. I have this thing hard-coded here. It's kind of an unusual way you install this. Typically, you expect it on npm and all that, but if you Identify, if you do npm i like this, it will actually install it from a Git tag on GitHub.

[00:04:47]
So we're gonna install it there, and then it has to do a little bit of compiling for your system. It's gonna release some findings. We still have the performance program running. So what I'm gonna do is I'm first gonna check out my second optimization cuz I wanna keep that optimization, removing promises did seem like the first major bottleneck.

[00:05:03]
So now any improvements to our program no longer has that as our real bottleneck cuz that actually was our first real win here. So from this one, I'm gonna check out the, Third optimization. And the reason why I'm doing that is because the first optimization though on paper looks really good, I don't know if it is really good yet.

[00:05:25]
And so I kinda wanna wait, and maybe it will become really good. When we get to the point where that's the new actual problem, it just hasn't been the problem yet. And so lots of paper wins, no practical wins, very sad. And so I'm gonna execute to get this little npm i, it's gonna do you networking WebSocket's, takes a moment to download.

[00:05:47]
And this will give us some delicious native sockets depending on your node version, it's kind of like what versions you need to download for you networking. It's a little bit complicated, but I'm positive you'll be able to figure that out. This one should work with pretty much the latest nodes stuff.

[00:06:05]
So we've installed your networking sockets, and so now we just gotta use them. I'm pretty lazy, I know how to do it, but just in case you guys don't know how to do it. What I'm gonna do is, I'm gonna go on the internet, and I'm gonna look up uWebSockets js.

[00:06:20]
I am gonna go to the GitHub. I am going to zoom in because this is 4k and that was crazy. And I'm just literally gonna copy everything, all right? We are programmers here, there's some percentage of what we do that is copying. I don't like to say it's all copying, okay, but there is some percentage some consts exists out there where the coefficient is less than one, and I'm just gonna paste this all there.

[00:06:45]
And before I do that, I'm gonna remove WebSockets, because we don't want WebSockets anymore. We don't want JavaScript anymore. 12 down, I'm gonna remove server. When I remove server, it's gonna cause a bunch of issues down here. And what I'm gonna do, I'm gonna delete these ones right here, but I'm gonna keep these two, because I wanna move the logs over.

[00:07:04]
All right, so now let's do the upgrade here. So I'll go up here, I'm gonna import MicrowebSockets, let's see. It's star as MicrowebSockets from uNetworking, or uWebSockets, there we go. So now we have this beautiful new librarian, it's nice and fast, very excited about it. And we'll delete out this thing, uwebsockets.

[00:07:29]
We're not gonna do SSL, okay? I mean, what do they think, I mean, come on. This isn't that kind of course, we're just here doing stuff. All right, so we have our app. We're gonna listen on Slash. I am not gonna do any of these items though, I'm sure there's some of these that could be really good, should we do per packet compression, all these kind of extra options in WebSockets, well, I haven't tested any of that.

[00:07:52]
Right now, I'm just simply gonna say no, and we have three functions here. Now, what makes this really interesting, Is how they set this up. They set it up in such a way that I think actually makes it way nicer to work with, and then we also wanna open.

[00:08:09]
So, I want you to think about how WS does it. They hand you a WebSocket, and then you need to add listeners to it. The event listener has a map, it does a bunch of stuff, and then you have to remove that part blah blah, there's quite a bit to it.

[00:08:24]
Whereas with this one, every single function that's called, they also just hand the WebSocket to you, which is a little bit different. You don't have to do things nearly the same way, which I think is is just a nicer way to do this. I like that because that means you can do more performance based activities, because you can make your program instead of having a reference to the WebSocket, and then listening for messages.

[00:08:48]
You can have a message listener that passes it through to some sort of queue specifically for that WebSocket cuz you already have that WebSocket. We could dangle it off the WebSocket, we could just add properties. In some reality, we could just be like queue, right? We could just add our own to it, because this is JavaScript you can do whatever you really want to do.

[00:09:04]
If we want it to be all kind of like efficient with our time, there's some cool stuff that you can do. But for now, we're not gonna do any of those things. I'm gonna erase this Git because, well, we don't need to do that. And we definitely don't listen on 9001, we listen on port 4269.

[00:09:18]
So there we go, we have this, and let me move over our nice delicious logging. We'll do that, I don't know what happens specifically in this file. But whatever, I know this is such a scale issue moment I'm having right now. For whatever reason right here, tree sitter falls apart, and cannot indent right here, and it just it just drives me nuts, and I already know if I were to look at frontendmasters.com, or look at Twitch, it's gonna just be a wall of the worst skill issue.

[00:09:48]
OK, we're not doing it, we're not looking at it. So there we go, we have this delicious nice listening function, everything's great. I will now delete this right down here, and I'll move this up here, so we have logging in proper for ordering. Yeah, get out of here.

[00:10:04]
All right, beautiful and beautiful. So now we need to kind of reorganize our app to work with this. Now how are we gonna do this? It might be a bit more tricky. First off, we probably don't need code or message, we really just need this. And so I'm gonna create something called onClose.

[00:10:20]
That's just gonna take a WebSocket. I'm gonna create something, let's see for, for open, well, we can actually just pass into the runner, the WebSocket, right? Because before that's what we were doing is, whenever a new connection came in, we just handed it to the runner, and said, here you go.

[00:10:35]
Here's a new web socket. Lastly for the message, let's do something like on message, and just pass in the WS. And then let's go buffer from, cause this is technically, it's an array buffer, right? So we're gonna go buffer from message. I'm sure there's a nice way to do this, but we know we're getting stringified messages.

[00:10:54]
We already know it's not binary data. So I'm just gonna pre-emptively springify it. So we need to create this on message, and onClose in our game. That can kind of browse through based on the WebSocket, so we can start updating these queues and everything. And so hopefully this is faster.

[00:11:08]
Hopefully my hypothesis that less JavaScript equals more speed actually bears some fruit. All right, let's find out. So the next thing we need to do is create those functions. So I'm gonna go all the way up here to the tip pity-top, and well, we don't have to wait for open anymore do we know, we don't pick us the thing tells you when the window sockets open.

[00:11:29]
Awesome, all right, so let's do our two functions. We're gonna go like this. We're gonna first do WS to state. And this is gonna be a new map. I don't need to do a week map, no one's a week around here, and this will be WebSocket to state.

[00:11:41]
I guess we're gonna be using a lookup like this. I have some ideas to make this better, but I'm just gonna keep it simple. Use a map, I know I'm gonna have a lot of them, it's not gonna be 20, it's gonna be 1,000, 2,000, 4,000, so,