Callback Problems: Inversion of Control

[00:00:00]
>> [MUSIC]

[00:00:03]
>> Kyle: So I've alluded to this already. Let's try to be more articulate about the two major problems of callback hell. We saw the symptoms of them. The symptoms of callback hell where that managing that times the complexity was rather awful. It's very manual, it's very hard coded, hard to do.

[00:00:23]
But there's something deeper even than that. So to try to articulate those two problems, I'm gonna have to take some conceptual steps back a little further away from the code.
>> Kyle: First, I wanna talk to you about a principle called inversion of control. By the way if you do a search for inversion of control you gonna get lots of different answers in different software domains as to what inversion of control means.

[00:00:53]
So try not to confuse all those different overloaded definitions with the one that I'm going to give you here. What I'm gonna give you here is the inversion of control means. There's part of my program that I'm in control of executing. And then, there's another portion of my code that I'm not in control of executing.

[00:01:12]
And the way that we express that is to take the first half of my program that executes now and the second half of my code that executes in the callback, and when I give that call back to somebody else. That's what inverts the control and it puts them in control of when and in what manner to execute the second half of my program.

[00:01:30]
That's all and only what I mean by inversion of control. I'm going to assert to you that this is one of the two evils of callback hell, this notion of inversion of control. So let's go back to this fundamental setTimeout. There's a line 1 and a line 2, those represent the first half of our program, lines three and four represent the second half of our program.

[00:01:50]
We are in control of lines 1 and 2, they're happening now. We are observing them happening now, we are in control of them. Lines 3 and 4 are going to happen at some unspecified later time and we are completely not in control of how those lines, how and when those lines get executed.

[00:02:11]
We have given that control over to someone else, in this case it's the timeout utility. It is entirely in control of when and if this ever even gets executed. Now, with a set timeout that's a built in JavaScript engine facility and even though there's some weirdness around timers we can all probably say we don't worry too much we don't worry we don't have a trust issue.

[00:02:35]
Around the time or not calling the call back or doing something weird. So let me give you a slightly different scenario. What if it's not the built in engine? What if it's some other utility, maybe even a utility that comes to you by way of a third party library.

[00:02:50]
Let me give you a scenario set this up because I like to teach to silly metaphors and scenarios and things. So let's imagine that you are tasked with building an e-commerce engine for your company. Your company sells high end expensive TVs to celebrities. $10,000 TVs to celebrities. Your task of building the e-commerce site that sells those to people.

[00:03:14]
So you go through all the work of making this e-commerce site all the cart and all that other stuff. And the very last step of the e-commerce checkout, right is the person is clicking the checkout button you have a third party utility like an omniture or something like that.

[00:03:29]
They provided you some utility called Track Checkout which you need to call with the details, the order details And they will track that in the analytics so that your marketing folks know what's happening and your product folks know what's happening. They're gonna track that purchase info asynchronously, and then they're gonna call your call back to let you know that it's time for you to go ahead and charge the customer's credit card and print the thank you page.

[00:03:54]
So it's the last step of the puzzle. Everything seems great you're right this code is pretty straightforward, the API makes sense. I passed in this callback, you test it against their test bed and everything works great. So you go live and day one, celebrity start buying your awesome $10,000 TVs and everybody from the CEO all the way down to you is happy at the company.

[00:04:17]
So we fast forward in the scenario six months. One morning before work, you're sitting in a coffee shop casually sipping on a latte, and you get a call from your boss. And when you pick up, your boss is not usual casual self. Complete panic can barely forms full sentences.

[00:04:37]
Says there's a major crisis, huge PR problem. George Clooney tried to buy a TV on our site and his credit card was charged five times for the same TV and then he tweeted about it and now a bunch of people are complaining about our side of all this huge PR nightmare, you got to rush into work right now.

[00:04:55]
Your head spinning you don't even remember writing this code but you're rushing to work in your loses. Well, they've refunded his credit card but we still have this huge PR nightmare that we are working through but as engineers we've been told we have to figure out how this happened and they sure it never happens again.

[00:05:10]
Sound familiar to you. You've probably had your own version of a fire that you get told you have to make sure this never happens again. Sorry for your PTSD, if I'm triggering memories for you. But we can identify with this because this is our software works, works great until there's some huge fire and then all of a sudden our eyes are open to a set of problems that we didn't even know existed or maybe we thought existed but we really hoped we'd never run into and then we did.

[00:05:37]
In this case, you didn't even realize there was a problem. There's no obvious clear case of what the problem is. You don't even remember writing the code but you go pouring over your code that after you're looking through stuff, you arrive at this part of your code, you see that's the only place where we charge the customer's credit card and you turn a check.

[00:05:54]
Could you maybe of like click the button five times that you already have code in there to fix that and like what could possibly be the problem? You arrive eventually at the conclusion that there's no other possible way for the credit card to have been charged five times other than for the truck checkout utility to have called your call back five times.

[00:06:13]
It's the only logical conclusion. You scour their documentation, there's not a single mention of them calling your call back multiple times. So you call it Customer Support. You say, I got this huge PR issue, I need you to help me out. Seems like this callback has been called multiple times and I don't see any party or documentation.

[00:06:32]
Well, the support representative is similarly surprised, scours their documentation, never heard of it, got no idea. So we'll escalate it, talk to your support and we'll get back with you with the resolution. You've got nothing else to do, so you go over the day, come back the next morning and in your email first thing, you have this long email, this long detailed email, is kinda like when a company has a security breach and they put out their blog post that shows how everything happened.

[00:06:58]
They're like, at 9:02, we discovered this and at 9:18 we shut this off Is this big long email describing everything, but what it boils down to as they say this. Some of our engineers working on an experimental branch of the code, where they were going to try to call your callback, and if they didn't hear a successful return value from your callback, they would try once per second for five seconds and then eventually fail out, timeout after five seconds.

[00:07:23]
Now, this was experimental code. It was never supposed to go live, but there was a breakdown in our process and it accidentally went live. It was only live for five minutes before we rolled it back, but it just happened to be the five minutes that your site got caught in it.

[00:07:37]
We've checked all of our processes, we've implemented all this stuff, and we promise you this will never happen again. That's the real solution, you print out that email you take it your boss, you're like, this is what we got, they say it was their fault they called or callback.

[00:07:51]
Your boss looks at you and it's at this point of the story that I want to ask you, what are the chances that your bosses response is gonna be, what sounds good to me?
>> Speaker 2: [LAUGH]
>> Kyle: I would say approximately zero. Because at this point what has been exposed about your system is something you didn't even realize before.

[00:08:09]
You did not realize that there was a trust point when you passed in a callback. When you passed in a callback you are trusting that that callback will be not called too many times or too few times or whatever so they think well what I got to do the fix is because your boss to me like you've got to make sure that can never happen again.

[00:08:29]
Well, the light bulb goes off in your head and you say, I'm brilliant, I got an idea. I'll setup a Boolean flag. It'll start out as false. First time it goes through I'll flip it to true, and I'll make sure that that darned thing no matter how many times they call it, I'll only ever charge the credit card once.

[00:08:44]
Feel pretty good about yourself. You're like, this is awesome. So you check in that code, you produce a test case for this. You go to the QA department. Now, I'm about to bash on a QA department, but it's lovingly, so I apologize if you're in QA. Everybody loves QA.

[00:09:00]
We love to hate them, but we need QA, we need you. But you got to QA department I got this fixed for the bug, here's the reproduced case, blah, blah blah. And then, that QA person does what they always annoyingly do. They ask you that question that didn't even occur to you, you didn't want them to ask.

[00:09:18]
Well, what about if they never call our callback? Do we have any code in there for that? No [LAUGH], so now you're starting to panic like, no, I don't have a solution for that problem but I guess I could set up some kind of time out for that or whatever.

[00:09:36]
And then, he goes on he says what about all the other places in our code that we use callbacks. Do we have fixes for all of this and all of those places? And you get that blank stare on your face and you're like, maybe I can slowly back away and pretend I did not hear him say it.

[00:09:49]
Maybe I just can pretend I didn't hear any of this because all of a sudden what's been exposed is how deep this rabbit hole goes. The trust issues that we're involved in passing callback the inversion of control trust issues we were trusting that when we pass a callback to some other utility and by the way it doesn't have to be a third party utility, it could be somewhere else in your same program even stuff that you wrote.

[00:10:13]
But we pass it in and we trust that it won't call too early, not too late, not too many times not too few times won't lose out on any of the context it needs to pass and won't swallow any errors that may have occurred and actually the list is goes on and on and on.

[00:10:26]
These are all things that you have been implicitly trusting every single time you've ever passed a callback anywhere in any JavaScript program you've ever written. Until this moment, most of you have probably never asked the question, what happens if this trust falls apart? It's interesting to me that developers are actually pretty good at understanding why it's important to have that trust but verify thing with what's passed in.

[00:10:49]
Like, when you have a function that's supposed to add two numbers, and you assert that they are actually numbers before trying to add them together. We're pretty good at mindsets like that. And we've never once implemented any notion of checking that our callbacks are behaving the way that we expect.

[00:11:05]
And so, we run into this problem. So this is how I would say it. If you've ever had a callback in any JavaScript program that you've ever written and you do not have a solution in place for every single one of these trust issues. That program currently has a bug even if you've not been bitten by it yet.

[00:11:22]
This is a fundamental deficiency in callback design is that the callback itself does not have a solution, there is no solution for this part of callback hell, this inversion of control trust issue. Our particular one was not too many times but there are quite a few others. So if you wanna scramble and solve every one of those you might be able to come up with an ad hoc solution for every single one of those impacts that one solution and then you gonna start thinking I've got hundreds of callbacks in my program.

[00:11:56]
We're gonna have to solve this ad hoc in every single place. All of a sudden your codebase just Boolean by three or four x. It's completely unmaintainable and impractical at this point. We're gonna need something better, that's designed specifically to solve all these trust issues.