Declarative Data Flow

[00:00:00]
>> Kyle Simpson: So back to the candy factory. You've been thinking about this for a while and you think you've come up with a brilliant solution. I don't need to change anything about the machines. All I need to do is wrap one nice little clean box around the whole setup.

[00:00:14]
Just make one nice big box that's wrapped all the way around all those internal machines. And on the inside, there'll be a nice access panel where I can open it up and get it all the machines and wires and buttons if I, the engineer, need to work with it.

[00:00:30]
But the factory workers just have one on, off switch. They have one input valve at the top where they pour stuff in and out from the bottom comes the candy wrapped and nice, ready to go. And you're thinking to yourself this is the best of both worlds because the factory workers only care about input and output.

[00:00:49]
But I, the engineer, I care about how it gets done. What have I done? I've created an abstraction for my composition, all right? I've wrapped it in a function. That's what this big box is conceptual. You implement this solution, and then you get some feedback from the boss.

[00:01:13]
He says, all the factory workers love it. They think it's great. Every day they're coming in, they're telling us how easy it is to push the buttons and our other maintenance engineers can open up the access panel whenever they need. And it's all nice and great job, great solution.

[00:01:28]
Six months go by, everything's been humming along nicely and the boss comes up to you one day and says I've got another problem I need you to solve. This is a very needy boss, right, got a lot of problems for you to solve. That's good job security anyways.

[00:01:40]
All right, so he's got another job for you to solve, another problem. You see, what we've noticed is that our competitors are able to come up with new kinds of candy almost on a daily basis. They try out one kinda candy one day and the next day they're producing an entirely different kinda candy.

[00:01:59]
It's amazing how quickly they can just spin out new kinds of candy. And we're not able to keep up with all the different kinda experimentation that they're doing. We just only produce the one kinda candy over and over again. And we've tried, we've experimented, but it's too slow to make these single boxes with all these machines configured.

[00:02:17]
Can you come up with some way for us to do this more efficiently? For us to make these machines that make our candy for us, we want to make them more efficiently. Without even realizing it, the boss is asking you the most ludicrous thing of all. He's saying, could you make me a machine that makes machines?

[00:02:38]
>> Kyle Simpson: A machine that takes machines as inputs and outputs other machines. Could you make me something like that? Go invent it. This is a real head scratcher. It's gonna take a while to try to invent that, okay? But you'll work on it. Let's switch back to the code.

[00:02:55]
We've been thinking about that shipping rate calculation, and it's been going along fine, and then somebody else in the company says, we actually have different kinds of shipping rates. We have international shipping rate, we have domestic shipping rate, we have express shipping rate, and overnight shipping rate. But everywhere we keep defining all these different functions is cluttering up our code.

[00:03:18]
Could you make it easier for us to define these shipping rate functions? So you, the programmer, think well sure, because there's a pattern involved in defining each of these functions, which is that each of the shipping rate functions takes a different set of three functions and calls them in succession.

[00:03:41]
So you think to yourself, I'll just make myself a utility. And maybe I'll call this compose three. It takes three functions fn3(fn2(fn1. I'll explain why that ordering in just a moment. It takes those three functions and it produces a new function, which we'll call each one of those in succession.

[00:04:01]
First one gets the input value. Output of that becomes the input to the next. Output of that becomes the input to the next. And the output of that is finally returned. And you notice now that you can just define as many shipping rate functions as you want. You make the original shipping rate function like this, but later you might say the international shipping rate, and instead of saying- 2, you say- 1.

[00:04:23]
And instead of saying triple, you say quadruple. You just pass in different functions, and you get a different shipping rate function as a result.
>> Kyle Simpson: Can you see why the definition of shipping rate on line five, not only is it point free, but it's more declarative? It says to somebody who knows what composition is, I know that it's going to go in order.

[00:04:52]
I know that it's going to go in order, from right to left. It's first gonna call passing any input to increment. Then it's gonna pass the input to triple, then it's gonna pass the input to- 2, and finally it's gonna return the result.
>> Kyle Simpson: Functional programmers know that a composition works exactly like that.

[00:05:11]
If you have a list of functions, it will execute them in right to left order, passing the output of each one as the input of the next, and finally giving you that result. And what we see here now is that composition is declarative data flow. It is the flow of data through a series of operations defined declaratively rather than imperatively.

[00:05:36]
Here is why this is so critical. Your programs don't mean anything if they don't mean data flow. The whole point of your program is to have data coming in, doing stuff, and then going back out. So your entire program is a series of data flows. It's a series of state transition managed data flows.

[00:05:59]
And the functional programmer knows if that's what our programs are about, then I should declare my data flows in a clear and obvious way rather than having my data flows implicitly linked. This function calls this one, who calls this one, who calls this one. Anybody got an application like that where functions call each other all over the place?

[00:06:20]
Tracking the data flow is pretty difficult, isn't it? How easy is it to track the data flow on line 6? It's as easy as going from right to left, isn't it? Composition is so critical to functional programming because the entire point of your program is to have data flow.

[00:06:38]
This is how we make it declarative. That compose three utility, by the way, that I invented for the purposes of this discussion, there's a general form of that, which we'll get to later, and it's just called compose, and it takes any number of functions. So you could compose one or two or five or 100.

[00:06:59]
It's a general compose utility and it's provided by all your favorite functional programming libraries.