Visual Processing

[00:00:00]
>> [MUSIC]

[00:00:00]
>> Douglas: Let me give you an analogy from visual processing. Visual processing is the opposite of computer graphics. It's where you take a signal from a camera and analyze all the pixels and try to determine what are all the objects in the scene and how are they all moving relative to each other in the camera.

[00:00:21]
You need to be able to solve this problem in order to walk in the world and it's a really really difficult problem. But we're all able to solve it. And in fact all the animals are able to solve it. And the reason we're able to do that is because we don't solve the general problem because the general problem is hard.

[00:00:37]
Instead we solve special problems. Problems which are much easier to solve, but which sometimes give the wrong answer. And we have techniques for revealing the ways in which we get this wrong, and we call them optical illusions. So this is an illusion designed by Edward Adelson of MIT.

[00:00:56]
And here we've got a checkerboard with white squares and black squares. And two of the squares are labeled A and B. And it turns out those two squares are exactly the same color. As some of you may be seeing that you may believe that I'm telling you that, but you're looking at that and one is clearly white and one is clearly black.

[00:01:17]
But in fact they are both that color, they're both that shade of neutral gray. in fact if I overlay my square on top of the picture. If A and B were different colors. Then you should see some break in continuity, right, you should see some edge forming and you don't.

[00:01:35]
Now many of you may be seeing a gradient. Who is seeing a gradient here? Seeing the colors going from black and white, that come on. Yeah, yeah, yeah. Your brain is lying to you, that there's not a gradient. It is that color, it is a solid colored square.

[00:01:51]
You know the truth of this image and yet you still can't see it correctly. It's because there's another part of your brain that Kahneman doesn't talk about which deals with inconstancy, which is a really important thing to have because if a computer system becomes inconsistent, there's a really good chance it's going to fail, right?

[00:02:10]
It's just [INAUDIBLE] boom. And biological systems can't afford to fail when they get confused. So instead we've got this system, which will reject information which is coming from the world through our senses if it contradicts our internal state. And that's why you cannot see this correctly. Because it is altering what you are seeing with your eyes in order to conform with your patterns of image processing.

[00:02:39]
And it turns out we do this all the time, it's not just in the visual system, it's in everything. And that's why we argue so much because we literally cannot hear what the other person is saying. The other person is saying something which contradicts us then we will misinterpret what they're saying, it's not intentional, it's just part of what we're doing in order to resist inconsistency.

[00:03:03]
And this is the mechanism. So it turns out none of this was news to the advertising industry. They had figured out a long time ago that they could design messages which would bypass the head that would go straight to the gut convincing us that we have to buy things that we don't need.

[00:03:19]
They've been doing this for a long, long time. What Kahneman does is provide the theory which explains why that works. And I don't think anybody understood this better than the tobacco industry. Cuz you look at tobacco, okay, so how do you sell tobacco? What what does tobacco do?

[00:03:38]
It makes you smell bad, it turns your teeth yellow, it makes you sick, and then it kills you. So how do you convince people, yeah, let's buy some tobacco. And they do it by creating messages which are able to confuse system 1. System 1 is easily confused about the difference between slow death and good for you.

[00:04:00]
So that is the equipment that we have for writing computer programming or for writing computer programs. And programs are the most complicated things that humans make. There's nothing else in human experience which is composed of as many tiny little pieces which all have to go together perfectly that have to work In real time with changing states and changing inputs in a dynamic situation.

[00:04:27]
Nothing else comes anywhere close to the complexity that we have in software. And it's amazing that we're able to manage this. But it's really hard. So when the first Von Neumann machine started coming online, and it was discovered programming is really, really difficult. So there was a lot of thinking that we need to figure out a way in order to make it easier.

[00:04:50]
In fact to make it unnecessary because otherwise the computers are not much use to us. And so that started research in artificial intelligence. And the goal of artificial intelligence originally was to figure out a way to have the computers write their own programs, because it was just too hard to have humans writing them.

[00:05:11]
AI has had a lot of amazing successes but they have failed completely at their original mission, that you cannot give a problem to a computer and ask it to write the program that solves that problem. If it were then we would ask the computer, well write a program that's better than you and keep doing that until they become our overlords.

[00:05:32]
And that hasn't happened, computers don't know how to write programs, so we write them. So as a result of the failure of AI, we are still writing programs by hand.
>> Douglas: So the most important tool we have for doing programming is the programming language. Because one thing that computers are really good at, is translating one formal language into another formal language and that's what programming languages do.

[00:05:58]
They allow us to write at increasing higher levels of abstraction, where we get more leverage where we can create more work with less effort. And then the computer or through the programming language figures out how to convert it into a form that the computer can execute. And so programming languages are really, really important.

[00:06:20]
The thing that makes programming so extremely difficult is the requirement for perfection. Programs have to be absolutely perfect in every aspect, in every detail for all possible inputs, for all possible states for all time. And the contract we have with the computer, is that if a program is not perfect in all of those ways, the computer has licensed to do the worst possible thing at the worst possible time and it is not the computer's fault.

[00:06:47]
Whose fault is it? It's your fault. Why did you give us a program which you knew wasn't perfect? So you'd think well okay, because of that requirement for perfection, we would never release a program to production until we know it's perfect. And we don't do that for two really good reasons.

[00:07:05]
The first one is, we would never release a program to production, and eventually management would figure, or why are we paying these guys, they never finish, let's figure out something else. But the second reason is much more insidious and that is we have no test for perfection. It may be that someone once created a complex program that actually was perfect, there is no way to know.

[00:07:31]
There is no test for perfection. We have tests for imperfection but no tests for perfection, so there's just no way to tell. Now, it's most likely that none of our programs are perfect. And so what we do is we put our programs into production hoping that we'll discover what's wrong with it and fix it before anyone else finds out, which is crazy, but that is the state of the art.

[00:07:55]
That is the best we have figured out how to do. So we're doing this work with the brains of hunters and gatherers. And this is not a metaphor. Our brains have not evolved since the last ice age. And there was nothing in that experience to have prepared us for computer programming.

[00:08:11]
Hunting and gathering required a completely different set of skills. So, it's kind of amazing that we're able to do this at all. It's sort of lucky, but there is nothing in our evolution which has prepared us for that. So we're making use of everything we have in order to do this.

[00:08:31]
So programming obviously is making use of the head, of system 2 because that's where the algorithms live, that's where we do the analytical stuff, that's where we do the logic in the state and the dynamics, all of that stuff. But we don't understand how we do it. You have all figured it out but not well enough that you can write down a list of instructions.

[00:08:53]
This is how you write a computer program, you do these things, and hand it to someone else and they can read it and follow the steps and then they can write programs. We can't do that. Somehow we have all figured it out. But not well enough that we could tell someone else how to do it.

[00:09:07]
And that's why we can't tell the machines how to do it because if we can tell other human how to do it then we can tell machine how to do it, and we don't know. And much of what we're doing involves intuition because we're always making tradeoffs. It's never clear what the best approach to solving a problem is going to be.

[00:09:28]
And usually we're trying to solve a problem with incomplete knowledge of what the solution is gong to be. And so we have to guess, and system 2 is not good at guessing, system 1 loves to guess. System 1 will guess without knowing it's guessing, it's great at that.

[00:09:44]
And so we look at a problem and we'll look at it top down and bottom up and take a macro view and micro view. We keep constantly shifting our point of view until eventually a program emerges. And we don't know how we do that. So system 2 is obviously involved but I think system 1 is involved there as well, that's part of the reason why we can't document it.

[00:10:09]
Now I have no evidence to support that argument but my gut tells me it's true. So I believe it.