Live Example: Thought Process

[00:00:00]
>> Mathew DesLauriers: So yeah, so I'm just gonna reset this to how it was before I started tweaking this stuff.
>> Mathew DesLauriers: So, remember we had mesh basic material, which is just choosing a random color, and it's an unlet material. So let's say we wanted to actually give each of these cubes its own sort of shader surface, rather than just doing a single color.

[00:00:22]
So here's where we would replace mesh basic material with shader material. And three.js has these two different types of shader materials. It has one called ShaderMaterial, it has another one called RawShaderMaterial. And RawShaderMaterial is for more advanced uses like if you really wanna have full control over the shader, and you don't want three.js doing anything helpful or fancy, then you can use raw ShaderMaterial.

[00:00:52]
But usually, I like to use ShaderMaterial, three.js just makes a few things a little bit easier, and so we can do that. Another thing I'm gonna do is before I get too far into this, I'm gonna rerun this command with hard loading, just cuz I think it's really cool, and now when I say it's just gonna be nicer and so, that's all.

[00:01:15]
So, to get started with this, you have to first use ShaderMaterial and then you specify fragmentShader.
>> Mathew DesLauriers: And this is a string that we have to pass, and so we're gonna create the string up here somewhere.
>> Mathew DesLauriers: And it'll just be ESC templates string.
>> Mathew DesLauriers: And it's gonna look like this, so again it's our shader.

[00:01:38]
You'll notice immediately a couple of different things, one of the things that different. Is that we don't have to say precision high P float at the very top of our file. We don't have to do this precision thing, that's because three.js inserts it for us. And if we were using raw ShaderMaterial, then we would have to specify our own precision and stuff like that.

[00:01:59]
But if you just want the basic shader, just use it like this, and so here we have our shader code. And let's just try making everything red to start off, my like go to.
>> Mathew DesLauriers: And if it all works you should end up with this red mesh.
>> Mathew DesLauriers: And now we can actually do basically the same stuff we were doing before around things like creating cool colors and creating different ramps of colors and stuff like that.

[00:02:38]
One thing is sometimes in three.js, I don't know actually how this is gonna work, vUv might exist by default in their shader or it might not, let's see. So if I just use vUV.x, it's gonna give me an error, and this is where things get a little bit more in depth.

[00:02:59]
Let's see,
>> Mathew DesLauriers: Varying vec2. And it's most likely because the default vertex shader of three.js doesn't include this information. And this is where we get into a bit of a higher concept, we have to actually write our own vertex shader, as well. And again, it's a string, we just pass it into our shader material, and our string is gonna look like this.

[00:03:29]
Void main, and in a vertex shader, we're not putting the frag color, we're outputting the GL position. And just like frag colors in vec four, it's talking about the xyz and w position and, it's actually the screen position. Anyways, we just imagine it as the position of each vertex in our cubes in all of our scene.

[00:03:56]
And the way it's gonna work is you have to just kinda memorize this line.
>> Mathew DesLauriers: So it's a long line, and the reason we're memorizing it like this, is because three.js gives us a couple built in utilities that don't exist if you're using a raw ShaderMaterial. But they do exist when you're using just a regular shader material.

[00:04:21]
And what's happening is that we're matrix math, whereas this is a matrix, and this is a matrix, and this is a vector. And this is kinda why I didn't wanna include this in today's lesson, but it's a little bit more scary. Because matrix math is something, maybe, not everyone's familiar with.

[00:04:39]
But, what's happening is we have a matrix that defines the way that the camera looks so this projection matrix is gonna either be perspective or orthographic. Just for reference just remember the difference, this is perspective versus orthographic, and you can sort of wrap up this information of whether its this way or this way.

[00:05:03]
You can wrap that up in a matrix, in a set of numbers. And this set of numbers is kinda contained in there and then model view matrix, that's just saying that's basically gonna position each cube in the correct place. And this last part is basically just gonna position each vertex and each cube in the correct place, and remember how with a pixel shader everything is running on each pixel in the surface.

[00:05:36]
We're talking about that grid of pixels, and how each pixel shader runs on its own pixel. Well in the vertex shader, every vertex runs its own vertex shader. And so it's a similar concept of running many, many times, and when we talk about a vertex, a triangle is made up of three vertices is plural.

[00:06:01]
And a cube would be made up of a lot of vertices because it have one face which is two triangles, and then another face and then another face and then another face. So there is a lot of vertices in a cube, and this method, this main method, for the vertex shader has been called on each vertex.

[00:06:19]
So it's still, you're gonna get an error if you look in the console. And that's because it's varying UV doesn't exist anywhere yet. And we hae to pass it down from the vertexShader to the fragmentShader. And this is a bit of a mind bender is the way the opengl pipeline works is that you specify vertices.

[00:06:45]
So you create a triangle, or you create a cube, or you create a sphere and that sphere is gonna be made up of lots of these vertices. And then you have a shader that describes each of those vertices. And then you'd pass down your information from your vertexShader, down through the pipeline, to the fragmentShader.

[00:07:04]
And the fragmentShader will use those things that you passed down, it'll use those varying things. So, uniform is something we were setting from JavaScript varying is something that's set from the vertexShader. And already we can see with this code we have a custom shader on our mesh. We can see that it's a bit like a gradient ramp, like we were doing earlier.

[00:07:32]
The only difference is that it's just red so I can I can just do the same code we were doing earlier.
>> Mathew DesLauriers: Just using gradients, this is kinda cool actually interesting, if I set the clear color to,
>> Mathew DesLauriers: White. I mean, it's kind of a cool image on itself, these gradient cues.

[00:07:56]
And this is kind of a nice part of creative coding in generative art, is when you just like tweaking things and you're like, this is kinda cool, I like this. So that's kind of nice, and then with this code, we already have everything we need to manipulate either the pixels of the mesh, or manipulate the actual vertices of the mesh.

[00:08:18]
So let's say we wanted to manipulate the vertices of the mesh, which is quite an advanced topic but we're gonna just do it anyways.
>> Mathew DesLauriers: So the way it would look is a bit like this, we would first, make sure we have position. We just pull it out as a vec3 variable here, just reference it from here.

[00:08:43]
And we're going to just multiply it by some value sine of time, and we're gonna get an error. And that's because time doesn't exist in our shader anywhere. Just like with our 2D code, we need to specify a uniform for time, because it's being set from JavaScript. And uniforms are set from JavaScript, varyings are set from vertex\Shaders.

[00:09:11]
>> Mathew DesLauriers: So here we go, first step is to define it in the vertexShader. And then the next task is to pass it down from JavaScript, and this three.js, the syntax is a little bit different, you can tell it's vertexShader, and fragmentShader. Different syntax, same with the uniforms, it's not passed down in the same way that we're passing down the uniforms in our other sketches.

[00:09:35]
But in this case, you just have to say uniforms, time, it's kind of a tedious syntax, but this is how it looks. You say uniforms, it's an object, and within that object you have other properties. These properties have to be the same name as the value you're putting into the shader.

[00:09:58]
And then instead of just an actual number, you have to wrap that number in an object that has value.
>> Mathew DesLauriers: And I might have broken something.
>> Mathew DesLauriers: I did break something, sort of, because right now we're setting the time to zero and we're not changing the time. So this is where three.js is a little different than this shader utility I was using.

[00:10:32]
And the reason I had us all working with the shader utility is just cuz the syntax is really tight and really simple. But when we actually go into three.js you have to set the uniforms, you have to define it in the shader. And then you have to update it on each frame, so here we are in the render function.

[00:10:50]
Let's expose the time prop, and let's go through all of our meshes. So we have to use a foreloop or a foreeach function, and we have to actually get a value that's our list of meshes. So I'm just gonna create this array, it's an empty array, and I'm gonna push each mesh into this array.

[00:11:12]
So meshes.pushmesh, cuz I just wanna store a reference to all of these mesh objects that I've created, so that's all I'm doing. And then down here, when we're rendering, before each time we render, we're gonna do meshes.forEach, and we're gonna loop over all those meshes. And we're gonna update each of the meshes uniform time values to be the same as the time it's coming from canvas sketch.

[00:11:39]
So it has to look like this mesh.material.uniforms.time.value = time. And make sure you have that value at the end here, it's kind of important.
>> Speaker 2: Are they not all sharing the same like shader program? They all have their own-
>> Mathew DesLauriers: So they're all sharing the same exact code from the shader.

[00:12:06]
And the only way you're gonna produce differences between different cubes is by setting different uniform variables or different values like that. Yeah, so because each cube is it's own mesh, we can give each cube a different uniform, right, but the actual code isn't changing. The code itself is always this exact string and it's just, if we introduce another uniform which actually we should do, let's introduce a color, cuz in our original thing we had each cube as a different color.

[00:12:40]
So a color is gonna be vec3, and color being RGB, so that's why we're using vec3. I'm sorry, I'm messing up, this color stuff should go in our pixelShader. Not in our vertex here because pixel shaders we to site things like color and those kind of things. And let's just remove this one here cuz we all ready have it here, and for now we'll just do color times vUv.x.

[00:13:12]
You'll see everything is black and that's because we're not passing any color down through three.js. So here inside of our uniforms, we had a new uniform called color, and three.js has this nice color constructor.
>> Mathew DesLauriers: And so the color, let's say we're gonna set it to be red, you'll see all of them are gonna be red or orange, all of them are gonna be orange.

[00:13:43]
And then this whole thing down here, this color attribute, is not even being used and it's actually giving us an error in the console and it's warning us about that. And it's because when you use shader material, you don't just have a material attribute called color anymore. You have to actually wire up all of your individual attributes, whether it's color or anything like that.

[00:14:05]
So, here we're just gonna replace orange with random.pick pallette.
>> Mathew DesLauriers: So now, we'll end up With cubes with multiple colors and gradients, and all of these cubes their vertexShader is actually changing the size by multiplying something in the vertexShader. Which is still kind of boring, and this is where you can like start to really like combine all the stuff we've talked about, like let's say glslify.

[00:14:41]
You can just import that and use it, if you wanted to introduce noise into your shader, instead of just using the functions that are built in. So we can actually do like pragma glslify, and then noise = require('glsl-noise/simplex/3d'). We're actually going to use four dimensional noise, so, so far we've used 2D and 3D and we haven't used 4D yet.

[00:15:10]
And that's because instead of using a grid position of uv or grid position of 2D coordinates, now we have xyz. So we just have to use four dimensional noise if we wanna use the xyz and the time together. So I mean if I was just to do something like this, noise that for because it's a four dimensional noise.

[00:15:36]
This a position.xyz and then time, and all of a sudden we're gonna get some cool crazy stuff going on in our screen here.
>> Mathew DesLauriers: We can even do something like this where instead of just adding the noise value the negative one to one value we take normal and this is the surface normal vector of the cube that's coming from three.js.

[00:16:04]
And this is a bit more math stuff, but it's not producing a very nice image yet so we can tweak something we could say,
>> Mathew DesLauriers: Fix this no.
>> Mathew DesLauriers: Anyways.
>> Mathew DesLauriers: This is getting into like weird territory of weird images, it could be kinda cool though. You can start to like really play with these values and create spiky monsters.

[00:16:43]
>> Mathew DesLauriers: But that's as you can see, it's running really efficiently, and it's because we're using a vertexShader. And so all of the points inside of these cubes is moving around with the shader here.