Live Example: Advanced Topics

[00:00:01]
>> Speaker 1: What would you do to this one, you have that rotation of the whole scene set up to loop perfectly. What would you need to do to-
>> Mathew DesLauriers: It's really complicated, so there's something that I've, that's an issue that I'm struggling with sometimes. And if you have, okay I think it's if you have 2D noise there's a way of using multiple noise things so that it loops perfectly.

[00:00:30]
But it's a big thing that I haven't yet solved properly in all of my work. So let's say I was to try and do something like that.
>> Mathew DesLauriers: I mean maybe here you could say, instead of using time, you could use the playhead value times 3.14, which is pi.

[00:00:54]
>> Mathew DesLauriers: But you'll probably sometimes see a jump. And I think there's definitely ways of doing it, but I think you'd have to look into it a bit. So I'm just gonna replace time with the playhead everywhere.
>> Speaker 1: Is the noise that's the tricky part?
>> Mathew DesLauriers: Yeah, the main tricky part is if you need.

[00:01:19]
The common way of doing it is let's say you have 2D noise like the image that we had before, this gradient-looking image, and you wanted it to be seamless, so that it's a seamless image. So that when it gets to one end, it's perfectly gonna loop and go to the next end, one approach to doing that is to sample the noise from a higher dimension.

[00:01:40]
So you use, instead of using 2D noise here, which is what we're using here, you'd use 3D noise and that third coordinate would help you create this seamlessness to it. And so that works if you're just using image and you want seamless images. But when you get into positions that are x, y, z, then you have to use 4D noise.

[00:02:01]
And if you have a 4D noise function already and you wanna make it seamless, then you might have to use five dimensional noise. And I haven't used five dimensional noise yet. That sounds a bit advanced. But, I mean people have done that, and then there's other approaches as well.

[00:02:17]
One approach is instead of always using a grid like u and v, you can imagine a virtual circle that's going around the noise map and it loops sometimes perfectly. And then if you wanted to extend that into 3D, then you can imagine instead of a circle, it's actually para-metrically mapped onto a taurus.

[00:02:42]
It starts to get more complicated, but yeah. Basically the answer is, it's complicated. What's up, Mark?
>> Mark: How do you determine when you need just 3GS plus pixel shaders, or when you need to go further into vertex shaders.
>> Mathew DesLauriers: Depends on what you want to achieve. So if you wanted just to tweak the surface textures and the surface colors of your mesh.

[00:03:10]
So let's say you had a cube and you wanted instead of it to be like a red color. You wanted it to have like a funky gradient to it, or you want it to have a funky, some sort of texture or pattern. Or even if you wanted it to make it look a bit more like a certain metal, or a certain material in the real world?

[00:03:29]
That's when you'd really be working with pixel shaders, and fragment shaders. Whereas if you wanted to manipulate the position of the cube, so if you wanted to make it so that a cube morphs into a sphere. Or if you wanted to make it so that the cube is exploding into many triangles like confetti, that's manipulating the vertex information of the cube.

[00:03:49]
So it's the difference between the surface of the mesh, versus the actual topology or the makeup or the construct of the mesh. So actually a really good thing to test, if you're if you're getting to this point and you've got a vertex shader in a fragment shader working together.

[00:04:07]
The coolest and easiest thing you can start to do with vertex shaders is to use a sphere instead of using a cube. So I'm just gonna change this geometry instead of box geometry. I'm gonna change it to sphere geometry. And sphere geometry takes in a coordinate that is the radius of the sphere, and then it takes in how dense the sphere is in terms of the sub division or the segments.

[00:04:30]
And maybe what I'll do is I'll just use one mesh just to visualize it.
>> Mathew DesLauriers: And we'll just put it in the center just so you can see what's going on. And so,
>> Mathew DesLauriers: Here we have our simple set of shaders. We have a simple vertex shader that passes along the UV to the fragment shader, we have a position that's not being affected.

[00:04:55]
And then here, our fragment shooter's doing some weird thing with color and the uv map. Can just set the color to just be normal. But then what we're going to do is, we're going to manipulate the vertices of the sphere here. So we're gonna try and do pos, which is the position, +=noise{vec4(pos.xyz, playhead)}.

[00:05:20]
And you'll notice that everything is being offset in one direction or the other, like diagonally. What we want to achieve is more of this like, weird, bumpy, uniformly bumpy sphere. And the reason is because we're just doing position += to some value between -1 and 1. So it's all being scaled along the same sort of direction.

[00:05:39]
So instead of doing that, we're gonna do normal * noise, and now it's gonna use the surface normal, which is a vector pointing outwards from the surface of the sphere.
>> Mathew DesLauriers: And one more thing I'm gonna do here,
>> Mathew DesLauriers: Let's see.
>> Mathew DesLauriers: There we go, so when we change the pixel shader a bit, we can visualize our sphere a bit more.

[00:06:10]
And now we have this kind of cool, I don't know, flowery type shape. And we can play with the noise function, we can give more frequency, make it really spiky. Or we can make it really less frequent and make it more like bulbousy and more soft and rounded and stuff like that.

[00:06:29]
Something else you can really think about here is, using noise functions you can layer them up to create,
>> Mathew DesLauriers: Very, much more organic looking values. Cuz right now, so far with noise, we're always seeing it either being really spiky or not very spiky. And so what you can do is, you can do multiple layers of noise.

[00:06:48]
You can take this layer of noise, and make it really soft, and really subtle, by multiplying the whole thing by 0.1,
>> Mathew DesLauriers: And then there. Now, we'll see, it's a bit more like a planet surface, and that's cuz we just scaled down those noise values a little bit.

[00:07:07]
And just by multiplying that by 0.1 or 0.05 and tweaking this frequency, we can start to get a nice edge to it. And then we can layer things up by copy-pasting, doing again the same thing, but changing these values around a little bit. And so, making it so that we introduce a bit more of these hills and valleys.

[00:07:28]
So maybe using a low frequency like one or two, and a higher value here. This value would be considered amplitude. So if you're talking about wave signals, and sound waves, or electricity or something like that, this would be amplitude, this would be frequency. And just by doing this and by layering up multiple noise offsets, we can introduce these kinda interesting planet like features.

[00:07:56]
And maybe you don't always wanna use the playhead. Leave that at zero.
>> Mathew DesLauriers: And now all of a sudden we're sort of creating this look and feel of like a sort of planet almost. Let's just try one thing here, and you can combine this with actually a different fragment shader and all of a sudden your surface could also look noisy and bumpy in the same technique here, just by copy pasting our noise stuff.

[00:08:35]
Except instead of using 4D noise in our fragment shader, which is just two dimensions of information, x and y we can use three dimension noise. So that might look like this where we have,
>> Mathew DesLauriers: Let's say we have a color.
>> Mathew DesLauriers: That didn't work for some reason. It didn't work because I need to wrap it with glsify, and that will allow us to import modules.

[00:09:03]
If we don't wrap something with glsify, then it's just a regular GLSL string.
>> Mathew DesLauriers: And here we can, let's say, increase the color offset by some value. And this value can be based on the noise function.
>> Mathew DesLauriers: And we'll just do this and then that'll be that.
>> Mathew DesLauriers: So we do uniform float playhead; taking in the playhead value from the code.

[00:09:36]
And we run into an error and that's probably gonna give us a very specific line number. And I can already tell what it is, but I'll just open this up. And you'll have to sort of, as you're working with shaders, you'll have to get familiar with some of this horrible debugging experience.

[00:09:53]
Just like looking through lots of console logs, but the problem here is that noise returns a float. It doesn't return a vec3. And so your types have to match.
>> Mathew DesLauriers: And now we're starting to get something weird. I can tweak this. Just play with the numbers and change them then all of a sudden we can start to get a bit more diversity maybe in the surface of the mesh.

[00:10:21]
Just using these different various functions.
>> Mathew DesLauriers: Yeah.
>> Mark: Is it possible to write your own vertex shader to go along with the shader that the standard mesh, I can't remember what the standard one was called. But can you write your own vertex shader, but then get the lighting and everything back from your JS?

[00:10:51]
It seems like we're sort of coming up on all that.
>> Mathew DesLauriers: Yeah, yeah, yeah, so that's a problem that a lot of people run into. And there's not a super solid solution, it starts to get a bit more complicated. Basically,
>> Mathew DesLauriers: I think there is a way to,

[00:11:10]
>> Mathew DesLauriers: You can do include or something like that, and three.js has this special syntax that is a little bit like glsfi, but theirs is more focused on their own engine. Yeah, it's complicated. There's a way of doing it, and what I've done in the past, is just extending their mesh basic materials.

[00:11:38]
So if you search custom mesh standard material with glslify. This shows an example of how you can do that. You have to extend their mesh standard material. This is using old school ES5 syntax but you can use class extends mesh standard material. But there's a bit of like a setup required.

[00:12:05]
Then the annoying thing is you can't just write it from scratch. You actually have to copy and paste all of their code and then just like insert your own stuff in between. So that's definitely possible, and a lot of people are doing that, and there's a lot of people that are trying to find different ways of making it easier.

[00:12:23]
And hopefully in the future three.js will make it much easier to do that kind of stuff. But right now it's a bit messy and it's not super easy. But it's definitely possible if you're keen.