Managing Container Repositories

[00:00:00]
>> So when we talk about like, okay, cool, I just showed you a whole bunch of code we programmed for a little bit and we looked at automation. What is the point? The point is, we need to create tools and functionality that help us do what we need to quickly, right?

[00:00:22]
Why are we using Pulumi? Because it's cool, no. I mean, it is cool, but no, that's not why we are using it. We are using it so that we can lean on the language to create well-defined interfaces for our resources. We're leaning on it so that we can then also create those resources in a language that gives us a lot of flexibility with for loops, or composing, and things like that, abstraction, right?

[00:00:47]
And another reason why we're doing this is we have that reusability with components where I can essentially say, all right, well, I have a standard for how frontends should look. And anytime I change that standard, every frontend should get that update. It's not about, this frontend should get it or, this frontend shouldn't.

[00:01:06]
It's about a standardization and saying, all right, well, all right, I'm a developer, again, I need a new frontend, this is the way we do it, right, not you go and create it yourself. And yeah, that's really, again, from a service directory perspective kind of how this looks.

[00:01:30]
If we wanted to get more in depth into this, what would we do? We just create more resources and services, right? So we've solved for frontend, right? How would we solve for backend? New file backend.ts. How would we solve for gateways? New file gateway.ts. You could just keep doing this, this is a convention that you can keep leaning on over and over again.

[00:01:57]
So, that if we go, ooh, that's the wrong VIM. If we go to backend, right, go back to frontend, we can again just reuse almost the whole thing, and then just literally do replace bucket. Actually, no, sorry. Replace frontend with backend. And now we're already taking steps towards automating our backends as well.

[00:02:27]
Maybe our backends need buckets, maybe they need some other things. So for example, if we wanted to compare a frontend versus a backend, right, we said that a frontend needs a container, right, container of some sort. So when we talk about automating things, we really wanna talk about automating everything as much as humanly possible, so that the developer has the right expectations when they leave this automation.

[00:02:59]
So in this sense of a backend, what we would wanna do is we would want a new resource probably for maybe an ECR repository, right, maybe something that we could put our images in, right? Now, we talked about how a frontend has buckets, well, a backend is gonna have those same types of resources.

[00:03:19]
And so again, just like we did with the bucket, we can then take all this even if we wanted to, go to ECR repository, paste it, and then just get rid of all the stuff that's not related, right? So let get rid of all this. There we go.

[00:03:38]
And then we can even say ecrName, which is interesting because if you think about it, we're literally just reusing the same component for the most part, but changing some of the names with it. And so we'll change bucket to EcrRepository. Or if we wanted to generalize this, so we weren't vendor locked in, we could actually just say, DockerImageRepo, or DockerRepo or something like that.

[00:04:08]
And this is something I would suggest, don't name things based off of the cloud provider you have, mostly because you're kind of locking yourself in without even realizing it. If you try and keep things named simply what they are, it's easier to move those things around especially. And what I mean by that is say, for example, we're like, all right, you know what, we're gonna use ECR for now.

[00:04:34]
So let me just go grab the ECR thing cool though here. UCR and then repository. Copy this. And then now it's up to you, again, with anything, you can put components and components if you want. For now, I think this is the only thing we really need for the ECR repository anyways, so I'm just gonna use this main component here.

[00:05:14]
And if we do import from Pulumi AWS like such, right? I'll do ECR. Now, we have the ability to create our ECR repository. So get rid of this, again we already know we've got name, so we just do our exact name here to scan on push true. Maybe we don't want to have scan and push enable, maybe we don't wanna have to pay for that extra or something like that.

[00:05:44]
And maybe we wanna have more optimization or more choices around our immutability, right? This is where we can now lean on the DSL. Is it immutable, right? Sure, cool, things like that. We won't need public, right? So we can get rid of that. Yeah, okay, cool. And so now what we'll do is we will say Name, yeah, its name, cool, ecrName like that.

[00:06:25]
Now, I'm not gonna actually do this exactly, can anyone tell me why I wouldn't wanna put the stack name on an ECR repository or basically on a docker image name? So basically, it's because docker has the ability to use tags, right? So if we created the repository like this, then that means we'd actually be doing docker pull my image.dev: latest, right?

[00:06:53]
And you could do that if you wanted to, you could totally do that. A better way of approaching this might simply just be to do my image and then dev, right, and have that as an actual tag. So that's just something to note here, and that's actually what I'm gonna change about this is, sometimes naming conventions work and sometimes they don't.

[00:07:15]
Depending on the software technology you might be using, they may not suggest it to all be in the same word or something like that. And so being aware of that can just help you, also making sure you're using the technology, right? Because if you did it this way, you'd have four repositories, or three, or two, or whatever, for the same source code, right?

[00:07:37]
And when you do a docker push, you really just wanna push to one place and then pull from one place. So yeah, so we're gonna do that. So we're actually gonna get rid of that one and we're gonna say resourceName like this and we don't need stack anymore, right?

[00:07:53]
So we'll save this, get rid of that, and I think that's about that. So now if we go to backend, what do we do? Well, we just grab our front image repo. Right, and we do the same thing we did before. Name: args.Name. Product: args.products. Cool, so now, if I go to index, and somebody was to say, well, hey, Eric, not only do I have a frontend, but I need a backend that supports that frontend.

[00:08:46]
Services, and look at this, do you notice that I'm only importing services in the main file? That's purposeful, because we have a layer of construction or an order of construction when we create these resources. The developers only mess with the components we create, right? We mess with the components inside of the components that they create, all right.

[00:09:10]
And so that's why here developers should only need services, I need a new service of a specific type, right? So now if we were to say, example DevOps course, what's cool is we can actually use the same names even if we wanted to. Now I will say this, if you have resources that will conflict on some level, then of course, you will get that confliction.

[00:09:30]
So be aware of that. But if your goal is just to be able to have the same name for your backend that matches your frontend, you could still do that. So now if we do up. Okay, what's wrong with this? There's something wrong with this, and I've already explained it.

[00:09:54]
What might be missing here? Look at the resources and how they're stacked.
>> Stack things.
>> I miss, what's that?
>> Stack.
>> Yeah, well, not the stack, but what was the other thing I did to make sure that they were grouped together, started with a p?
>> Or the public privacy?

[00:10:18]
What are you talking about?
>> No.
>> It's like, what are you talking about? [LAUGH]
>> Parent.
>> We need this again.
>> I didn't set the parent, so because I didn't set the parent, this is what I mean by the JavaScript of it. JavaScript will just see everything on the same top level and go, cool.

[00:10:33]
And if you don't catch that, and then you hit yes, you're gonna have to recreate everything. So yeah, you really wanna catch how it structures those things so that you do have that good isolation. So now if we go back to our backend, backend, I should see, yep, there's no parent this here.

[00:11:04]
Okay, but apparently, it didn't see that change, that was so weird. Okay, so we'll go here and then we'll just say parent this. And I actually caught it twice, we didn't do it here either. So not only did we do it or we missed it on the top layer, we also missed it on the component inside of it.

[00:11:26]
Now, if we do this, there we go. So we'll see here that I actually got rid of a whole frontend, I don't need it anymore. So it gets rid of the bucket, gets rid of the component as well as the main service component. And then it also says that, well, we need to create a backend, right, and it's gonna have, what's it called, an ECR repository in it.

[00:11:57]
Okay, another thing I wanna kinda point out here is the names. You'll notice that the DevOps course example has literally the exact same name as the frontend, right? The one we already made, that one's getting deleted, but the first one has DevOps core example. Just a guess, can anyone tell me why we can have a backup resource with DevOps example and a frontend resource for DevOps course?

[00:12:28]
No guesses?
>> Is it they're different resources?
>> Yep, that's it, you nailed it. Yeah, components are isolated to their resource types. So, what you can do, just like with Terraform, for example, is you can have a bunch of different resources with the exact same name. And it can spread across, okay, well, this is the backend for this, this is the frontend for this, this is this and this, and you don't have to uniquely name everything.

[00:13:02]
The only time you actually have a conflict is when you have a same resource with the same name. So you can lean on naming convention at a high level to say like, okay, the backend and the frontend have the same name but they use different resources and stuff like that.

[00:13:19]
So I hit this. Another thing to note, Pulumi always creates first. It always creates first because creation normally is the first thing you want to do and then delete something. And so what it'll do is it'll create replacements and then delete those old ones as well. However, if it's a circumstance where it's a conflict, sometimes you have to manually delete it and then run it again because it's trying to create something that already exists.

[00:13:52]
So now, if we go here. What do we have? Well, okay, I have a frontend, I know I have a frontend in my automation so what should I have? I should have a bucket. There it is, devops-course-example-dev, cool. And then if I go to ECR. I should have a repository.

[00:14:19]
And if we go down, keep going. There it is, devops-course-example. So what does this mean? What does this mean? Well, it means that the developers literally only had to add a few lines of code, they can start pushing up images if they need to, they can start putting files and buckets.

[00:14:39]
And it not only can they put files and buckets, but the hosting is already set up. So they can literally put files in buckets and immediately access those buckets over the Internet if they needed to, right? If I was a backend developer and I wanted to start pushing up images for docker, I already have my repository sitting here ready for me, and I can easily just start using my AWS credentials.

[00:15:01]
This is also something we do at my company, where we do everything through AWS policies and things like that. So the main ideas is that we create these resources, and then we have granular permissions that are attached to them, so that if you're a developer, you can modify but not delete.

[00:15:21]
You know what I mean? If you're an admin, then you can delete. And again, because this is such a low level thing, right, if I wanted to make sure that every backend docker image repo got those same permissions, I just have to do it in one place right here, right?

[00:15:38]
And again, when we talk about scale, if I have 300 backends and I wanna make it so that the policy on all those backends have essentially the exact same rights, or permissions, or things like that. This is the power of using automation like this and being able to simply make small little changes and then it goes out everywhere, you know what I mean?