Standardizing & Automating Buckets

[00:00:00]
>> So we know that we have a bucket. So let's go ahead and grab that bucket. And like I said, just copy pasta. Pretty simple. We're gonna paste this in. Then we're gonna start editing it. So we're gonna say, okay, well, we know that we wanna create an s3 bucket, sorry AWS s3.

[00:00:22]
And then what we're gonna do is is we're gonna import AWS from Pulumi AWS, right? And then now that we've got that imported, I don't know why that's doing that, we can start customizing it a bit, right? This is where our opinionation start to really play, right? So because we have a component that we are now creating, technically we can give it options, right?

[00:00:46]
And one of the first options, we're gonna give it, is we're gonna give it a name, a name of the bucket. However, we're not gonna do it here. And you might be like, well, why? It's literally right there? Because we talked about using a standardized simple interface. So we're literally going to create an interface.

[00:01:10]
We're gonna say type FM bucket, and then we're going to create a type for that resource essentially, right? And so, and you can say like options if you wanted to or something like that, I guess args might be a better one here. But the main point is, is that we're going to create and use this as what is now going to be known as our DSL.

[00:01:32]
All right, this is going to be our domain specific language for creating buckets so that developers never, ever touch this. No, never again. Sorry, no, never again. So what do we need, right? Well, we know that we need a name, so let's give it a name, right? So string, but maybe there's some other stuff there that could be interesting, maybe like product, right?

[00:02:01]
Where we can say all right, nope, sorry, not production, product where we can say like, okay, what we'll actually do is we'll use this naming convention where the product is first, right? And then the name is second. So that when we do other things we know okay, well, this is the product, right?

[00:02:20]
And then the name under that product. And if we've wanted to if we really said like, okay, well, buckets are always attached to services. Then we could also say, service, if we wanted to, and what's great about typescript is if we had a different type, say for example, we said that not only do we have an FM bucket, but maybe we have like an FM, something else, right?

[00:02:41]
We could just say FM service. And now because it's typed, we have complete control over that domain specific language and what developers are using, right? So they literally could just say, here's my service, give this bucket to this service. For now we're just gonna use name and I'll actually go with the product one because I really liked that as an example here.

[00:03:04]
So then what we're going to do is, we're going to go here and we're just gonna change this to args. And then we'll say FM bucket args like that. These options are actually Pulumi options. If we look at one of the buckets here, you're gonna notice that it says under the definition of it, right?

[00:03:28]
Which we're looking at here, you're going to notice that it actually has ops with a question mark, and then it says component or custom resource options. This variable is passed through every component. The reason for that is because these are the generic options in Pulumi, that make it so that you can depend on things, you can make things parent of things.

[00:03:53]
So it's always good to have these options available to you, because you can do hierarchical changes if you need to, where you wanna say, this component actually belongs to this parent, or so forth and so on, we'll show you that in a second. But for now, we just wanna type it.

[00:04:10]
We just wanna make sure that it has some type of type. So I'm just gonna say Pulumi.customresource options and that will just make sure that we have that to use there. Now that we've got this, we can actually start talking about how we're going to standardize, how we create buckets a bit further.

[00:04:30]
And the next one is going to be the naming convention, right? What's great about having this in code is, is now I can literally create a convention right here where I can say you know what? Let's say that this is the name actually. And what we're gonna do is is we're going to say args, sorry, args.name and before that, args.product.

[00:04:57]
There's our naming convention, right? And then what I can do is, is instead of having this is, well, I guess it's gonna use that anyways, but basically anywhere where name is used, I can now use that as a standardized name, for any of my other services. And again, remember because this is code, I can make a function that says generate bucket name.

[00:05:20]
And then I can do things like okay, like exactly like we did before, name, product, right? And then if I wanted to, I could then come back down here and just say, all right, well, let's even standardize that and let's say generate bucket name, args name, args product, right?

[00:05:39]
And then you could test that. Now we have testing in our automation so that we can confirm that things are getting named the way that they should be, right? Again, you can do a lot of this stuff in terraform or other things, but when we look at what you can actually do with a programming language when it comes to automation, it's way more powerful.

[00:05:58]
So yeah, so we're gonna create our naming convention here, product name, right? And then we're gonna go down here. And what we're gonna do is we're gonna actually make a small change to this. And we're gonna say that this is just args.name. And you might say, well, why are we following a standardization here?

[00:06:23]
But we're not doing it here, right? So whenever you create a Pulumi resource, there's a couple of identifiers that are created with it. The main two are the URL or URN of the resource and Amazon or whatever. But its also got its own URN, which is called a Pulumi URN.

[00:06:45]
If I go to back here, and I do Pulumi stack There's a way of showing resources, I just can't remember exactly what the command is, that's you. There's no resources, okay, that's why. Let me go to something that actually has resources, so Pulumi Github for example, and then we'll go here and we'll say, secrets.

[00:07:20]
Pulumi stack-U. So when I do this, what we're gonna see is that, it's gonna output a ton of what we see as URNs, right? And that's actually what we're kind of seeing is being started here if we go back to our bucket, right? Package, index, FM bucket and if we go here you'll see URN pulling me did it.

[00:07:46]
So it's totally up to you but when we create the component it gets a standardized name meaning technically inside of that component you can either reuse the same name if you want. But, the reason why I don't do this is, in the off chance in the future say you need two buckets, right?

[00:08:09]
And you use the same name here as you do here, you're gonna have a conflict later, right? So, one of the ways I'd avoid that is I say that the component is what has the standardized name. And then anything else kind of just uses the main arg name passed down to it.

[00:08:27]
But we still use the standardized name in specific places where we want to. The nice thing about that standardized name is that it's like a fully qualified domain. If there's a place in our automation where we want that fully qualified domain say, you create a bucket. Do you want your bucket to just be called food bar?

[00:08:47]
Or would you rather at least be called front end masters food bar, [LAUGH] you know what I mean? And that's what I mean by you can choose when you want to lean on this automation or this standardization and when you don't. Another, solution here would just be to find a completely different name to call the bucket, for now, it's fine, we'll leave it the way it is.

[00:09:06]
What we're gonna do is, we're going to standardize the bucket name in Amazon though, that is something we definitely want to do, right? And so, if we go, I believe, if I'm not mistaken, I believe it's a bucket, yeah. A bucket, I don't know why they don't just use name.

[00:09:27]
But if we wanna actually give the bucket a specific name so that we make sure we're giving it that standardized name when it gets created, then we simply pass that standardized name to the bucket, right? And so again, in Pulumi, we'll just see args.name as the resource. But in Amazon, when it gets created, we'll actually get the full resource name, right?

[00:09:53]
And so if we wanted to make this a little easier, we could say that this is the resource name, right? And then this is just the normal name itself. And here's another nice little fun thing as well, is you'll notice that we have environment here. We can even standardize environments, right?

[00:10:11]
Because we're using Pulumi, we could simply say const stack equals getStack. Stack, yeah, getStack. And what happens now is Pulumi will go out and figure out the name of your stack, and then once it does that, it will bring that as available as an option or as a value to you.

[00:10:36]
So if I really want it, to get super standardized with this, I can even go further and I can say you know what I want my buckets to actually their names, right? To have their own convention as well, right? Well, where I can say like const bucket equals, and what we're gonna do is we're literally gonna take the resource name, and we're just gonna append the stack at the end.

[00:11:03]
So that now when we create buckets in Amazon, we simply use that new standard, and we can have that there as well. Oops, let me call this bucket name. And to make this even simpler, we could also do this. Put the stack here, and instead of having this be the bucket name, well, the name in this case is probably fine because we want them to match.

[00:11:39]
Another thing you probably could do is just get rid of that, don't really need name twice or hitting their one. So, it's up to you. But having the tag for the environment at least would also be super nice, if you're feeling like search to tags and stuff like that.

[00:11:54]
Again, it's totally up to you. If you find yourself in a scenario where you just wanna use the resource name, you can do that. If you wanna use the bucket name with the resource name in the stack, you can do that, right? But the main point of what I'm showing you is just simply, these are the steps to automation that we're trying to take, right?

[00:12:10]
So now that we've created this very simple bucket, the next thing we're gonna do is we're gonna actually add a few more opinions about it, right? So what are those gonna be? Well, first off, we're gonna wanna make sure that we have this bucket completely locked off from the rest of the world, right?

[00:12:28]
And so to do that, we actually wanna add some bucket policies, public access, here we go. Yeah. So what is one thing that you normally wanna make sure is on when you create a bucket? [LAUGH] I'll give you a hint, it's in blue. You wanna block public access at all costs, right?

[00:12:47]
And you know what would be even better if you could walk into your SEC ops team and tell them that that's how it works by default, right? This is the power of DevOps and automation. So what we're gonna do is is we're literally gonna solve that problem simply by saying all right, well tell you what, we will add a policy to every bucket.

[00:13:09]
And we will just make sure that whenever we create a bucket, we also have that access block there too, right? And so we can do this, or we can say args.name again, right? And here's what's neat. We're using variables, right? Pulumi knows about its own types and things like that.

[00:13:29]
So we can even just now use our bucket just like we would with any other automation variable and then get the ID from it, right? And so you'll see here that now our public access block, this block is set up to our bucket, and we block public ACLs, public policy, ignore and so forth and so on, right?

[00:13:52]
This is a little closer to like a standardized bucket, right? We are only telling it the name and product, and out of the box, we get a bucket. It's ACL set to private, as well as making sure that any public access blocks are also removed.