Quotas & Persistence

[00:00:00]
>> So now to answer the questions about quotas and I promise you that after these we will go to our project and we will start coding. Quotas and persistence, so quota includes. There is one quota for all storages. All the data from local storage indexedDB and cache storage, they go to quotas or they will go to quotas in the future.

[00:00:23]
Today, maybe local storage is kind of separate. Also, when you're using the origin private file system, remember that the Photoshop WebKit API. Well, if you're storing 500 megabytes, it goes to your quota. Also service worker registrations. So when you register a service worker, the service worker itself, the file, and all the dependencies of that service worker are going to restore in your quota.

[00:00:54]
Yeah, typically they are charged confines they're not like in the terms of megabytes, but anyway they are using your quotas, okay? Also Web App Manifests for installed PWAs. So the Manifest and the icon from the Manifest or the data is also been it's part of your quota. Quota does not include cookies, files cached by the browser.

[00:01:20]
So if the browser is caching some of your images and logos and CSS using the standard HTP cache headers, those are not part of your quota. Session storage is not part of your quota. Files is created with the file system on the real file system because they are being stored in the real user file system.

[00:01:39]
So it's not part of the quota that means that using that API you can actually use or download gigabytes terabytes of data if you want, okay? So you don't have a limit. Of course, you need permission from the user. That's the difference, okay? Because there is an explicit permission from the user, yeah, you can bypass any kind of quota.

[00:02:07]
Now the real answer code as per browser, how does it work? Chromium is 60% of the total disk space per origin. So I have 2 terabytes, so 60% of 2 terabytes is what I can store. Actually the algorithm will be more complex, but it doesn't matter to get so deep because Chrome as the whole browser can start up to 80%.

[00:02:39]
But then is 60% per origin. But 80% in total. So it's a mix, okay, but roughly 60% of the total disk space, which is a lot. Remember that you don't need permission from the user to store data. Firefox, today it's 50% of the total disk space with a maximum of 2GB per group of eTLD +1.

[00:03:08]
So on firefox, well I have two terabytes it will be two gigabytes because that's the maximum per group. So my domain and also my sub domains are part of the same group. On Chromium, every origins so every subdomain has its own quota. Make sense? What about Safari? Today it's 1GB per partition.

[00:03:37]
Remember that if your web app doesn't work as an iframe, it's your host, your domain, your origin. So it's 1GB per origin, but this is the only browser that allows the user to bypass that limit with a user's permission. Is the only one. So there is no permission API to do this.

[00:04:04]
At the moment that you you store one bite over your quota, Safari will render a dialog asking the user Hey, do you allow this website to store up to 1.2 gigabytes? And when you go and say one more right over that, another dialog will appear. Do you allow this website you store 1.4, gigabytes, and so on.

[00:04:30]
Okay, make sense? So you can see that safely 1 GB is everywhere. That is a lot to be honest, for a web app, unless you're doing video editing or something like that, okay, for most web apps out there should be good enough. You have any questions?
>> This might be getting into the weeds, feel welcome to be like, yeah, we'll discuss it later or whenever.

[00:04:58]
But for partition origin eTL D+1, conflict. It's like I recall in previous position, one reason we elected not to go with indexedDB was a fear of silent purges of like larger data. Because depending on what the user already had on their computer from other origins.
>> Okay, yeah.

[00:05:20]
>> Is that something we'll discuss later is that we, [CROSSTALK]
>> Will see that later that later is now anyway. [LAUGH] So but before that there are more complexity on top of that. For example, Chrome in incognito mode instead of being 60% of the total space is 5% of the total space, okay?

[00:05:40]
And also when you close incognito mode, everything will be cleared out. That's the idea, right? If Chrome is enabled with a setting that says clear cookies and site data when you close on Windows, your quota goes down immediately to 300 megabytes only. Fortunately, not a lot of users are marking that option anyway, okay?

[00:06:07]
And other browsers in private mode, it depends on the browser, but sometimes you cannot use the API, so the API is not available. Or the storage is ephemeral, so it's going to be removed. So you don't have any quota. So at the end it will be removed. So it's memory only, okay?

[00:06:27]
So it's complicated in that situation, yeah.
>> Clarification, so when the API's are not available, does that mean that they'll throw or an exception?
>> They'll throw an exception.
>> Interesting.
>> Security exception or DOM exception, it depends on the API you're using.
>> Okay.
>> Different exceptions will be formed.

[00:06:51]
Another example, okay, before answering the question is, but this is a quick one. This is a website that I've been demoing for almost 10 years, filldisk.com. When you get into fillthis.com. In fact, it's not even HTTPS. So look how well the website is it's not even HTTPS. So filldisk.com, what is going to try and to do it depends on the browser.

[00:07:23]
And how it works, okay? It will fill your disk with images of cats in local storage, okay? Not even in IndexedDB. And you say, okay, how is this even happening? Okay, a local storage and it will go over your one gigabyte quota, okay? Of course, here you can stop the madness.

[00:07:52]
Hey, so once you already know here, in fact, look at the website, whether it's talking about, IE. And the Firefox is immune. He is Hackery. So this is a very old website, and this is still working on some browsers. I will explain what happened this is trick, okay?

[00:08:11]
But it actually works. So and also not nonsecure will you go over one gigabyte, what do you think? We are close to it. We can wait a couple of seconds while it's the cuts are in my harddrive. Yep it went through over the quarter so let's stop the madness playing what's going on.

[00:08:36]
This website among changing the background and changing the background color, can you see this changing. And also adding cuts to my computer is doing this filldisk.com, right? So using iframes it's opening an iframe to 0.filldisks.com. So we'd save in five megabytes of cuts in local storage here then one.filldisks.com.

[00:09:02]
Another five megabytes of cuts on NBC will iframes that you don't actually see they can be see what I'm not to that filldisk.com. Actually what they have is a Wilker on DNS so star.filldisks.com will actually point to the same server to the same script file. And then they have you can store cats 5 megabytes per infinite subdomains.

[00:09:30]
Because Firefox has this eTLD+1 thing, that's why it was immune.
>> [INAUDIBLE] And they're rating these images, these flat files, to local storage?
>> Yeah, how, Base64. So because they are on local storage, you can only store text strings. So then you take the image, and you convert that into base 64.

[00:09:57]
But this is just to finish the part of quotas, and how tricky can this be. Mostly when you can have I frames and things like that, okay? Makes sense? And because local storage is not actually part of your real quota. That's why partitions are not taking effect in Safari here, because when you have iFrames remember that it's creating partitions.

[00:10:23]
But anyway, it's occupying my hard drive. So you can get to a website that has a hidden code. That it's not showing you, you can not stop the madness. And then it's eating your heart right prespray easily, okay? So the next question has to do with addiction. So because yeah, if I can store gigabytes, terabytes of data, we have a limit, okay?

[00:10:52]
So it sounds scary. That every website that you visit can store up to 1GB without your consent, without your permission. Well, the storage per origin in different browsers can be defined as best effort or persistent, okay? So, on best effort, that is the default state Per origin on every browser.

[00:11:20]
On best effort, the browser can clear the storage on storage pressure. That's when the user is with low storage on the phone, on your computer. It depends on the browser. The final algorithm typically is around 10% of the available space. So if if there is less than 10% of the available space in the harddrive, then the browser will say okay, it's time to start deleting data from Origins, okay?

[00:11:50]
Also after some time of inactivity that's Safari, Safari today, they say, but it's to be honest. This is actually difficult to prove that it's true, okay? I tried to prove that, I couldn't prove that. But officially in their blog they say that after seven days of inactivity, Safari will clear storage of websites without activity.

[00:12:18]
So you go to disneys.com, if you don't come back for seven days, so far it says, it may clear everything that you have. Service worker installation, indexedDB, local storage and everything. That sounds scary, but give me a sec. Also with users intervention, it can be really and the user can go to settings and clear the data, okay?

[00:12:44]
But persistent storage will keep the storage unless user intervention happens, okay? That's what typically we want as a web app developer. We don't want our data to be deleted in the storage pressure. So that's why we have persistent. Persistent storage in our origin will keep all the data unless user intervention happens.

[00:13:13]
So the user is always in control. So if the user goes to settings and clears the data, the data is gone, no matter what. But also see if the device is reset and so on, but that's all. No storage pressure, no inactivity. So, at this point, you have the phase of "Okay, how can I get persistent storage?" Because it doesn't clear the data on storage pressure.

[00:13:40]
So, this is something that we need to request. So first, on iOS and iPadOS, with Safari, it works in best effort. So eviction can happen on a storage pressure after 7 days of inactivity, or if the user goes to Settings, Safari, Clear, okay? With installed PWAs is persistent storage by default, that's good, okay?

[00:14:08]
So if the user is installing your PWA, eviction can only happen if the user goes to settings and clear the data. So that's an advantage for the user when installing the PWA. Firefox, and chromium based browsers, they are also best effort by default and eviction can happen on a Storage Pressure or under settings.

[00:14:32]
If the user goes to settings, okay, when uninstalling the PWA, so if the user has installed PWA. If the user uninstalled the PWA, the user may have the option to delete the data. So this is something that from a UX perspective that Chrome has added. So think about this.

[00:14:51]
You install a PWA. Okay, so when you do that, you see an app, right? So you see an app On Android or an iPhone on Windows or Mac or Linux, it's an app. So think about now you uninstall the app. The user thinks that if you uninstall the app, you delete everything, right?

[00:15:11]
But because the PWA is kind of a browser tab at the end, the data is still in the browser. So because of that when you uninstall a PWA, today you receive a message saying I can show you this in action. For example, this is a PWA Squoosh that I have installed.

[00:15:29]
If I uninstall the app, I can also remove the data from Chrome, because I can. If you uninstall the app, your data will still be there in Chrome.