Media Session API

[00:00:00]
>> To finish the, the UI part before actually start writing code that executes in the background. See how that works. I mentioned that there is a new recent use case for background execution that's media playing. Okay, so let's see how that works and see it in action in our project.

[00:00:18]
In the case that your web app is playing audio or video, or it seems like it's playing audio or video, okay, there are special cases for background playing. We have two API's picture in picture for video. So in that case you have like some kind of a secondary canvas, okay.

[00:00:39]
So you have maybe the user is minimizing the browser completely but you still have a picture and picture floating window on your operating system. So you have like a canvas where you can still render content, that canvas is not DOM based, so you cannot render HTML, you can just render videos, okay?

[00:01:04]
So it's a video element, but it can be live video, and because it can be live video, you can inject that with images or with whatever you want. Okay I will show you examples of that in a minute, the other API is called the media session API and this is for metadata.

[00:01:24]
So why you're playing video or audio in the background you can send to the OS metadata about what you're playing, but that metadata is bidirectional or that action is bidirectional. You are sending data from your web app to the OS, but also you can listen for actions from the OS.

[00:01:50]
Like the play, the pulse, actions, seek, next, previous. So every OS will render its own UI. And then when the user clicks on one of those actions, your web app will get executed in the background. And it's not just the web app. Today, when you are executing background audio or video, you will be connected not just to the OS.

[00:02:23]
For example here you have Android, here you have Windows, but also just smart watches. So if I'm playing some background audio from a web app from my iPhone, I will see the information in my Apple watch. And the action from the Apple Watch will trigger JavaScript events in your background web app.

[00:02:46]
Nothing makes sense? And that's not happening in the service worker's scope, that's happening in the window scope, in the global scope, okay? And it's not just your smart watch, also Android where supports that. It includes your car. If you are with Android Auto or CarPlay for iOS it will also work.

[00:03:10]
So, the actions from the car even from the wheels buttons will trigger JavaScript your phone and that magically works. So this is a media session API. So it's navigator.mediaSession.metadata. Where you set the current metadata by the largest album on an image, and then you can set different action handlers per action.

[00:03:43]
Examples of the second part, something like this, you set an action handle for play, post, stop, seek backward and forward. That's typically when you have +10, -10 seconds, so it will execute this. And seek to is when the user is actually moving the finger, is dragging, okay? The thumb around the slider.

[00:04:09]
Previoustrack, nexttrack, skipad, in the case you have another and you let the users keep the ad on YouTube. You can do that. And there is like a second version of this API that is currently available on most browsers and you will understand why. There is adding more actions for video callings, so if you're in meet Zoom or Teams meeting.

[00:04:37]
Now you can send that meeting for example to picture and picture and through this API, you will be able to hang up, to mute your microphone, okay. And your transcript will get executed and function with this API, okay? Make sense? So when you are doing this and you're sending audio or video, if the user gets out of your website, minimizing the browser, or going to another tab, you are in the background and you will continue executing.

[00:05:12]
It's not going to dispose your tab.