Reading from the File System with fs

[00:00:00]
>> Will Sentance: Folks, so we have now set up. [SOUND] We're ready to go to start getting data from out of our file system, our file storage, what we saved earlier. We did this manually. We literally just, I don't know, dragged and dropped this big old file of data that we want to then use JavaScript labels for Node C++ feature that's written in C++ that does have access to our file system.

[00:00:29]
Perfect. What is the JavaScript label? Can anyone spot on here, Zep, what's the JavaScript label we're going to use to give us access to our Node plus C++ built feature that gives us access to the file system?
>> Zep: FS?
>> Will Sentance: FS is the label, FS dot read file is the specific label bill into JavaScript that gives access to Node.

[00:00:53]
Now remember we have to require it in. But it's as good as built in as a label to trigger Node C++ base functionality, all right. Let's get going. By the looks of it, it's gonna take an actual position of a file on our file system. I guess we're gonna rely on Node to know how to go and look into the file system grab that file and start bringing it back here.

[00:01:16]
Andrew line 1. What are we doing?
>> Andrew: D clear a label called Clean tweets. And that stores the entire function of Clean tweets.
>> Will Sentance: Very nicely put exactly we're not gonna say how it works we just know it's like a black box. We just know that if it takes in data it'll return out clean data of bad words.

[00:01:34]
Okay, next line. Matt.
>> Matt: The next line is read Declare Function called use imported tweets.
>> Will Sentance: Use imported tweets, there it is my friends. Excellent, thank you. And now we get triggering Node functionality. Sarah Rose. Where is FS.read file going to have the majority or is important consequences in JavaScript.

[00:02:01]
>> Sarah Rose: No in the Nodes C++.
>> Will Sentance: Exactly. So I'm bringing out my purple pen. My Node purple pen parts in slash tweets dot JSON which is just to be really clear folks, to JavaScript that is just a string of characters. It has no idea what that means. That doesn't mean it follows a path to it.

[00:02:26]
It's got no idea. Node is going to look at it and say, okay, it's asking specifically for this location in the computer's memory, that computer's hard drive. The first input we always pass through, where do we find out about how FS.readfile works, everyone?
>> Speaker 6: Documentation?
>> Will Sentance: The node documentation.

[00:02:42]
And it will tell us, first input is the path, the location of a file on your computer. Second input is the function that we want to have what?
>> [INAUDIBLE]
>> Will Sentance: Autorun, there's the word I wanna hear. Auto, Automatically run. Thank you Mohammed, exactly. Use imported tweets is the function we want Node to auto run.

[00:03:09]
So this here is a command to our dear friend. Node and it's saying take this path, use it. Take this function have it auto run at the right moment. When do you think that function is going to auto run? Sam? When would it make sense to have it?

[00:03:29]
It takes a bit of time to grab that tweet data into Node and JavaScript. So when do we think that function is gonna auto run? When would it make sense to have it auto run, Michael?
>> Michael: When it completes getting the tweets.
>> Will Sentance: Getting tweets, exactly. Getting the JSON data into Node and into JavaScript accent.

[00:03:47]
All right, so we're going to set up FS in Node. There it is Fs, that's just its name for connecting to the file system via live up to what information does it need? It needs this path. There it is and it needs a function to auto run, which is what?

[00:04:12]
Roman, what function is going to auto run?
>> Roman: Use imported tweets.
>> Will Sentance: Use imported tweets thank you, man. Use imported tweets. There they are. I have a feeling the auto-created and inserted data the argument might just be the data from tweets.JSON, that'd be nice, right? Okay, good, setup done.

[00:04:37]
Setup done immediately. It's going to start from Node sending. It's gonna first figure out what this path is. Dot means current folder in which you are running Node. Which happens to be fortunately where we saved tweets.JSON. Slash says look in that folder. Find the tweets.JSON. There it is and make a connection through to it.

[00:05:02]
No does too much detail but just so you know. Unlike for when we speak to the network where we Node and libuv does not handle the actual opening of the socket and the focusing of a thread on waiting inbound message. A thread is the computer's ability to do a task and focus on a single task at once.

[00:05:26]
We are not responsible and in either Node or libuv for having a thread dedicated to awaiting an inbound message. That is handled by the computer's internals, the operating system itself. A thread is just the processing power of the computer to focus on a single task in this case listening for an inbound message.

[00:05:43]
We don't do that in Node or libuv. It's done by the operating system itself. However, because there's too much, this is my understanding, too much variety in how different computers implement. Access to file storage and file system libuv, said we are gonna handle the setting up of a persistent thread that's going to focus on pulling that data into Node.

[00:06:11]
And we're gonna be in charge of that, on any computer operating system you use this is one of those big senior dev questions in node. Which I/O, this is known as I/O, input/output from the computer system. Which I/O in Node sets up a thread dedicated within Node, handled by libuv for doing that task, and which one doesn't.

[00:06:34]
File system access sets up a thread. I/O through a network or a socket rely on the computer to do the focusing on awaiting that inbound message. Okay, all right. Good, so it's true. I do stamp my foot, John told me I stamp my foot when I'm trying to start a new section.

[00:06:52]
Do I always do this? That's very interesting. Good point, John. Excellent, all right. I'm going to do it like I'm doing now. All right, so there you go. Now this we got to wait a while people. It's a big old file. That is a huge file. Time is passing.

[00:07:13]
What is it it's like. It would be 15 seconds. I mean roughly 15 seconds, but it's a long time, while this data is flowing in. 15 seconds later, people. I want you to know, that's not a short time. To save stuff to memory here takes 100 nanoseconds, or a nanosecond, 100 nanoseconds, very, very, very short.

[00:07:37]
That means that 15 seconds is millions, it's orders of magnitude, millions times longer. The time it would take to do something locally in JavaScript. This is like a big, big, big, long time in computer time, long, longer than often going to a data center and bringing data back for another machine that's ready, there available, accessible in RAM on the other machine, is really, really slow task.

[00:08:00]
I just want to really flag that to everybody who's gonna mean before you want to break it down later on. So anyway, income to the data and 15 seconds. Good news. It's done. What functions gonna auto run when the data comes in Zip?
>> Zep: Using point tweet.
>> Will Sentance: Using point tweet.

[00:08:17]
Let's throw it up and let's start executing it. Use imported Tweets. I should have probably done it in purple, but I'll talk about the execution of it in purple. Here it is. Huge execution context for this, cuz I wanna do a bunch of stuff in it. So here it is, the execution context for it to run its code.

[00:08:37]
>> Will Sentance: Okay, into it we go. What's the auto-created and inserted data? Folk, it's that JSON formatted stringified version of an object. So it's gonna look something like this, JSON data you have to put double quotes around every damn thing. So Toneiger. Those who are regulars using it know that if you get one double quote out either the whole thing gets corrupted.

[00:09:02]
You gotta figure out where the error is. By the way the helpful thing for that is there's a bunch of online tools that will help you spot your JSON error. There's always one, you try to edit it manually you move the comma in the wrong place and that is infamously a very very and the pass function is going to turn it into an object in here isn't like some sophisticated parts that can go past finally go.

[00:09:25]
They missed a quote that. No problem. I'll figure it out. Here is a simple simple function that's going to do only one thing which is convert perfectly formatted JSON data into a job as an object. That's all it can do. It's not like a sophisticated algorithm for figuring out your errors.

[00:09:40]
Okay, ao anyway the date comes in the first tweet is always hello. There we go. And then there'll be another one tweet. So forth. Huge it's 1.5 billion characters long long long long. It's like tons of tweets by the way still nowhere near all the tweets but like a lot of tweets.

[00:09:59]
Okay, this is gonna be auto inserted into here. Does it get a name? Not by Node, do we give it a name?
>> Speaker 10: Yeah.
>> Will Sentance: Yes, yeah. There you go. We give it a name. Two auto inserted inputs. Damn it. Two was inserted. This is the second one.

[00:10:16]
Was the first auto inserted input? This is called the error first pattern in Node. A lot of Node features, their auto triggered function, the auto run function will take two inputs. The first one, I don't know, damn, the first one will represent any error data. Let's put it up here above so it's really clear.

[00:10:38]
The first one will also represent any error data. The second one, with the actual data we care about. It's there's no-error data, this state is clean, it's fine. No-error data, the first one would default to?
>> Speaker 11: Undefined?
>> Will Sentance: Undefined is a really reasonable thing to think. Actually, here, it defaults to?

[00:10:52]
>> Speaker 12: Initialized data.
>> Speaker 13: No.
>> Will Sentance: No. It's spawned from, Virginia, there it is, null. So, the first input will be null. The second input will be all our JSON stringified data, okay. Let's go into the execution context, let me try to give enough space. Yeah, [SOUND], probably about here.

[00:11:16]
Okay, in the local memory, this is a more complex function than I'd like, but I want to show you something.