Logging & Testing the Query

[00:00:00]
>> So now that we have that, all we need to do really is just kinda like log this out. So, we'll just log out the answer. And then we can look at the sources on which it got that from. So let's do that. So, we'll say console.log. We wanna say answer, and that's gonna be response.choices[0]., [LAUGH] message.content.

[00:00:34]
That was a lot. Okay, so we got that. And then I'll make a new line here, and I'm gonna put, like, what is your source or sources? And then for the sources, I'm just gonna map over, pretty much the same thing here. I'm just gonna map over all of these, except I only really want the metadata.source property from them, and I'll just combine them with a comma.

[00:00:57]
So I'll say, give me all the results that came back from the similarity search cuz those are the ones that we fed, map over all those and grab the metadata.source property from them, and then just join them on a comma separated list, like that. So you'll see, we might get back, like, we'll probably get duplicate sources, it might show YouTube more than once, depending on what number we put here.

[00:01:26]
So in this case, 2, so it might find two chunks in the YouTube transcript to help us answer the question, so it'll put the YouTube thing twice. We're not de-duping these sources right now. We could, but it's showing a source for every chunk, and some of those chunks come from the same source.

[00:01:41]
So it's gonna show it twice, right? It's gonna be like, I got this from YouTube, I got this from YouTube. Versus you probably just wanna know where you just got, you don't care if you got it twice from YouTube. You're just like, show me all the different sources, not how many times you got the same source.

[00:01:53]
So it's gonna show you how many times you got the same source. And that's something we will fix, not the AI. Okay, so, I don't know. Let's give it a try. So, let's run the same question that I had. What is the Xbox warranty? And I don't know if the PDF talks about that or not.

[00:02:13]
Let me see. We got an error here. So, it doesn't exist or I don't have access to that, so I don't know, not that model, I guess. Let's try, GPT, I don't know, let's just do 4, I guess, maybe that's good. Let's try that again. There it is.

[00:02:37]
Yeah, it got it, great. So the answer, the warranty as per the manufacturer's limited hardware warranty agreements. Yeah, it literally got it, and you could see the sources are xbox.pdf, xbox.pdf. So it found two chunks from the similarity search, they both were from the xbox.pdf. It combined both of those chunks together, Here, so those two chunks got added here as context, the original question that I typed in got added here, and then we told it to answer it.

[00:03:04]
So it did, all right? We can do another one. And in this case, you could probably just do one for Xbox questions, since the chunks are good. But for the YouTube one, you definitely need at least two, I would imagine, cuz it can split at any time. So let's ask something about Brian.

[00:03:23]
I don't know, where did Brian work? I don't know if he talked about that. I think he did talk about that. Yeah, he did talk about this. So let's see if it figures it out. And if it doesn't, it should say, I need more context. I don't actually know how to figure that out.

[00:03:40]
So you could see it's taking a long time because every time I run this, it has to go and create the index from scratch every single time. We're not saving the index anywhere. So it's like going to YouTube. It's doing the YouTube thing, it's chunking it, it's doing the PDF thing.

[00:03:53]
Here we go. Brian worked at several places. He started a company called Needle as a senior front-end engineer. Did he? I don't know.
>> Instead of giving it just the raw text, if you basically gave it structured data, like the content plus the metadata that had theoretically the source in it, couldn't you ask the question and then say, hey, specify what source you actually use so it would give you like the particular ones?

[00:04:17]
>> Exactly, so the question was, can we just have the AI tell us what source it use? Yes, I could. The reason I didn't have the AI ask what source it used, because I know what source of used, I gave it the source. So I was like, I'll just skip that and just put the source myself cuz I know you used it because I gave it to you, and save myself some money.

[00:04:36]
But yeah, in other ways, I think from what I've seen in most of the UIs, there's typically a separation from the answer and then the sources itself, they need that to be structured. So I find that if you have the AI respond with the source trying to pull that out from the AI to show it somewhere else in the UI is a lot harder.

[00:05:00]
Whereas if you just have the source here or something like that, it's a lot easier to show. But yes, I guess, like in a chat interface, yeah, you'll probably do that. This isn't a search drop down that show stuff, like I did in Mendable, but it's like a chat interface, like ChatGPT.

[00:05:18]
Yeah, you would probably just have the AI respond with the source in which it got stuff back cuz there's nowhere else that the chat is the UI. So yeah, I think that makes sense.
>> Can we do this with GPT-2? GPT-2 is great for handheld devices where we can keep the data locally and we would need limited Internet access.

[00:05:36]
>> You could do this with GPT-2 as well. I don't know how good it'll be, I think it'll at least be accurate. You might not get good results compared to this. I think the better alternative to keep it in his local would be to use an open source model like Llama or something like that, and running that on your machine.

[00:06:00]
There's tons of really good open-source models out there. So if you go on Hugging Face, there's just so many of them. That's probably the better alternative in my opinion. But let's try to get it to not answer a question, right? So, what is the weather, right? Obviously, it should not know that.

[00:06:20]
You can tell there's ways to speed this up. Like I said, it's loading everything up. You can have it to where it doesn't load up something unless it thinks it needs it, right? So we'll get into that, like adding indexes dynamically, like how to, well, do I need to even index this YouTube video if someone's asking about an Xbox?

[00:06:40]
Why am I even indexing this thing? So it's like, you can have dynamic indexes and indexing on the fly or, there's different stuff. But as you can see right here, I'm sorry, but I cannot provide the weather as the context you've given me does not include that information.

[00:06:52]
And for some reason, it tried to figure that out by looking at the YouTube videos. It just couldn't figure it out. Cool, any other questions on that? It's quite powerful. So if you just imagine replacing these PDF loaders and YouTube loaders with any one of these ridiculous loaders.

[00:07:16]
You can really do QA and whatever you want on any set of data, anything. It's quite impressive.