Gaia Grosso: The conscious aware research support.
Gaia Grosso: Okay.
Mariarosaria D'Alfonso: Yes, so.
Gaia Grosso: Started. So yeah, please take it away.
Mariarosaria D'Alfonso: Okay, I'm riosarian from Mit group we work also in Cms
Mariarosaria D'Alfonso: and together with Pietro Kristoff and team, and Matthew Russo from the other part of Mit from the computer science group. We are trying to
Mariarosaria D'Alfonso: set up Archie.
Mariarosaria D'Alfonso: So let me explain you what Archie is so, Archie, stay for
Mariarosaria D'Alfonso: AI augmented research, chat intelligence, and is, the vision is to have it as an open source, end to end framework.
Mariarosaria D'Alfonso: and with the idea to provide the large language model services that access. Also some specific knowledge relevant to specific user case
Mariarosaria D'Alfonso: the agreement of the knowledge is done via rug, as was also discussed in in few stocks.
Mariarosaria D'Alfonso: And the idea is to focus
Mariarosaria D'Alfonso: this service for education operation research purposes.
Mariarosaria D'Alfonso: So let me go a little bit to explain some basic element so first.st
paus: Just me? Or is there an interruption in service? Because I don't see the slides.
Gaia Grosso: Yeah, no. The slides are black right now. Maybe it's a problem of Internet connection.
paus: I think it's better if somebody else actually shows the slides who is on a more reliable network. And then we kind of just say next slide, or something like this.
Mariarosaria D'Alfonso: Yes.
Gaia Grosso: So so I can do it or pause. Do you want to do it for her, as you prefer?
paus: I don't. I mean you. You do it. Why not?
Gaia Grosso: Can do it. Yeah, yeah. Then, let me just quickly open them.
Mariarosaria D'Alfonso: Yes, I don't know why, but okay.
Gaia Grosso: Okay, it's okay. I think it's just the Internet connection. It's not super good, but can just
Gaia Grosso: Yeah, just give me a moment I'll open that.
Gaia Grosso: There it is.
Gaia Grosso: Couldn't
Gaia Grosso: all right.
Gaia Grosso: Can you see the slides now?
Mariarosaria D'Alfonso: Yes, yes, we were here.
Mariarosaria D'Alfonso: So then the idea is to start from a relevant set of documents, informato Pdf. Markdown, HTML. A text file.
Mariarosaria D'Alfonso: and then proceed through the step of chunking, embedding and then building this vector, database
Mariarosaria D'Alfonso: and then when the user ask I mean, ask their query, there is basically the need to retrieve information from those dedicated source and then use it to provide the the final answer.
Mariarosaria D'Alfonso: okay, next slide.
Mariarosaria D'Alfonso: Okay? So then, Archie is compatible also with different Llms.
Mariarosaria D'Alfonso: so from one side there is. Basically open AI,
Mariarosaria D'Alfonso: and on their side, we have all this other open source part like llama hugging face, long chain.
Mariarosaria D'Alfonso: and what we saw so open eye. Of course we can interact with it is ready to use it cannot perform in some area.
Mariarosaria D'Alfonso: But on the other side, having this open source, Llm. Give us flexibility, cost saving.
Mariarosaria D'Alfonso: and also customization and also security control of the information that goes in next slide.
Mariarosaria D'Alfonso: So what we try to do is also interface with different tools that are used for every day on the research
Mariarosaria D'Alfonso: so first, st the help desk and useful ticketing system like clear red mind or jira
Mariarosaria D'Alfonso: various chat, for like slack mattermos discourse and piazza
Mariarosaria D'Alfonso: then some of the documents, for example, beyond the certain Sso. So we have an interface to interact with this.
Mariarosaria D'Alfonso: and also to retrieve information from the web. And, as I said, the sort of the files that are also can be injected in the container, because this is a containerized
Mariarosaria D'Alfonso: basically deployment. Basically, we can inject and read the markdown and the python files.
Mariarosaria D'Alfonso: Okay, next slides.
Mariarosaria D'Alfonso: So these were the basic elements that we try to do. So. Now I will go over kind of
Mariarosaria D'Alfonso: 2 pilot projects, and then it's the next one we are trying to develop now.
Mariarosaria D'Alfonso: So first, st we deployed in the previous academic year for classroom, for example, for general mechanics.
Mariarosaria D'Alfonso: in this case we were able to give both small documents, but also full text size textbook.
Mariarosaria D'Alfonso: And the student and Archie interact in the Piazza Forum.
Mariarosaria D'Alfonso: And basically, Archie is a plugin that help to prepare the answer basically to those questions and actor, basically as a tutor for students and to reduce the load of ta and lecturing and support stuff.
Mariarosaria D'Alfonso: And it's been used with the goal of not only answer frequently asked a question, but also to help with the real Pset and add in the learning exercises.
Mariarosaria D'Alfonso: Now for what we saw, there are a few challenges that need to be addressed in the next iteration for next classes, for example. So one thing is the content. Evolution. Past knowledge is relevant. But of course, like the content evolves, and using the old one can lead to misleading answers
Mariarosaria D'Alfonso: in some case makes sense also to wait responses. Because, for example, if there is an expert answer can be more usable than a general tutorial that was prepared and get it from the web, and also new information that are added year by year can be more relevant to the one of previous year.
Mariarosaria D'Alfonso: And then, of course, you don't want to reveal the full answer to the solutions of the pre set. So it should be a way to avoid the some issues.
Mariarosaria D'Alfonso: And yeah, so those are the things that I've tried to incorporate and control the next version.
Mariarosaria D'Alfonso: So next slide
Mariarosaria D'Alfonso: so we have another project that we successfully which we use at the this, Archie.
Mariarosaria D'Alfonso: So we have analysis facility at Mit, that is for the full physics department.
Mariarosaria D'Alfonso: And we decided that it's very flexible in terms of storage computing software
Mariarosaria D'Alfonso: and so that the the user can utilize basically according to requirement.
Mariarosaria D'Alfonso: And here the lelem helps in 2 ways in one way helps the expert to handle the tickets
Mariarosaria D'Alfonso: that are submitted.
Mariarosaria D'Alfonso: And then the other side is also, from user point of view, is useful because they can directly ask a question how to do one thing, or.
Mariarosaria D'Alfonso: Oh, why doesn't work so next slide?
Mariarosaria D'Alfonso: So let me give you a concrete example. Here I simply asked the
Mariarosaria D'Alfonso: one thing that is basically how I can install my collaboration wider software and the customer library and package to do my own analysis.
Mariarosaria D'Alfonso: And then, Archie, basically answer it in the various way that we provide. So they can be. I mean, what is the pre-stal software? We have the containers, the Cvmfs. And this answer is basically trained on the user guide the Github example and also past us. Submit the tickets.
Mariarosaria D'Alfonso: And then so you can see this at the bottom of the page that you can basically, the user can finally easily the the, basically, the the query for the prompt
Mariarosaria D'Alfonso: next slide.
Mariarosaria D'Alfonso: Now, I compared this one other question in chatgpt versus Archie.
Mariarosaria D'Alfonso: And yet I was asking basically, the most wanted things people want to know.
Mariarosaria D'Alfonso: How I can get an account to to submit. If you are not, for example, mit, you don't have an id so how you do
Mariarosaria D'Alfonso: so the question on the left is from Chagbt
Mariarosaria D'Alfonso: is correct in the sense that you need to get a sponsored account, but it's too vague, because this will ask for sure how I'm going to do. I'm going to require my Ssh key and on so forth.
Mariarosaria D'Alfonso: And then instead, the plot. The picture on the right shows the detail question from Archie
Mariarosaria D'Alfonso: that basically all the steps the user will be able to do. And actually, it can directly
Mariarosaria D'Alfonso: save time. Because basically, you already precompied all all needed.
Mariarosaria D'Alfonso: yeah. So in this case, clearly, Archie is useful in reducing the time, the various follow up question.
Mariarosaria D'Alfonso: Okay? Then, last slides on exactly, Archie.
Mariarosaria D'Alfonso: So this is an easy way to deploy. So 1st you go in the give tab area. You.
Mariarosaria D'Alfonso: you download the code, and then you basically build your image, and then you can configure in this yam file
Mariarosaria D'Alfonso: the few parameters you you want to do. Which way is the Urlm, or which are your documents?
Mariarosaria D'Alfonso: So and so forth. So, for now it runs inference on dedicated hardware. So we have 2 Nvidia Tesla, v. 100
Mariarosaria D'Alfonso: and then what you see in the bottom right is the way we access the various metrics. And we have basically Grafana interfacing this for us.
Mariarosaria D'Alfonso: Yes, next slide. Thank you.
Mariarosaria D'Alfonso: So now, given the expertise and the experience we got from
Mariarosaria D'Alfonso: the classroom and the analysis facility. So we were trying to thinking how we can extend to Cms. So there are 2 areas where this can be useful. So one area is operations. So our group is responsible for tier 0 data management.
Mariarosaria D'Alfonso: So this camera in play tier 0 is where basically the raw data they go out of our experiments. Cms, and then they get reconstructed.
Mariarosaria D'Alfonso: And then in the other completely opposite case, is that we want to use it also, at the end of analysis chain so the the user gets a very precompiled format for analysis. And basically, I can use the standard package like root or python, like environment.
Mariarosaria D'Alfonso: So in the 1st case, in operation case, all the documentation are internal to Cms, so these are not available to Chatgpt.
Mariarosaria D'Alfonso: And so this is clear a case where, if you watch LGBT, basically, the answer just doesn't know.
Mariarosaria D'Alfonso: And this Archie can facilitate onboarding on new members because those are shifters that basically maybe stay there for 2 weeks, and then they go
Mariarosaria D'Alfonso: so this knowledge is to be passed and then can be assistant. For example, for numerous jira tickets that people submit.
Mariarosaria D'Alfonso: or matter most of various organization, for example, the shift itself.
Mariarosaria D'Alfonso: And then also to have it to spot anomaly in the monitoring system.
Mariarosaria D'Alfonso: So next slide.
Mariarosaria D'Alfonso: because I actually, as it was also mentioned in before. So you want to basically agent that beyond generating response can also take a decision and perform a task autonomously.
Mariarosaria D'Alfonso: So this is the direction we are trying to explore. now, maybe something more in the future. When we sort it out.
Mariarosaria D'Alfonso: Okay, next slide.
Mariarosaria D'Alfonso: Yeah. So then let's come to the other use case. And in the end, user analysis of the documentation of our standard package are public
Mariarosaria D'Alfonso: and so you already Chatgpt knows auto stuff, but still we can help to provide context, aware format.
Mariarosaria D'Alfonso: and can be. The Chatbot can provide more specific Cms example or be assistant. For example, often we give tutorials to newcomers, and behind this simple exercise, the students, they are fresh. They ask basically. And they can ask the charge. Now, what the
Mariarosaria D'Alfonso: basically to do, how to address a problem.
Mariarosaria D'Alfonso: And also we want to use it as a analytics. In sense, we want to spot the area where we want to improve.
Mariarosaria D'Alfonso: The the setup.
Mariarosaria D'Alfonso: Okay, next slide.
Mariarosaria D'Alfonso: Okay. These are a lot of in this picture you have here is basically tell you all the package that in principle you need to use to do your end analysis. And also you don't do analysis often on one place. But you do analysis actually, in various facilities that we have in Cms for some
Mariarosaria D'Alfonso: same as others, but also there are others that are being developed and also being developed in slightly different way.
Mariarosaria D'Alfonso: And so then it's true. That's a lot of documents is already available in in the web.
Mariarosaria D'Alfonso: But so we don't need to implement all this but in additional she can use all the user expert chat and various tutorial recipe.
Mariarosaria D'Alfonso: So next slides.
Mariarosaria D'Alfonso: and, for example. Most of communication happen on mattermost is impossible to go retrieve like already, one year of chats. So what I've done here is basically summarizing the area where people ask questions.
Mariarosaria D'Alfonso: and you can really understand that, for example. You can. I mean, it's better. You address the the most recurring, recurrent questions, so that the efficiencies improved.
Mariarosaria D'Alfonso: For this we can for each specific subject we can inject dedicated information.
Mariarosaria D'Alfonso: Answer Directory. This actually matter most
Mariarosaria D'Alfonso: and then mostly what we saw was in the analysis facility that the feedback loop. When is incorporated, it just improved things over time.
Mariarosaria D'Alfonso: and then also find a way to if some document is not complete and some question
Mariarosaria D'Alfonso: gets repeated. That means that this is this existing information is not
Mariarosaria D'Alfonso: fully, and then we need to find a way to announce.
Mariarosaria D'Alfonso: And you want to do this automatically.
Mariarosaria D'Alfonso: So one example in next slides.
Mariarosaria D'Alfonso: So I asked it a simple question
Mariarosaria D'Alfonso: this is actually before injecting the real dedicated material. Let's say
Mariarosaria D'Alfonso: so how I do set up my Kofia is one of the our programs on Lx plus and Lx plus is the one of the facility for computing. That's it.
Mariarosaria D'Alfonso: So the answer you provide the look in the web at the beginning is correct, but it's very generic. It's basically the same as Chatgpt.
Mariarosaria D'Alfonso: But then this question actually was asked in this matter, most channel. And you see that the beginner
Mariarosaria D'Alfonso: user ask it. And then the expert replied.
Mariarosaria D'Alfonso: and it's basically they just said, instead of rebuilding your thing, you just source a file, and everything is much easier. So the goal of Archie will be, for example, in this case, tailor, the question for beginner and expert
Mariarosaria D'Alfonso: so that it addressed better than the audience.
Mariarosaria D'Alfonso: Okay, next slide. So I'm going towards concluding my talk.
Mariarosaria D'Alfonso: So so Archie came, comes with the unique set of advantage.
Mariarosaria D'Alfonso: That's I think I tried to show you that is fully customizable.
Mariarosaria D'Alfonso: Use open source. And for this is also secure.
Mariarosaria D'Alfonso: and then try to interface with many services as you need, and also you can reach it fast.
Mariarosaria D'Alfonso: Okay, next slide.
Mariarosaria D'Alfonso: This is really conclusion. So then I showed that this has been used in the context of education and research, and we are extended to operation.
Mariarosaria D'Alfonso: And the the real key part here is that is a common framework that has been constantly developed and multiple instance that basically teaching each other.
Mariarosaria D'Alfonso: Okay, this was my talk.
Gaia Grosso: Right? Thanks, Maria Rosaria, yeah, we open for for questions from the audience.
Gaia Grosso: So any any question?
Gaia Grosso: maybe I can start with one just related with this last slide about the the educational aspect of the sort of 1st project you presented.
Gaia Grosso: So you so you mentioned students are using this platform as a sort of to substitute some 80. And so how does exactly work like is it just like they are using the platform to get answers to like doubts? Or is this also a platform where, like, through which they try to solve
Gaia Grosso: problems like like exercises, or this kind of things. So to which extent, let's say, the tool is used in the class.
Gaia Grosso: Yeah.
Mariarosaria D'Alfonso: I think it's more on
Mariarosaria D'Alfonso: the part of asking question, and not the platform for really solving them. But Christoph knows better, because he was running those classes. Probably.
paus: Yeah. So there's there's actually 2 different ways that this has been used.
paus: So a
paus: we have in the beginning of the semester made sure that people are aware that they are allowed to use large language models as long as they make sure, when they use it, that they actually note down that they did use this for various things.
paus: Of course, the problem sets are not the entire grade, so we've explained them. It's in their interest not to just use it. And then basically also believe it right? Because well, it might actually be wrong. And then they get into all kinds of problems. And they're not really learning anything.
paus: So so that was number one, number 2 is, we did an exercise in which we let every student ask, kind of like the same question.
paus: and let them interact with the large language model that was actually that just contained the information from the class as the addition.
paus: And it was supposed to then answer what the result? What actually the answer to the question is, and the student were supposed to take that answer and then discuss it in detail in the as a as one of the problems.
paus: Now, in some cases the answer came out to be wrong, and others it came out right, and they were supposed to kind of explain the answer and make sure that they either trust it or not, because we believe
paus: one of the most important things is to be critical and be able to distinguish between whether it's correct or not, what the large language model tells you. And then so that was really the main thing. There were a few other smaller things that we did, but these were the 2 main applications that we have for for Archie.
Gaia Grosso: Alright. No, this is really interesting, because I think it's the only presentation today that talked about the education. That's why I was very curious to hear more details.
Gaia Grosso: And yeah, I guess.
paus: Of course the students try to use it in different ways. It's asked Archie. Should I do my laundry today? And it has some really funny answers
paus: so there was a really an interesting type of interaction with this and it was a positive experience. But there's a lot more that we need to do to make this more useful and make it really something that is
paus: helping the students to learn more. I mean, one of the interesting things is with.
paus: while there's some studies paper published on using large language models in teaching is that the most important factor in learning with large language models. As funny as this might be, is the amount of time that the student spends with the model is pretty much proportional to how much they improve in their performance.
paus: which is, of course, not surprising, because in the end the student has to learn and learning you can't do just by reading a sentence and then say, Oh, yeah, great, that's not enough. You have to actually interact with it. And so this is actually, really, I think, what we have to build in that we built in interaction where there is a back and forth. And the student actually really has to say things. And you read things and then go back and forth and have this type of discussion. So that's
paus: kind of something that we're kind of looking into. Of course.
Gaia Grosso: No, that's really cool.
Gaia Grosso: All right. I don't know if there are other questions for the talk.
Gaia Grosso: I don't see any raise hand.
Gaia Grosso: Okay.
Raghav Kansal: Yeah, maybe I have a quick one. Have you started the the scraping Cms documents yet?
Raghav Kansal: Or what? What would be the plan? There.
Mariarosaria D'Alfonso: No. So we started to to implement this Sso, for example, tier 0. And data management.
Mariarosaria D'Alfonso: that's the word you're asking. Right?
Raghav Kansal: And like, have you started trying to, you know, go through all the Cms twikies and analysis.
Raghav Kansal: yeah. So both Cms documents. Yeah.
Mariarosaria D'Alfonso: Yes, the the more so, for example, depends on the specific application instance, the best
Mariarosaria D'Alfonso: that the documentation is one or the other. For example, there is a lot of Jira tickets for operation stuff. So not really many tweakies.
Mariarosaria D'Alfonso: So this we started to
Mariarosaria D'Alfonso: to do. And this is that was beyond this. So.
Mariarosaria D'Alfonso: but in general yes, the application drives the type of documents and what it it does.
paus: Maybe I can make a quick comment about this. I think Maria Rosaria said it in our talk, but I think just kind of pull it out here again for making that even more clear. Maybe
paus: so when you say Cms. Twiki, the publicly available twikies
paus: Chat Gpt, or Claude, or any of the major models.
paus: They know all of this information. They've scraped the web. They have that information. As long as you give it a detailed enough question, it will give you the correct answer.
paus: I think the key point for for us here is
paus: to really enhance the model, using the data that is really private that you can't get at. So
paus: the specific example, if let's assume you're on shift.
paus: You're running Daq. Shift in Cms. At the at p. 5,
paus: and there is a long logbook for well, the data acquisition, all of the problems that ever happened
paus: when the when, when this entire logbook information has been fed to the model.
paus: then the shifter will be able to say, Oh, I have this problem, and the model will be able to go through all of the documentation, find out what went wrong, try to see what was done in the past, and also
paus: point you directly into the logbook, which were the entries that were interesting to look at, so that the shifter comes in and can actually rely on somebody to talk to. Well, something, whatever it is.
paus: and get actually an answer that is coherent, and has actually the record so that they don't need to look at the logbook for the last 10 days, or whatever, and try to see what were the most important kind of failures, and this, of course, is automatically updated all the time, so that nobody has to take care of it. It's just automatically done. So that is really where I think the the most powerful kind of impact of this of this setup will come about.
Raghav Kansal: No, I totally agree. Yeah, I mean any internal document. I was just wondering what the status was of the scraping like. Do you have a have a database
Raghav Kansal: already. Have you done like one iteration? I wasn't sorry if I missed this in the talk.
paus: Well, I think that the full, I mean, we're not trying to create a database for all possible Cms internal documentation, because there's insanely many. And it's also eventually becomes not useful. Because if you ask for data acquisition problems and you have an information which is about the paper process or the publication kind of policies whatever. There's no connection, and it's not needed. On the other hand, you see, the important thing is
paus: what Maria presented is a framework. It's an open source framework. So if you are interested in kind of a specific thing, you can basically just take this, install it. And you can put the information in, and then you can actually just
paus: run such a model, and then it will all work right.
paus: So it's a framework. Everybody can kind of use it, and just kind of do whatever they want with it.
Mariarosaria D'Alfonso: Yes.
paus: We have a limited amount of people that work on this and they, we have some specific projects, but there can infinitely many projects could be added to this in the end.
Mariarosaria D'Alfonso: Yes. But also, let me add, one more point so we are trying not to build a database of information, because, what we are trying to do is to do a web scraping of various things behind Sso.
Mariarosaria D'Alfonso: So in that way, if the commands also get Updated in the real place. Then you just don't have to download them in your database.
Gaia Grosso: Alright, so it seems like the conversation short is transitioning towards more of an internal Cms. One. So maybe this is like about the right time to kind of wrap up the workshop and and move to the like final roundtable discussion, internal to Cms. For for the Cms members that are still connected.
Gaia Grosso: So yeah, maybe we can just like, thanks everyone at this point for joining workshop and stay tuned for our next, let's say, bite foundation models coming hopefully soon on one of the other themes of conversation. We listed at the very beginning. And yeah, we'll just transition now to the internal roundtable. And I think Raghav is gonna take over here.
Gaia Grosso: Yeah, I just want to. Yeah, thanks. Once again.