Gaia Grosso: Yeah.
Ke Li: As you can see it.
Gaia Grosso: Okay, now, we can.
Ke Li: Let me quotate it.
Ke Li: Yeah, just one second.
Gaia Grosso: That that's good.
Ke Li: Okay. Thanks.
Gaia Grosso: Alright. So Kay is going to talk about Lms-based physics analysis assistant at best. 3.
Gaia Grosso: And so yeah, just give it away.
Ke Li: Yeah, thanks and thanks for organizing this event. And so I'll talk about what we are doing at S. 3. So we want to use the large language model to build a an analysis assistant we call it Dockside. And I will explain to you on why we have this name later.
Ke Li: okay, so the the goal. The final goal of this project is to use the Lm to build a virtual robot system so which can analyze the high, the high energy physics, experimental data automatically and
Ke Li: so the general ways to use the Lm to build some several agents, and each agency is designed to work on one kind of task
Ke Li: and in general, so they can work together and solve and and do a complex task.
Ke Li: And so we started from the best. We experiment because the analysis is a left hand side, and the analysis is much simpler than that at Sams or Atlas. But in principle we can also easily expand it to other experiments.
Ke Li: And so, first, st I will show you the
Ke Li: reason why we want to the the Om to do it. And yeah, I know. I think all you are familiar with the analysis. So I will skip this part. But just want to tell you that the lots of we we do have lots of human computer interactions when we do the analysis to the physics analysis. And this kind of work. Yeah, in future it can be replaced by using the Lm, I mean you once we have the good, of course.
Ke Li: Because the current. But even now, then, the is we're good at the text or code generation. And in principle we can start to to do something to build some system so it can help us to work on some simple task automatically. And
Ke Li: the key of this project is how we can let the Lm know the or understand the experimental related knowledge. By knowledge I mean how to do the physics analysis, how to call some tools and how to write the dominant codes.
Ke Li: So how to model the analysis on the the whole physics analysis.
Ke Li: So that's the most difficult part for the for for this project, and the current air or current I am is I mean the the precision is not acceptable. But if we want it to do some some real real things. And one of the reason is that they do not understand or do not have such kind of a knowledge.
Ke Li: And so we start to start it from the laptop experiment, and we want to embed the analysis knowledge into the Iom
Ke Li: and here it's a just a short introduction by 3. And and I think most of you are not familiar with this experiment. So this one is experiment like Lepton Collider, and we do have the the highest luminosity around the 4 or 3 G way. And
Ke Li: So this year we have an upgrade, and after this upgrade we expect to have a a 3 times higher luminosity? And so there is a actually a rich physics programs at by 3. So it's different with SMS or Atlas. It's a kind of a tall charm factory, and we have lots of studies on the headrons on the exotic headrons.
Ke Li: And so up to. Now we have more than 300 people working on this experiment, and have published so hundreds of physics analysis, which is good for the analysis modeling, and I will show you why.
Ke Li: okay. So here, this page just show you all or what we want from the Lm. Or how the IM can help us. And because it's good at the text or code generation,
Ke Li: but the the data we have the experimental data, I mean, the the rules for this data is totally different with the language, so we cannot use the Lm directly to analyze the data, but we can use it as a drive a driver, or as a robot to config or to drive the whole workflow.
Ke Li: And by the whole workflow, I mean, I think you're all familiar with it. We just use the
Ke Li: the the current tools. But we use the Lm to write the text because we're talking about the the procedure, the the tools, the workflows here. That's
Ke Li: for the steps between these 2 workflows are kind of a text or code, and this can be generated by the lm, by the dock side. And
Ke Li: so we have from this project 2 years ago, and we built the multiple AI agents system about the large language model and to automate the whole data analysis workflow. And it's similar to the self driving. But of course, I think it's more difficult than the self driving. It's possible, actually, because in few driving in next few years I think that I will have a
Ke Li: a big step forward and for us, the one or the the main thing we need to prepare or we need to do now is to investigate the way to model the workflow or the knowledge embedding so that we can let the I know or understand the the knowledge or the logics between behind the the data analysis.
Ke Li: Okay, so we have this project when the name is Dr. So it's short for Doctor Science and all doctor cyber in Chinese. And I do like this name, because eventually we we do want it to behave like a doctor and to work on the physics analysis and
Ke Li: the current dockside project. It's it's it's built based on the auto gene from framework from the Microsoft day. So it's a a multiple agents framework. And each agent. So we have the dedicated Lm, and we also have a dedicated memory by memory. We are using the Ig to store some internal data.
Ke Li: and and we also
Ke Li: fine tuned. Our own version of Lm name is version 2 is fine tuned on llama 3. And now we are preparing a new version of this.
Ke Li: and in the meantime we also investigating the approach to build a better, for example, to build a new one based on the deep sick using the reinforcement learning.
Ke Li: So in in general, the the it's switchable. And you usually switched. And for most of the task, or for most of the result. I will. I'm going to show later. So the result are based on the Gpt or deepseek.
Ke Li: Okay. So here is the timeline for this project, and at last year at last June. So we have released it. The 1st version of Dockside, and it's kind of a demo only used to test the facility.
Ke Li: So we have a Chatbot. And we can have the internal data as as the Ig vector store. And it's kind of actually capable of doing some very, very small task like writing a small Mc generator configurations.
Ke Li: And now we are upgrading to the washing tool, and for the washing tool we expect it can automate or can do the workflow up to from the user's query to the preliminary. Even the selection part.
Ke Li: And for this part, we yeah. So we we already have some good result, and we expect it to release it at the end of this month. So unfortunately, so this time I cannot show you the the current result
Ke Li: and the
Ke Li: at next few years, and we also have made a plan, and we have a good funding from the some funding agency to work on it. And now we have a 20 people in our working group
Ke Li: to work on it.
Ke Li: Okay? So that's the general timeline. And here I will show you the one of the key part of this project, which is the analysis modeling. So it's different with other modelings. Other machine learning applications at Cms or Atlas. And for this for the dock side, what we want to model is how we analyze or how we do physics, analysis, and the current.
Gaia Grosso: Okay, I think we lost you.
Gaia Grosso: seems like, there is an Internet connection problem.
Gaia Grosso: Start the recording. Yeah, that's great. Okay?
Gaia Grosso: All right. Thank you. Yeah, just please go ahead.
Ke Li: Okay, yeah, thanks. So the experience we we we gain from the our development is that the current Lm is not reliable. So we have to make a new thing to to constraint. What we are doing is to develop a domain specific language. So we check which is kind of a a new language to show how we what's the analysis procedures?
Ke Li: And but the the the bad thing is that we have to do it by our own. But manually, I mean
Ke Li: So the current best 3 has more than 600 physics analysis, and we have to prepare 600 kind of a data set by our own. So it will take some time.
Ke Li: And so this kind of data is can be served as a guide to the Lm. To the dockside, and in this way it can works well. Account works better.
Ke Li: Okay. So page 10 is the Ig, and I think the previous 2 speakers already mentioned, and about it. So I will skip this part.
Ke Li: And for our multi agent system. So we we, for now we developed several agents, and each one agent is designed for one kind of a specific task.
Ke Li: and we have a host to communicate with human, and also communicate with all the other agents, and we have a planner to
Ke Li: decompose the all the complex task into simple tasks.
Ke Li: and we have a coders which can write the C plus plus or python code, and the tester will be used to test to do a unit test and also to execute this this code from Coder. And so that's how this the the multi agents system work.
Ke Li: And so when user will ask a question to the host, so it will sync and will select the correct the next agent.
Ke Li: and pass the necessary message to this agent, and then it will also evaluate the messages from the previous agent.
Ke Li: And now we are trying to test a better definition of the agents and also the better logics, the the interaction logics.
Ke Li: And and we have a simple web ui, and but now it's it's in upgrading. So
Ke Li: so the current status of our doc site is that we are preparing the version 2 of Doc site and is expected to be released at the end of this month, and for this washing we will have a better
Ke Li: I mean, every components are
Ke Li: upgraded, and the main thing is that we have the Dsl to make the result from the Iom more reliable.
Ke Li: And we're also building an evaluation system and the benchmarks.
Ke Li: So yeah, so because for for the new docsite for the washing tool, we don't have any public result. And this time I only show you some result we got one year ago, and this is the one result from the docsite on how we do, how to decompose the complex task into simple tasks
Ke Li: that will skip the details. And this one is how how it can be served as an assistant, so we can ask it question one question, and it can do a search in the Ig and can conclude correctly.
Ke Li: Okay, so here is our experience by developing the the doc site. So the key of this pro, the the keep, all, all. The most difficult part is how we do the knowledge, representation and embedding how we let the I know this kind of knowledge, and the current solution is to interpret the analysis procedures into Dsl manually.
Ke Li: and for the next we will align the Dsl. With the tools and also the the course, and we also try to interpret the analysis procedure as a Markov chain. As you can use the reinforcement learning to build a new.
Ke Li: and of course, in the meanwhile we are also discussing with other groups or companies to see if there is a a better way to do it.
Ke Li: Okay, so let me summarize quickly. And
Ke Li: yeah. So so for the Lm for us. So in addition to a chatbot, so we can also use it to to automate the analysis.
Ke Li: And a demo of this kind of work. The dockside version one looks good. And so that's why we. We put many efforts to develop the dockside version 2, and it will be ready very soon. And for this version, so we can hope it to automate the workflow. So the user can ask or can can freeze the the physics channels
Ke Li: like the hex to Bb or other channels to it. And you, if we want to work on, work it at Cms or Atas and then the dockside will write The- the C plus plus code will submit jobs. We'll write the floating code
Ke Li: and do a very, very simple preliminary selections. And that's what that's is what we will achieve by the end of this month.
Ke Li: And the
Ke Li: yeah, so so. And we also have a a machine learning collaboration at, I have to push the the AI for half studies. And it's very welcome to discuss or collaborate on any of these efforts. Okay, so that's all from assets. Thank you. And sorry again.
Gaia Grosso: That's okay. Thank you again for your talk. It is really interesting. Maybe we have time for one question. If anyone has one
Gaia Grosso: from the audience.
Gaia Grosso: yeah, I see Daniel has one. Please go ahead.
Daniel Murnane: Yeah, thanks very much. This is really, really cool. Very jealous of
Daniel Murnane: how much effort you put in here. The Dsl idea! Do you have some kind of concrete
Daniel Murnane: understanding of how much the Dsl.
Daniel Murnane: Improves versus like unstructured approach.
Ke Li: Yeah, so we are still developing and the the evaluation system. So we don't have the correct numbers, but from the but by so we can see. If we do not have the Dsl, the then the the results. It's it's it's not reliable. It sometimes it can pick up the the wrong agents, and sometimes it can write very, very
Ke Li: bad court, but once we have the the Dsl.
Ke Li: In general, it improves a lot, I I will say, but we do not have the the exactly numbers.
Daniel Murnane: Okay, that sounds promising.
Gaia Grosso: All right. So let's thank Kay again, and we move to the second session of this workshop. So we start with Nikhil.
Gaia Grosso: I think he's connected.