Help us make Indico better by taking this survey! Aidez-nous à améliorer Indico en répondant à ce sondage !

19–25 Oct 2024
Europe/Zurich timezone

Xiwu: A basic flexible and learnable LLM for High Energy Physics

TUE 39
22 Oct 2024, 15:18
57m
Exhibition Hall

Exhibition Hall

Poster Track 6 - Collaborative software and maintainability Poster session

Speakers

Ke LIMr Siyang Chen (IHEP, China) Yiyu Zhang (Institute of High Energy Physics) Zhengde Zhang (中国科学院高能物理研究所)

Description

Large Language Models (LLMs) are undergoing a period of rapid updates and changes, with state-of-art model frequently being replaced. WEhen applying LLMs to a specific scientific field it is challenging to acquire unique domain knowledge while keeping th emodel ifself advanced. To address this challenge, a sophisticated large language model system named Xiwu has been developed, allowing switching the most advanced foundation models flexibly and quickly. In this talk, we will discuss one of the best practices of applying LLMs in HEP including some seed fission tools which can collect and clean the HEP dataset quickly, a just-in-time learning system based on vector store technology, and an on-the-fly fine-tuning system. The results show that Xiwu can smoothly switch different models such as LLaMa, Vicuna, chatGLM and Grok-1, and the trained Xiwu model is significantly outperformed the benchmark model on the HEP knowledge in question-and-answering and code generation.

Primary authors

Ke LI Ke Li (University of Washington (US)) Mr Siyang Chen (IHEP, China) Yiyu Zhang (Institute of High Energy Physics) Zhengde Zhang (中国科学院高能物理研究所)

Presentation materials

There are no materials yet.