Speaker
Description
The ATLAS detector at CERN and its supporting infrastructure form a highly complex system. It covers numerous interdependent sub-systems and requires collaboration across a team of multi-disciplinary experts. The ATLAS Technical Coordination Expert System provides an interactive description of the technical infrastructure and enhances its understanding. It features tools to assess the impact of interventions and document expert knowledge. However, the detector’s complexity is inevitably reflected in the expert system. This can complicate information retrieval and can diminish the user experience.
This submission presents a proposal to improve the Expert System’s user experience with natural language-based interfaces. A large language model is used to interpret user queries, and return the responses as written texts. This paper discusses the key aspects of the system's architecture and model selection, the process for building a domain-specific knowledge base, and evaluating the improved user experience from an initial pilot. The pilot is based on the results of a survey conducted across a range of users of the tool, that defined the expected interactions with the new system. The pilot interface is evaluated based on factual correctness, task success and task execution time. It is expected that this approach will improve the tool’s efficiency for regular users and ultimately increment its usage and impact within the ATLAS collaboration.
References
ATLAS Technical Coordination Expert System https://doi.org/10.1051/epjconf/201921405035.
Understanding ATLAS infrastructure behaviour with an Expert System https://doi.org/10.1051/epjconf/202125104003.
Graph-based algorithm for the understanding of failures in the ATLAS infrastructure https://dx.doi.org/10.1088/1742-6596/2438/1/012045.
The ATLAS Alarm Helper http://dx.doi.org/10.1051/epjconf/202429502014.
Significance
The work described in this submission is a significant change in the user experience for the ATLAS Technical Coordination Expert System. It is an important step for improving the understanding of the ATLAS infrastructure and, ultimately, the collaboration. The growth of machine learning, particularly large language models, helps reduce the gap between technical descriptions in the expert system and the user needs. As a result, an increase in tool usage is expected, along with a reduction in mistakes caused by misunderstanding the interventions' impact.
| Experiment context, if any | ATLAS |
|---|