![Bites of FM4S: [2] LLMs for experiments in fundamental physics](/event/1543967/logo-1303698739.png){kind=logo}
Speaker
Description
Gravitational wave research at the Advanced LIGO observatories integrates complex, interconnected elements of experimental physics, computational simulations, and theoretical astrophysics. However, decades of valuable knowledge remain scattered across unstructured, multi-modal data and fragmented codebases. Efficient dissemination of this knowledge using large language models (LLMs) can significantly accelerate scientific discovery. In this talk, we share experiences from developing MARVEL, a modular, multi-agent research framework designed to provide scientific assistance in highly technical domains. MARVEL leverages open LLMs to ensure data privacy and is designed to be flexible enough to accommodate a broader range of scientific domains. We highlight challenges, including limitations in fine-tuning, pitfalls of naive Retrieval-Augmented Generation (RAG), model hallucinations, context window constraints, and difficulties in processing scientific documents via optical character recognition. To enhance factual accuracy and reasoning capabilities, MARVEL integrates tool usage and leverages test-time computing at the expense of increased latency. Finally, we emphasize the importance of modular workflows and custom benchmarks to adapt to advances in foundational models rapidly.
