Speaker
Description
Large-scale nuclear and particle physics experiments face a dual preservation challenge: maintaining long-term access to vast data volumes and the tacit scientific knowledge embedded in internal, often private or restricted, collaboration records. Public large language models (LLMs) cannot address this need for private data. To solve this, we developed SciBot, a locally deployed, domain-specific AI assistant within the RHIC Data and Analysis Preservation Program (DAPP), which provides secure natural-language access to preserved RHIC knowledge.
SciBot uses a Retrieval-Augmented Generation (RAG) architecture and the Model Context Protocol (MCP) to integrate local and remote LLMs, ensuring provenance and access control. Crucially, private documentation and communications are processed by locally hosted LLMs, guaranteeing the data remains strictly local and secure even when integrating external LLMs. Production-readiness efforts include performance and scalability testing of inference engines (vLLM, LlamaCpp, Ollama) on various GPUs (A6000, A100, H100) to optimize deployment cost. A key change is the migration from ChromaDB to Qdrant for the vector database, enabling scalable metadata filtering and strict segregation by collaboration level. Federated authentication (OIDC via CILogon) enforces experiment isolation while supporting shared knowledge.
These advances establish SciBot as a secure, performant, and extensible AI-assisted knowledge preservation service, offering a practical blueprint for facilities like the Electron–Ion Collider.