<p>Facility is exploring purchasing options to determine costs and timelines</p>
<p>Meeting coming up:</p>
<p>  - HEPiX</p>
<p>  - LHCOPN/LHCONE</p>
<p>  - CHEP</p>
<p>Work is progressing to debug SciTags support in dCache</p>
<p>Genesis NOFO may be out Friday</p>
<p>HTC26 planning is underway (Jun 9-12 in Madison)  - Put a calendar hold.  We will have 2 of the 4 days for USATLAS focused meetings and discussion (pre-scrubbing, data-challenges, purchasing, etc)</p>
<p>=============<mark>AI NOTES=</mark>=====================</p>
<h2>Quick recap</h2>
<div>The meeting focused on facility coordination and updates across various teams. Shawn provided updates on HTC26 planning, noting uncertainty about meeting space availability on Monday and discussing potential costs for room rental. Brian reported on OSG LHC activities, including Frontier Squid testing and issues with Google's certificate changes affecting X509 client authentication. The Tier 1 report from Carlos indicated smooth operations with ongoing work on HT condor-related code refactor and new SELinux access controls. Frederick provided updates on Tier 2 operations, including procurement efforts and network equipment pricing. Rui reported on HPC operations and data transfer requests, while Qiulan discussed work on the Jupiter hub authentication workflow. Fengping shared plans for a maintenance window on March 23rd for system updates. Ofer provided updates on continuous integration and operations, including Data25 reprocessing testing and FTS4 testing with CERN. Ilia presented on analytics operations, including work on AI assistants for monitoring and the implementation of new varnish caching setups. The conversation ended with a discussion about FTS4 deployment timeline and testing requirements for the upcoming DC27 data challenge.</div>
<h2>Next steps</h2>
<ul>
<li>Shawn: Ask Janet what it would cost to book a meeting room on Monday for HTC26/US Atlas meeting and report back to the group.</li>
<li>Brian: Send a brief email to Kaushik (and CC Ivan) about the Google certificate changes and their possible impact on Harvester and other services; Kaushik to forward to the Harvester team.</li>
<li>Brian: Contact the CRIC team (cricDevs) to coordinate on topology fetching and authentication changes, and offer a ticket for further discussion.</li>
<li>Horst: Send an email to help@osghc.org to investigate possible impacts of the new certificate usage rules on their XRootD service.</li>
<li>Judith: Double-check with Rob and, if approved, send the post-mortem write-up regarding the recent incident to Ofer and the relevant group.</li>
<li>Ilija: Complete work on restoring user branch analytics data for Tilla within the next week or two, pending final input from Tilla.</li>
<li>Carlos: Send Ilija the contact information for the Dikach group working on AI for operations.</li>
<li>Ivan: Keep Shawn and the group informed about the status and readiness of FTS4 for DC27, especially regarding the timeline for deployment at BNL.</li>
<li>Hiro (and BNL team): Provide a rough estimate (e.g., one week) for the time needed to deploy FTS4 at BNL once it is released, for DC27 planning.</li>
<li>Shawn (and/or relevant planning group): Consider and communicate if DC27 schedule needs to be adjusted based on FTS4 release and deployment timelines.</li>
</ul>
<h2>Summary</h2>
<h3>Meeting Space Planning and NOFO</h3>
<div>The team discussed facility space options for upcoming meetings, including HTC26, HEPICS, LHCOP, LHC1, and CHEP. Shawn reported that while Monday space is unavailable, they could either pay for a room on Monday or meet on Tuesday and Wednesday, with the possibility of using hotel space. Alexei suggested booking a hotel room, and Rafael humorously proposed meeting at the pier with beer and sausage. Shawn mentioned that the Genesis NOFO might be released on Friday, requiring a quick response from multiple organizations.</div>
<h3>US Atlas Meeting and Updates</h3>
<div>The meeting covered several key topics. Alexei discussed uncertainty around whether the upcoming funding opportunity would be a NOFO or a different type of call, with potential for around 100 awards. Kaushik and Shawn discussed the need for a US Atlas meeting, with Shawn still awaiting confirmation from Paolo and Verena about hosting it. Brian provided updates on OSG LHC, including testing of Frontier Squid and recent changes to Google host certificates, which could affect various services. The Tier 1 report, presented by Carlos, highlighted smooth operations, ongoing HT condor-related code refactor, and new strict SELs for NFS access, along with some storage issues that were being addressed.</div>
<h3>Operational Updates and AI Monitoring</h3>
<div>The meeting covered updates on various operational aspects, including procurement, funding profiles, and maintenance schedules. Frederick discussed ongoing procurement efforts and shared a cost-effective Arista network equipment quote. Rui reported on HPC operations, highlighting stable job production and requests for group disk storage. The team discussed updates to the analysis facility, including authentication workflow improvements and potential simplifications to the Jupiter hub frontend. Fengping outlined a scheduled maintenance window for system updates at Chicago. Ofer and Ilia discussed AI monitoring efforts, with Ilia presenting an AI assistant for monitoring various services and communications. The team also addressed concerns about varnish caching at Southwest Tier 2 and discussed the timeline for deploying FTS4 in preparation for the Data Challenge 27. Ivan noted that FTS4 testing is ongoing, with plans to move production transfers to FTS4 once ready, and Shawn emphasized the importance of ensuring FTS4 meets the requirements for DC27.</div>