CTA deployment meeting

Monday 25 Jan 2021, 14:00 → 16:00 Europe/Zurich

600/R-001 (CERN)

Michael Davis (CERN)

Getting LHCb into Production

• Next step is to follow up at IT/LHCb meeting on Wednesday

Getting PUBLIC into Production

CAST

• Everything is ready for final test, Vova has given Marios the go-ahead
• Tapes will be migrated on 26/01/2021, after repack is finished
• Everything is green for going into production on Wednesday

AMS

• In December Baosong tried to use FTS to transfer files. The files were archived but there was an error. To be followed up at SME meeting.

NA61/SHINE

• Vova has asked Luca and Eddie to assist NA61 in setting up the storage at the their DAQ endpoint.

Repack of public_user

• Around 200 tapes to go = 20 days approx. (~15 February). Should be done in time for NA62 migration on 22 Feb.

CTA/Max Drives Allowed

• Cédric will add MaxDrivesAllowed for archival and for retrieval as parameters per VO.
• MaxDrivesAllowed will be removed from the archive and retrieve mount rules.
• Setting max. no. of partial tapes per tapepool is also used to limit the number of similtaneous writes (as well as limiting spread of data across multiple tapes). This parameter should be equal to or less than the MaxDrivesAllowed for archival per VO.
• See ticket #948 for details.

Monitoring

• David will modify the RunDeck job that populates InfluxDB from the CTA statistics table, so that there is only one row per day in InfluxDB (aggregated from one row per hour in the statistics table).
• The total for all VOs should not be a row in InfluxDB, it should be aggregated on the fly when the data is plotted in Grafana.
• Julien will create a ticket for the above.

Pre-GDB and GDB Meetings

• Oliver will present at the pre-GDB meeting (9/2/2021) on the CTA roadmap. Main points: migration from CASTOR, archival back-end for CASTOR backup use case, removing dependency on Oracle.
• Michael will present at the GDB on CTA Migration, but this is postponed to the March meeting (10/3/2021). By then we expect LHCb will have been migrated.

AOB

• Eric can update KB0000486 to replace CASTOR with CTA for new requests for archival storage.

14:00 → 14:10 Getting LHCb into Production

• See Putting EOSCTALHCB into Production (CodiMD)
• See #203 Migrate LHCb from CASTOR to CTA
• Maria's Planning tests for LHCB Storage Run-3
• Networking for LHCb DAQ has been installed, see RQF1724886.

Schedule

• w/c 25 Jan: continue HTTP integration and testing (FTS multi-hop with QoS change+transfer, orchestrated by Dirac). See #209 Test HTTP TPC to CTA for archival workflow
• Wed 27 Jan: IT/LHCb Coordination Meeting: finalise schedule for DAQ test/T1 transfer tests/migration date
• SARA: tested migration to new dCache, should be ready by end of Jan
• IN2P3: needs to fix some dCache config issues, should be ready by end of Jan
• (Feb) Publish OTGs
• CASTOR LHCb to recall-only mode
• CASTOR LHCb disabled
• Migrate to CTA
• EOSCTA LHCb in production
• (Mar?) Data flow tests

14:10 → 14:20 Getting PUBLIC into Production

• Migration of SMEs from CASTOR to CTA, see Putting EOSCTAPUBLIC into Production
• Schedule for migrating CASTOR PUBLIC experiments to CTA

This Week: Migrate CAST

• CAST tests and migration to CTA, see #184 CAST tests on EOSCTAPUBLIC PPS and #198 CAST issues transferring to EOSCTAPUBLICPPS
• Repack and migrate CAST tapes
• Final test on EOSCTAPUBLIC
• In production 27/1/2021

Ongoing Tests (Vova)

• AMS: waiting for Baosong
• COMPASS: Recall testing after the spinner space is configured.
• DUNE: see #213
• NA61/SHINE: will they use DTO+FTS for their DAQ? Progress on integration work?
• NA62: Recall tests completed. Re-test with spinner space, see #72.
• n_TOF: test offline workflow with spinner space

TO DO

• Finish repack of public_user
• "ALICE-like" spinner space on EOS PUBLIC PPS, to test with NA62 and n_TOF (#161)
• Knowledge Base article for users to access files in CTA, see #214 Test and document workflow for retrieving user data from CTA
• Migrate data from legacy experiments (Aleph, Chorus, Delphi, Nomad, Opal, ...).
• Bartek (NA61) is following up on issue of experiment data in CASTOR /user part of the namespace

Schedule

• 22 Feb: migrate NA62 to CTA

14:20 → 14:30 CTA Status Update

• CTA 3.1-14 deployed
• Next: CTA 4.0-1 with v4.0 schema update. Main features of CTA v4.0 are: superseding superseded (#922), tape lifecycle (#186, #943), max mounts per VO.
• #883 Tape verification in CTA, done and will be merged for v4.0-1
• #959 Cedric Caffy's departure preparation

14:30 → 14:40 Mount Policy and MaxDrivesAllowed

• See #194 Put a limit on allocated drives per VO, #948 Move the max drives allowed parameter from the mount policy
• TO BE READ BEFORE THE MEETING: Codi document written by Cedric

14:40 → 14:50 Monitoring

• Review of what work David should prioritise this week

14:50 → 15:00 Pre-GDB and GDB Meetings

• Pre-GDB on Tape Service Evolution (9/02/2021)
• GDB (10/02/2021): have requested a presentation on the CASTOR to CTA migration
• Note: Michael is on holiday that week

15:00 → 15:05 AOB

• Update of KB0000486: should new requests for archival storage go directly to CTA?