GridPP Technical Meeting - LFC to DFC migration

Europe/London
Virtual Only

Virtual Only

Alastair Dewhurst (Science and Technology Facilities Council STFC (GB)), Andrew McNab (University of Manchester), David Colling (Imperial College Sci., Tech. & Med. (GB))
Description

Weekly meeting slot for technical topics. We will try and focus on one topic per meeting.  We will announce at the Tuesday Ops meeting if this meeting is going ahead and if so the topic to be discussed.


Current Status:
All new T2K production work is now using the DFC directly.  T2K users have been informed that we need to migrate away from the LFC ASAP.  The current breakdown of the file list by site can be found here:
https://docs.google.com/spreadsheets/d/1qt1G9M0m9PSKTG4OKxgcYV7nAfHWTIOZCxZjfEJrXSo/edit?ts=5bc60678#gid=0


Plan:
1) Starting on the 19th November we will make the LFC for T2K "read only”.
2) Tier-1 will provide an updated list of files in the LFC to Daniela.
3) Daniela will create a GGUS ticket for every site hosting T2K data providing them with a list of current file names (in the LFC) and new file names for the DFC.  
4) Sites will inform Daniela when they have completed the rename.  They will also need to provide the checksum and file size for each entry.
5) As sites complete their rename, the Tier-1 will remove the entries from the LFC.


Actions before the migration can start:
Tier-1:
- Generate a list of files registered under T2K (not T2K.org).  Email list to T2K who will check that nothing is still needed.  Tier-1 will delete any files if necessary.
- Contact all sites who have data in the LFC to ask them if they still have the data we believe is in the LFC.
- Find a way of cleaning up LFC in bulk.
- Tape migration - RAL to provide list of checksums and file sizes to Daniela.

T2K:
- Lukas will send out an email to T2K users to say that the LFC will be made read only from 19th November.  It will be
- Sophie to check the DFC to see if there are large sections that could be cleaned up.
- Lukas to contact KEK people to inform them that for the next data taking run, they won’t be able to use the LFC.  All KEK data can probably be removed as it should be transient, but Lukas will confirm that is the case.
- Lukas having issues with Python via DIRAC.  Lukas will email dirac-support@gridpp.rl.ac.uk

GridPP:
- Catalin will get new T2K credentials.  Daniela to get production credentials. (Sophie will approve)
- Understand how many /t2k.org/t2k.org/t2k.org/ we need in path name.
- Dan to look at recently written files to QMUL and confirm that the behaviour is as expected with Sophie, Daniela and Lukas.
- Sam to write instructions (or send around a link) for DPM sites to rename files in bulk, as well as how to extract the checksum and file size for a list of surls.  Daniela can do this for dCache sites, Dan can do this for Lustre.

 

 

There are minutes attached to this event. Show them.
    • 11:00 11:45
      LFC to DFC migration 45m

      https://docs.google.com/spreadsheets/d/1qt1G9M0m9PSKTG4OKxgcYV7nAfHWTIOZCxZjfEJrXSo/edit?usp=sharing

      Speakers: Alastair Dewhurst (Science and Technology Facilities Council STFC (GB)), Catalin Condurache (Science and Technology Facilities Council STFC (GB)), Daniela Bauer (Imperial College (GB)), Darren Moore (Science and Technology Facilities Council STFC (GB)), Lukas Koch (Science and Technology Facilities Council), Simon Fayer (Imperial College (GB))
    • 11:45 12:00
      AoB 15m