Choose timezone
Your profile timezone:
DOMA / ACCESS Meeting
- WLCG DOMA-ACCESS Coordinators:
- attended the last HSF Analysis Working Group, where the planning for 2020 was discussed (https://indico.cern.ch/event/880410/)
- agreed to collaborate to understand the future of analysis models and analysis infrastructures
- a joint workshop will be organised (date TBD, late February or early March) where info from computing experts, infrastructure experts, and analysis experts in the experiments will be collected
- David Smith - Integrity checking data in XCache, status report:
- the goal is to add integrity checking to XCache content
- xrootd protocol is being extended: kXR_pgread and kXR_pgwrite (requesting r/w data with checksum attached)
- new protocol ‘spec’ for new request types mandate: CRC32C attached to fixed 4KiB sized (expect for last) blocks of data, where the CRC covers block data
- backwards compatibility (more on slides)
- XCache: caches acts as clients when fetching data from origin, and expecting to interact with various origins (supporting or not kXR_pgread)
- idea to fetch data with the per page CRC32C values, and store (separately) the cache-block data and all the CRC32C values
- if the client is using kXR_pgread, CRC32C values are sent along with data
- if CRC is present and valid, different implications depending on where the CRC was generated and if the creator of the data was involved (more on slides)
- goal for XCache is to: avoid transmission errors on receiving from origin by using new protocol requests which include CRCs on data; allow for check of data (keeping the page level CRCs as well); check data on fetching from the cache-store (using CRCs); prepare the cache layer for integrity check failure from OSS, to be resolved by re-fetching the involved cache-block from origin
- TO DO: extend the OSS to handle CRC32C checksums; dispatching the kXR_pgread from protocol handling to SFS and on to OSS; client (XrdCl) support for kXR_pgread; changes in the cache for checksum awareness and error recovery; monitoring and reporting about errors and corrections
- Riccardo Di Maria - CERN Puppet-ised XCache & MockData:
- the goal is to acquire knowledge on the behaviour of an XCache using MockData (from D. Smith https://indico.cern.ch/event/866486/)
- monitoring comes with CERN Puppet setup
- CINFO files are used to collect useful XCache data info
- data are visualised through monit-kibana.cern.ch and monit-grafana.cern.ch
- multiple-layer XCache
- MockData as MockStorage, load-generator (client), and coordinator
- TO DO: integration with other storages within ESCAPE DataLake, investigate horizontal scaling with several stages, load balancing, etc; investigate other protocols such as http; stress test using HammerCloud (setup HC jobs from existing analysis functional tests and create specific HC test to stress the storage)
- MockData Live-Demo
There are minutes attached to this event.
Show them.
