Name: PASTE meeting
Start: 2006-03-21T10:00:00+01:00
End: 2006-03-21T12:00:00+01:00
Location: No location set

10:00 → 10:20

Stripping tools and production 20m

Speakers: Gennady Kuznetsov (Rutherford Appleton Laboratory), Mr Joel Closier (CERN)

a status report of the current activities: Stripping b-incl : second round CERN: 728 jobs 200 running 490 done and some in waiting for data transfer CNAF: 223 jobs in total 61 waiting 57 running 106 finished with 1 errors and waiting in data transfer PIC: 43 jobs All done (some of them in Waiting for data transfer) 2 errors After lot of problem at CNAF, it seems that the situation since Saturday is much better. The waiting for data transfer means that some DATASET has not been trasnfered to one or several sites. Test of Processing DB: On Friday the situation was good. I succeeded to run the whole chain for a stripping job: Creating a production, a trasnformation agent, submitted to LCG and run it successfully. Output check by Patrick and they are OK. During the week-end, Andrei made lot of improvement of performance and functionnality on the processing database which works well. Unfortunetaly, a bug has been introduced that the POOL XML slice which is created is not recognized by the job when the job is running. Problem with the SE syntax see below: So the pool XML catalog is like that when the job is running: <POOLFILECATALOG> <File ID="7ed313d2-ca7d-44d4-9c87-21b692738621"> <physical> <pfn filetype="ROOT_All" name="rfio:/castor/cern.ch/lhcb/DC04/00000718_00001646_5.dst"/> </physical> <logical> <lfn name="/lhcb/production/DC04/v1/DST//00000718_00001646_5.dst"/> </logical> </File> <File ID="a15cf506-218d-4a6e-96b0-a57c32f42be5"> <physical> <pfn filetype="ROOT_All" name="rfio:/castor/cern.ch/lhcb/DC04/00000718_00001432_5.dst"/> </physical> <logical> <lfn name="/lhcb/production/DC04/v1/DST//00000718_00001432_5.dst"/> </logical> </File> <File ID="27232730-9a04-4e2c-a464-a290503560a6"> <physical> <pfn filetype="ROOT_All" name="rfio:/castor/cern.ch/lhcb/DC04/00000718/00000718_00003460_5.dst"/> </physical> <logical> <lfn name="/lhcb/production/DC04/v1/DST//00000718_00003460_5.dst"/> </logical> </File> </POOLFILECATALOG> And I get this error: 2006-03-20 10:05:29: Global[INFO]: Executing step 1 2006-03-20 10:05:29: Global[INFO]: Executing module 1 : FileStage 2006-03-20 10:05:29: Global[INFO]: FileStage application execution 2006-03-20 10:05:29: Global[INFO]: [u'LFN:/lhcb/production/DC04/v1/DST//00000718_00003460_5.dst', u'LFN:/lhcb/production/DC04/v1/DST//00000718_00001646_5.dst', u'LFN:/lhcb/production/DC04/v1/DST//00000718_00001432_5.dst'] 2006-03-20 10:05:29: StorageElement[ERROR]: Unknown storage protocol for SE Uknown debug [] 2006-03-20 10:05:29: Global[INFO]: Missing PFN for LFN /lhcb/production/DC04/v1/DST//00000718_00003460_5.dst 2006-03-20 10:05:29: Global[INFO]: Setting LFN state to problematic 2006-03-20 10:05:33: StorageElement[ERROR]: Unknown storage protocol for SE Uknown debug [] 2006-03-20 10:05:33: Global[INFO]: Missing PFN for LFN /lhcb/production/DC04/v1/DST//00000718_00001646_5.dst 2006-03-20 10:05:33: Global[INFO]: Setting LFN state to problematic 2006-03-20 10:05:33: StorageElement[ERROR]: Unknown storage protocol for SE Uknown debug [] 2006-03-20 10:05:33: Global[INFO]: Missing PFN for LFN /lhcb/production/DC04/v1/DST//00000718_00001432_5.dst 2006-03-20 10:05:33: Global[INFO]: Setting LFN state to problematic 2006-03-20 10:05:33: Global[INFO]: command to send: /pool/grid/lhcb099/830902/globus-tmp.lxb6085.13080.0/WMS_lxb6085_013551_https_3a_2f_2fgdrb04.cern.ch_3a9000_2fZDUvFUt41FBEl1OgNZkKsg/DIRAC/bin/gfal_getlist file_srm.txt 2006-03-20 10:05:33: Global[INFO]: SRM 35584 2006-03-20 10:05:33: Global[INFO]: REQUESTID is

10:20 → 10:40

Updated DC'06 task list 20m

Speaker: Andrei Tsaregorodtsev

Task list send by Nick: A - Functionality High priority - Specification of destination sites or group of sites (Raja) - Job State machine : new definitions (Andrei, Stuart) - Enhanced logic of data files choice for automatic job generation: site exclusion, e.g. CERN for data reconstruction. (Andrei) - Data Manager tools to define: (bulk transfer request; bulk removal request; various data integrity checking and repair operations;tools to deal with files reported as problematic.) (Marianne) - Replacing of the Bookkeeping Replica Tables by the LFC catalog. Update of the Bookkeeping Web pages and command line tools to use the LFC catalog instead of the Bookkeeping Replica Tables. (Marianne, Carmine) - Implementation of the services common framework and migration of all the service to this new framework. (Adria, Ricardo) - Securing services where necessary. (Adria, Ricardo) - Various Processing Database optimizations: multithreaded Processing DB service, optimization of queries done by Transformation Agents, better TA steering ( creation, starting, stopping, removal ) (Andrei, Gennady) Medium priority - LFC Catalog Client talking to multiple LFC instances. Additional logic to treat the Master LFC instance as a final source in case of read-only mirrors failure (Marianne, Juan) - Job priorities rules should be defined and enabled in the job matching mechanism. (Stuart) - Definition and deployment of the USER job Accounting Service (Ricardo) - Implementation of a Message Agent on he basis of the Bookkeeping agent. (Andrei) Low Priority - Job throttling for certain types. A mecanism should be put in place to disallow running more than a defined number of jobs of certain types on certain CE's. Example is stripping jobs which we can not run more than staging pools can accomodate. The numbers and sites should be specified in a special configuration section. (andrei) - Introduction of the methods dealing with ACL manipulation into the LFC client. The help of the LFC developers is needed. (Juan) - Definition and implementation of the Accounting Service interface to query the information necessary for the policy decisions. (Ricardo) B - Deployment (ALL High priority) - Conversion of the SFN protocol replicas inot SRM protocol replicas in the LFC (Marianne) - New packaging of DIRAC (Andrei, Joel) - Releases procedure (Joel) - Deployment of the same DIRAC software version on the USER and PRODUCTION systems. Using on-demand pilot agents for all kinds of jobs. (Gennady, Raja) - Transfer Agent to be ready for deployment on the VO-box (andrew) - Tagging and releasing the stable version of the Production Tools. Deployment of the production instance of the Processing DB service and Job Repository (Joel, Gennady) - Inclusion of the ProcDBCatalog into the production list of catalogs (Andrei) - Intensive tests starting from the preproduction (Joel Gennady) - Definition and test of all the DC06 workflows. (Joel) - Definitions of the deployment procedure for the Request DB service and agents. Monitoring, logging, security for these components. (Andrei, Ricardo) - Definition of the association of the T2 to T1 centers and VO-box parameters in the Configuration Service. Implementation of the site dependant operations in a job needing communication with the VO-box (andrew) - Migration to Linux of the Bookkeeping Service frontend. Putting it in the same framework as other DIRAC services (Carmine)

10:40 → 11:00

DC'06 tasks status 20m

Going through the DC06 task list to update their status

11:50 → 12:10

AOB 20m