PASTE meeting

→ Europe/Zurich
2-R-014 (CERN)

2-R-014

CERN

Andrei Tsaregorodtsev
Description
CERN, bat 2 R-014 To join the telephone conference, click here: https://audioconf.cern.ch/call/0165202 To dial in to the conference: a. Dial +41 22 7676000 b. Enter access code 0165202
Production and PASTE Meeting, 25th November 2008, CERN Chair: A.Tsaregorodtsev Minutes: S. Paterson CERN: A.Casajus, P.Charpentier, J.Closier, Z.Mathe, S.Paterson, R.Santinelli, M.Sapunov, A.C.Smith, A.Tsaregorodtsev, A.Zhelezov Phone: R.Graciani Apologies: 0. Ongoing Action List (07) - 27/11/07 - Zoltan, Elisa - Ongoing new bookkeeping system development. Migration plan (57) - 24/06/08 - Vladimir,Roberto - Automatic publishing of BDII / GOCDB information in the CS using GOCDB site names and provide a summary of recent changes (83) - 01/09/08 - Stuart,Roberto - To perform glexec tests (90) - 09/09/08 - Joel - Use new ProductionOutputData JDL parameter in modules instead of creating LFNs (93) - 16/09/08 - Andrew - Improve RAW data monitoring tools and status of files in online runDB (94) - 23/09/08 - Roberto, Marcos - Using SystemLogging service for dynamic services status monitoring (95) - 23/09/08 - Roberto - SLS and SAM sensors development, upgrade SE tests with new DMS unit test (96) - 23/09/08 - Roberto - Update Dashboard SAM tests visualization (106) - 30/09/08 - Ricardo - Produce twiki documentation for dirac-install (107) - 30/09/08 - Ricardo,Philippe,Joel,Hubert,MarcoC - Produce proposal for how to perform standardised installation using install_project/CMT (109) - 14/09/08 - Matvey,Andrew - Expose DataLogging interface on the web page (111) - 14/09/08 - Philippe,Alexei,Andrei - Productions with file merging and associated data distribution implementation (113) - 21/10/08 - Ricardo, Adri - Document the CS procedures for both the mechanism to define authorization rules in DIRAC and also the current group properties (115) - 21/10/08 - Vladimir,Matvey - Update SAM Agent to monitor and manage submitted jobs and publish results for faulty CEs to the SAM DB (116) - 18/11/08 - Ricardo - Circulate a report on the recent Gauss CPU studies, steer the CPU related activities in DIRAC (117) - 18/11/08 - Ricardo - Add a site attribute to reflect whether generic pilots are accepted there, update the Director (118) - 25/11/08 - Roberto - Provide script to demonstrate GFAL thread safety issue and submit a Savannah bug Completed Actions: Done - (82) - 01/09/08 - Andrei/Adri - to provide information from Accounting and WMS to allow GridMap for LHCb VO T1 activity Done - (118) - 19/11/08 - Joel - Request increased swap space for volhcb09 and announce when available 1. Action Updates - Action(07) - Prototype Bookkeeping To be discussed later in today's agenda. - Action(57) - Automatic publishing of BDII / GOCDB information in the CS Roberto will send a summary mail with a report on this. - Action(82) - Provide accounting information to allow GridMap for LHCb VO T1 activity Andrei commented that this point will be ongoing, after the meeting last week the implementation is agreed and there will be some final tuning of the metrics associated with site behaviour. - Action(83) - gLexec tests Stuart mentioned that a simple script had been offered to the people certifying SCAS but no response was returned. Roberto commented that the request for supporting the pilot role was refreshed at the sites but not all the Tier-1s are doing this yet, it was suggested to reiterate this with a broadcast email. - Action(90) - Production Output Data parameter Ongoing. - Action(93) - Improve RAW data monitoring tools and status of files in online runDB Ongoing. - Action(94),(95),(96) - Service Monitoring Andrew suggested last week to use the Data Management FTS monitor as a guinea pig for the SystemLogging service. The steps for proceeding are for Andrew to send the collection of errors to the logging system such that an agent can be developed to perform the error analysis. Roberto spent some time this week on integrating the DMS unit tests from Andrew in the SAM SE suite (this is not yet critical but can be after gaining experience with the results). The LFC test from Andrew was also added as a sensor and allows to check the propagation of information between LFC instances. There was a bug in the CondDB test where jobs were sent to incorrect sites to reflect the Tier-1 CEs (now fixed). - Action(106) - Ricardo - Produce twiki documentation for dirac-install Ongoing, being prepared. - Action(107) - Produce proposal for how to perform standardised installation using install_project/CMT Ricardo will be coming to CERN in a couple of weeks to discuss the outstanding issues. - Action(109) - Expose DataLogging interface on the web page Ongoing. - Action(115) Matvey reported that Vladimir has been very busy with shift work and is continuing to work on this by hand, the automation is in the pipeline however. - Action(116) Ricardo conveyed that the information from his CPU studies would lend extremely well to evaluate a benchmarking factor for CPUs. Of 6k originally stalled jobs resubmitted for 3080 ~25% entered the same state, it appears there could be some infinite looping and Ricardo will perform further studies before contacting Gloria. - Action(117) There was some discussion regarding how to deal with a mixture of sites accepting generic and non-generic pilots. The suggestion was to refuse to send any LHCb user jobs to sites not supporting generic pilots (thereby adopting a secondary mask). The other option is to use VO software tags and it was agreed to proceed in this way ensuring that tags can be published remotely, using the DIRAC site name to guarantee the same picture for all relevant CEs listed in the CS. 2. Production Status Report Ricardo raised the point about GFAL and thread safety. The suspicion is that the OpenSSL in the underlying globus tools is causing the problem but it is unclear who can fix this. Roberto agreed to provide a script to demonstrate the thread safety issue and submit a bug. ( ACTION: Roberto ) - Report from the Production meeting A Recent failures at some sites highlighted a problem with file locking in NFS file systems, Stuart pursued a workaround with Marco Cl. and a new release of the LHCb package is expected in early December. Thomas reported some consistency problems and minor typos in the Bookkeeping so these should be fixed. Andrew summarized that the ongoing staging activity highlights CNAF and GridKa are currently below the expected transfer rates. IN2P3 is not currently working and a direct ticket was submitted to the local admins. The staging activity will be ramped up to the nominal rates in the coming days. Ricardo pointed out that the number of jobs for this activity approaches the limit of the gLite WMS and Andrew agreed to reduce this number by grouping more files together. 3. DIRAC3 Tasks - Plans for the new BK service There were some discussions regarding the BK migration plan, Zoltan attached a link to the agenda. Feedback has been received from users regarding the GUI and Matvey is preparing the web GUI in parallel. It was decided to announce the standalone browser to a wider range of users on lxplus to get further feature requests. 4. AOB None
There are minutes attached to this event. Show them.
    • 09:30 → 09:50
      Ongoing actions 20m
      (07) - 27/11/07 - Zoltan, Elisa - Ongoing new bookkeeping system development. Migration plan (57) - 24/06/08 - Vladimir,Roberto - Automatic publishing of BDII / GOCDB information in the CS using GOCDB site names and provide a summary of recent changes (82) - 01/09/08 - Andrei/Adri - to provide information from Accounting and WMS to allow GridMap for LHCb VO T1 activity (83) - 01/09/08 - Stuart,Roberto - To perform glexec tests (90) - 09/09/08 - Joel - Use new ProductionOutputData JDL parameter in modules instead of creating LFNs (93) - 16/09/08 - Andrew - Improve RAW data monitoring tools and status of files in online runDB (94) - 23/09/08 - Roberto, Marcos - Using SystemLogging service for dynamic services status monitoring (95) - 23/09/08 - Roberto - SLS and SAM sensors development, upgrade SE tests with new DMS unit test (96) - 23/09/08 - Roberto, Matvey - Dashboard SAM tests visualization, porting to DIRAC framework (pending deployment in production) (106) - 30/09/08 - Ricardo - Produce twiki documentation for dirac-install (107) - 30/09/08 - Ricardo,Philippe,Joel,Hubert,MarcoC - Produce proposal for how to perform standardised installation using install_project/CMT (109) - 14/09/08 - Matvey,Andrew - Expose DataLogging interface on the web page (111) - 14/09/08 - Philippe,Alexei,Andrei - Productions with file merging implementation (113) - 21/10/08 - Ricardo, Adri - Document the CS procedures for both the mechanism to define authorization rules in DIRAC and also the current group properties (115) - 21/10/08 - Vladimir,Matvey - Update SAM Agent to monitor and manage submitted jobs and publish results for faulty CEs to the SAM DB (116) - 18/11/08 - Ricardo - Circulate a report on the recent Gauss CPU studies, steer the CPU related activities in DIRAC (117) - 18/11/08 - Ricardo - Add a site attribute to reflect whether generic pilots are accepted there, update the Director (118) - 19/11/08 - Joel - Request increased swap space for volhcb09 and announce when available
    • 09:50 → 10:05
      Production status report 15m
      - Report from the Production meeting:
    • 10:05 → 10:35
      DIRAC3 tasks 30m
      - Plans for the new Bookkeeping service
      more information
    • 10:35 → 10:45
      AOB 10m