• Grid-Operator-on-Duty handover (5h0')
  From ROC France (backup: ROC Italy) to ROC AsiaPacific (backup: ROC Central Europe)
  1. Russian TOP BDIIs: Found repeated IS timeout problems on ru-IMPB-LCG2 and other RU sites, due to central BDIIs. Found several sites pointing GFAL to:
     lcg15.sinp.msu.ru (single host) or
     lcgbdii.jinr.ru (single host)
     plus others pointing to CERN TOP BDIIs. We had TOP BDII discussions at last few phones. I can't see any BDII status from Russia in the minutes. Should we:
     - suggest some reorganization?
     - wait for GFAL improvements and see?
  
  
  
  2. Some disturbance because GOC-DB unscheduled down:
     - 2007-03-20 morning
     - 2007-03-21 afternoon
     - 2007-03-22 late morning
     * if you want, Gridice http://gridice2.cnaf.infn.it:50080/gridice/site/site.php gives a reasonably updated cache of downtimes (GOCDB failover replica not yet ready).
  
  
  
  3. Again on the very long PPS tkt #15574 - PreGR-01-UoM. I don't know if ROC opinion: "It's a SAM problem" is right. I try to transfer it to PPS unit and ask at Operations Meeting if this is correct. If we close it, someone will open a new one soon because tests are failing!
  
  
  
  4. Just a remark at sites(/ROCs): many sites use SD after a problem is detected on them, and extend the SD while they are trying to solve it. We think this can be reasonable or not on a "case by case" basis, but in general it shouldn't be the regular practice on a production system.

• PPS reports
  PPS reports were not received from these ROCs: Italy, North Europe, Asia Pacific
  
• gLite 3.0 PPS-update 24 deployed. This update contains:
  • improved slapd cache on BDII
  • vulnerability fix in gsiopenssh


• gLite 3.0 PPS-update 25 coming soon. This update contains, among other, also patches for YAIM, so at leat "formally" all meta-packages will turn out to be affected. We'll try to be as specific as we can in the release notes, but sysadmins are asked to cross-check carefully to see which services are actually affected
1. Problems in submission of SAM tests to gLite Ces still under investigation. Submission of SAM tests has been re-started through a workaround [CERN]



2. gLiteCE (zeus76.cyf-kr.edu.pl) has been put into downtime, because of some unresolved problems [Central Europe]

Question from PPS-Coordination: Are the problems you experience due to the release? Any GGUS tickets existing?

• EGEE issues coming from ROC reports
  Reports were not received from these ROCs:
  
  1. (CERN ROC): Maintenace day correctly handled by GOC, but timezones in SAM were all wrong. They had the maintenance starting 8 hours earlier than it did, i.e. interpretted GOC time as UTC rather than PST. This lead to SAM reporting filure diring downtime, and affects efficiency stats. https://gusiwr.fzk.de/pages/ticket_details.php?ticket=12884.
  
  
  
  2. (CERN ROC): SAM test in every job on WN seem to take up to 300s at beginning and end of job - this is an enormaous waste of cpu, and makes job turnaround poor. Can we disable it? Do other sites see this, or could it be a local mon box(rgma) problem?
  
  
  
  3. (France ROC): How is a VOMS proxy mapped on a grid node (CE, SE, etc.) using LCMAPS ? Is there an official document that explains this mapping mechanism?
  
  
  
  4. (DECH ROC): 64-bit support: Do others have experience finding workarounds? (in addition to discussion e.g. on LCG Rollout, "Who's planning to move to SL/SLC/CentOS 4.x and when?")
  
  
  
  5. (DECH ROC): Problems with LFC upgrade - Impression: testing/certification of MySQL related middleware features has flaws. Improve MySQL support for the future? Is the current testing of MySQL in PPS enough?
  
  
  
  6. (SE Europe ROC): It seems that CIC daily reports for sites contain incorrect links to SAM failures details as of today: https://gus.fzk.de/pages/ticket_details.php?ticket=20043
  
  
  
  7. (SE Europe ROC): One site in IL reports that they get "submitter proxy expired" ggus ticket https://gus.fzk.de/pages/ticket_details.php?ticket=19854 any ideas?
  
  
  
  8. (UK/I ROC): The site is marked as having failed some replica management tests on 22-03-2007. However, the "details" link does not display any data about this job or the reasons for this job failure.
  
  
  

• Tier 1 reports more information  

• WLCG issues coming from ROC reports
  Reports were not received from these ROCs:
  
  1. (AsiaPacific ROC): Do we have an updated estimate of when the following will be available:
     * SLC4 WN
     * unified version of RFIO client for DPM and Castor
     We asking on behalf of our CMS coordinators.
  
  
  
  2. (Central Europe ROC): CE sites can migrate to SLC4 as long as VO software is running on SLC4. Specifically we identified ATHENA (Atlas VO) that could prevent us from migration to SLC4. The site reporting this was not sure about ATHENA status and migration plan. Can someone from Atlas VO comment on ATHENA status?
  
  
  

• Upcoming WLCG Service Interventions (with dates / times where known)
  Link to CIC Portal (broadcasts/news), scheduled downtimes (GOCDB) and CERN IT Status Board
  
  
  • On Tuesday 27th, the Castor system at RAL will be offline for upgrades, this will affect the ralsrm[a-f].rl.ac.uk endpoints, between 0900 and 1500, at the same time there will be some maintenance on the tape robot, preventing restores from tape on dcache-tape.gridpp.rl.ac.uk. Ops vo CE SAM Replica Management tests will be moved to dcache.gridpp.rl.ac.uk while ralsrma.rl.ac.uk is down.

  Time at WLCG T0 and T1 sites.

• FTS service review
  
  Please read the attached report.
  
  • FTS report index - status by site and by VO
  • Transfer goals - status by site and VO
  • Transfer Operations Wiki

• ATLAS service
  

  
  See also https://twiki.cern.ch/twiki/bin/view/Atlas/TierZero20071  and https://twiki.cern.ch/twiki/bin/view/Atlas/ComputingOperations for more information.
  

• CMS service
  See also CMS Computing Commissioning & Integration meetings (Indico) and https://twiki.cern.ch/twiki/bin/view/CMS/ComputingCommissioning
  
  -- Job processing: CMS MC production activities surveying on CMSSW_1_2_3installation, new round of MC prod is starting soon.
  -- Data transfers: lastweek was week-1 of the CMS LoadTest07 (see [*]) with focus on both T0-T1 routesand T1-T2 routes. Good stop&start exercise by PhEDEx to handle the scheduleddown at CERN due to Castor intervention (firmware upgrades, Wednesday March21st) with no problems, in good synchronization and communication withCastor@CERN people, and no problems seen on the CMS Castor pool after theintervention, also.
  --- T0-T1 exercises were quite smooth through all week: all7 T1's joined, and CMS ran at 300-350 MB/s of aggregate transfer rate to allT1's (daily average).
  --- T1-T2 exercises performed differently in differentregions. ~27 T2's joined, and CMS ran at 250-400 MB/s of aggregate transfer ratefrom T1's to T2's.
  --- This week we will focus on multi-VO transfers (asrequested by WLCG) and still exploring and debugging T1-T2 routes, also.
  [*]http://www.cnaf.infn.it/~dbonacorsi/LoadTest07.htm

• ALICE service

• LHCb service
  

  
  Only one major issue this week to report (and to be followed closely) and itregards the format of the tURLs returned by SRM that, from time to time (we hada previous experience at CERN more than one year ago) are incosistent with theunderlying application (ROOT) and then useless.
  
  From GGUS ticket #20160
  
  Itlooks like the format of the tURL returned by SRM for accessing data to CASTOR 1at PIC is not in a format that ROOT can understand. Even playing somemanipulation on the returned tURL string from SRM doesn't help and ROOT can'topen the file. This is a problems that reminds another one that has been facedat CERN a long while ago. The procedure we use in DIRAC is the following: 
  1.The SURL at the site is obtained from the LFC, given the LFN 
  2. The SURL isconverted into a tURL (and the file is pre-staged) using lcg-gt  rfio 
  3.This tURL is used by Gaudi/POOL/ROOT for opening the file
  
   An examplefollows: [lxplus014] ~/DIRAC > lcg-lrlfn:/grid/lhcb/production/DC06/v1-lumi2/00001464/DIGI/0000/00001464_00000722_4.digisrm://castorsrm.pic.es:8443/castor/pic.es/grid/lhcb/production/DC06/v1-lumi2/00001464/DIGI/0000/00001464_00000722_4.digi
  
  [lxplus014] ~/DIRAC > lcg-gtsrm://castorsrm.pic.es:8443/castor/pic.es/grid/lhcb/production/DC06/v1-lumi2/00001464/DIGI/0000/00001464_00000722_4.digirfiorfio://cfs0163.pic.es//stage/cfs0163/lh/stage/00001464_00000722_4.digi.133133575079557 0 
  
  For simplicity we just run a simple Python script that uses ROOT(5.13.04c) with just a TFile.Open() and here is the result: Executingresult =TFile.Open(rfio://cfs0163.pic.es//stage/cfs0163/lh/stage/00001464_00000722_4.digi.133133)====================================================== Error in: filecfs0163.pic.es//stage/cfs0163/lh/stage/00001464_00000722_4.digi.133133 does notexist Result is: None We have tried to remove the stager host name (this seemsto work at CERN?), without success =============================== New TURL is:rfio:/stage/cfs0163/lh/stage/00001464_00000722_4.digi.133133====================================================== Executing result =TFile.Open(rfio:/stage/cfs0163/lh/stage/00001464_00000722_4.digi.133133)====================================================== Error in: file/stage/cfs0163/lh/stage/00001464_00000722_4.digi.133133 does not exist Resultis: None 
  
  Here is a similar attempt (successful) at CERN Castor1: 
  root [0]TFile::Open("rfio:/shift/lxfsrk5504/data03/z5/stage/00001355_00034199_5.digi.162180")(class TFile*)0x8c1fdc0 
  
  The same problem was faced some time ago at CERN andthe only solution that was found so far was to return a simple tURL of the formrfio:/castor/pic.es/grid/lhcb/production/DC06/v1-lumi2/00001464/DIGI/0000/00001464_00000722_4.digi
  
  Is this something that can be configured in the SRM server at PIC? For thetime being this is a major show-stopper for any kind of reconstruction oranalysis job at PIC.
  

Item 1