Name: WLCG-OSG-EGEE Operations meeting
Start: 2008-11-03T16:00:00+01:00
End: 2008-11-03T18:00:00+01:00
Location: CERN conferencing service (joining details below)

- 16:00 → 16:01
  
  Feedback on last meeting's minutes 1m
  
  Minutes
- 16:01 → 16:30
  EGEE Items 29m
  - <big> Grid-Operator-on-Duty handover </big>
    
    From: NE and CERN
    To: Italy and France
    
    Report from Steve Traylen::
    
    Details of tickets reaching final step of escalation: ROC_North, ITPA-LCG2, GGUS:42015, Nothing since 16th October.
    
    Report from David Groep:
    
    JP-HIROSHIMA-WLCG: id#9165 - GGUS Ticket #41683 No response whatsoever from this site. Nothing. Not a single bit.
    SDU-LCG2: id#9164 - GGUS Ticket #41680 Absolutely no response from the site for 30 days. Not sure why we keep wasting our time on this one. The site has been dead with the very same "File not available.Cannot read JobWrapper output, both from Condor and from Maradona." error. Maybe escalation will trigger a response. Set expiry to 3/NOV
    This week saw a lot of SE downtime that affects the associated CEs. Especially ELTE-HU where the SE iSCSI interface is broken!
    R-GMA at ELTE should be up according to mail exchange of follow-up, but is actually still down.
    STORM front end at INFN-T1 remains unstable. The issue is acknowledged but associated errors keep popping up.
  - <big> PPS Report & Issues </big>
    
    As of last Monday, the VOMS pilot service is installed with the voms from PATCH:2390; voms proxies are available from it. All PPS sites are invited to re-configure their UIs to use this pilot service.
    As always, please find issues from EGEE ROCs and general information in:
    https://twiki.cern.ch/twiki/bin/view/LCG/OpsMeetingPps
  - <big> gLite Release News</big>
    
    Please find gLite release news in:
    
    https://twiki.cern.ch/twiki/bin/view/LCG/OpsMeetingGliteReleases
  - <big> EGEE issues coming from ROC reports </big>
    
    ROC Italy were the only ROC not to have submitted a report by the 14:00 deadline.
    
    ROC Germany/Switzerland:BDII Problems
    Region experienced problems with the new (TopLevel) BDII release: some queries give no output. With old versions this problem did not occur. Are other sites also affected?
    For example the WMS show entries like:
    DATUM -I: [Info] fetch_bdii_ce_info(ldap-utils.cpp:567): zeus: skipped due to empty ACBR.
    ROC SWE:SRM failures explained:
    PIC supplied details concerning one hour of SAM failures on 30-Oct-2008.
    ATLAS were running jobs at PIC which were reading several files via SRM, using lcgcp (up to 14k srmget/hour). This generated a high load in the SRM, which didn't service the SAM tests quickly enough.
    Solution: The ATLAS contact person has asked to change the local access protocol for reading from lcgcp to dcap (dccp). However, until the change is made, the problem could come back. As medium/long term solution they're thinking of a SRM server upgrade (x64+more RAM for catalina), and possibly splitting the service over several servers.
- 16:30 → 17:00
  WLCG Items 30m
  - <big> WLCG issues coming from ROC reports </big>
    
    AP ROC:No specific issue, but it might interest ATLAS to know that TAIWAN-LCG2 is currently working on a couple of problems:
    Source File Preparation Problem from TAIWAN-LCG2 Storage Element (ATLASMCDISK Space Token).
    File transfer problem at TAIWAN-LCG2_MCDISK in ASGC Cloud
    
    ROC France: ATLAS pilot jobs at CCIN2P3
    For several months now, ATLAS has been submitting a huge number of pilot jobs even when there is no task to be treated. Despite having notified French ATLAS production team of this, and attempting manual regulation of pilot job submission, 25% of ATLAS pilot jobs are still doing nothing once running.
    Could ATLAS Production please adapt its execution engine to automatically regulate pilot job submission according to the number of tasks in their central queue?
  - <big>WLCG Service Interventions (with dates / times where known) </big>
    
    Useful links:
    CIC Portal for broadcasts and news
    Scheduled downtimes (in the GOCDB)
    ATLAS site downtime calendar
    CERN IT Status Board
    
    Please consult the URLs above for details about this week's interventions.
    Some selected downtimes:
    
    UKI-SOUTHGRID-RALPP will be down as of Thursday & including the weekend to fix air-conditioning
    SEE/TR-03-METU will be down as of Wednesday, also including the weekend for similar reasons
    CE/BMEGrid will be down for 3 days starting today
    In Italy, ENEA-INFO is down while they sort out what they publish as Glue sub-cluster information, and INFN-CS are down while they solve some cooling problems (two weeks)
    
    Time at WLCG T0 and T1 sites.
  - <big> WLCG Operational Review </big>
    
    https://twiki.cern.ch/twiki/bin/view/LCG/WLCGDailyMeetingsWeek081013
    
    Speaker: Harry Renshall / Jamie Shiers
  - <big> Alice report </big>
    
    Item
  - <big> Atlas report </big>
    
    Item
  - <big> CMS report </big>
    
    Item
    
    Speaker: Daniele Bonacorsi
  - <big> LHCb report </big>
    
    Item
  - <big> Storage services: Recommended base versions </big>
    
    The recommended baseline versions for the storage solutions can be found here: https://twiki.cern.ch/twiki/bin/view/LCG/GSSDCCRCBaseVersions
  - <big> Storage services: this week's updates </big>
    
    No updates last week.
    Refer to the Wiki page here: https://twiki.cern.ch/twiki/bin/view/LCG/CCRC08StorageStatus
- 17:00 → 17:30
  OSG Items 30m
  
  Speaker: Rob Quick (OSG - Indiana University)
  - Discussion of open tickets for OSG
    
    Under particular scrutiny from Maria:
    
    GGUS:41670 Problem with the OSG voms server, assigned 1 month ago, not updated since.
    GGUS:42058 Problem to download dataset from Boston Univ. Discussed last week also, assigned Oct 8th, not updated since.
    GGUS:42221 ATLAS transfer problem from AGLT2. Discussed last week also, assigned Oct 11th, not updated since.
    GGUS:42646 ATLAs transfer problem from AGLT2. Seems to be closed in the OSG helpdesk system. Can the same supporters close the GGUS ticket too? It is marked *urgent*.
    GGUS:42647 ATLAS transfer problem from MidwestT2. Assigned on 2008-10-22, not updated since. It is also marked *urgent*.
    
    List of open tickets
- 17:30 → 17:35
  Review of action items 5m
  Open action items are listed here: https://twiki.cern.ch/twiki/bin/view/EGEE/WlcgOsgEgeeOpsMeetingMinutes#Open_Action_Items_from_Operation
  To summarize:
  - 279: Has OCC made the GOCDB and CIC Portal enhancement request for downtime announcement procedure? Any ETI (Estimated Time of Implementation)?
  - 281: LHCb to explain why so many of their SAM tests are failing.
  - 282: Progress on GGUS:42341? A valid use case has been given but, currently, a service can't be in more than one site at a time.
  - 283: ITPA-LCG2 Site doesn't respond, but SAM tests seem to be OK.
- 17:35 → 17:36
  
  AOB 1m

WLCG-OSG-EGEE Operations meeting

28-R-15

CERN conferencing service (joining details below)