Deployment team

Tuesday 15 Apr 2008, 11:00 → 13:10 Europe/Zurich

EVO - GridPP Deployment team meeting

Jeremy Coles

- This is the weekly DTEAM meeting - The intention is to run the meeting in EVO: http://evo.caltech.edu/evoGate/. Join the meeting in the GridPP Community area - - The phone bridge number is +41 22 76 71400. Th phone bridge ID is 124434 with code: 4880

Minutes dteam-20080415.txt

11:00 → 11:20 Experiment problems/issues

Review of weekly issues by experiment/VO - LHCb - CMS CMS report at ops meeting (Daniele): " * News on Development: Logfiles archiving: post-poned to ProdAgent v.0.9. Chained processing: implementantion largely in place, still scheduled for June release; dealing with large MySQL DBs: some improvement indeed came with latest release, still working on it. * Data certification, Processing at the T0: CERN very busy with RelVal production. Validated releases: CMSSW v1.8.4, CMSSW V2.0.0_pre9. High statistics RelVal samples could not be started at FNAL due to problem, had to use CERN. Tier-0 unavailable due to production, limited to relVal queue. Upcoming release is the 2.0.0. It will take precedence over 1.1.0_pre1 if necessary, the standard set will run at CERN, the high statistics set will run at FNAL in parallel to massive FastSim production. * Re-processing: still running the never-ending CSA07 signal workflows: allrequests finished, waiting for more input datasets, transfers seem not to work as good. Soups at FNAL: work in progress. The important 1.8.4 FastSim production has started: AlcaReco & physics requests, started at all T1 (also those in don, now are used, e.g. FZK and CNAF). Problems mostly at the config level and due to start-up, not really site issues (yet). * MC production: 40k cosmics data with CMSSW v1.7.7 now available to physicists in global DBS. 10M cosmics requet with CMSSW v1.8.4 has srated in OSG, plus some more samples. FastSim production: all requests injected in ProdRequest. * Data Transfers and Integrity, DDT-2/LT status: Low transfer activity (/Prod instance) from CERN to T1 sites (only RAL and FNAL, ~3 TB out of CERN). ~1 TB tape backlog from T1's seen at FNAL. The t1transfer pool at CERN had peaks all within 1k max files to be migrated to tapes. --- Running a campaign to overview production transfers which did not complete within 30 days from the subscription: it will help to cut the tails wherever useless and identify problems/bottlenecks in the production transfer system (or in the transfer tool), much work needed still on top on such provided lists, though. --- DDT status: We have 317 commissioned links (as of April 11th), +23 wrt last week (!). The breakdown is: all 56 T[01]-T1 crosslinks (some to be re-exercised to due back up&runnning after downs); 162/320 (51%) T1-T2 downlinks and 93/320 (29%) T2-T1 uplinks; 6 T2-T2 links. From the "Site Commissioning" pov, concerning the link testing, 37/40 T2 have at least 1 commissioned downlink upink to the associated T1, and - among these - 30 have at least 2 commissioned T1-T2 downlinks. In total, 93% of the previously commissioned links have already PASSED the new metric as of April 11th (2 months after the start of this DDT-2 phase). --- Day-2-day details at https://twiki.cern.ch/twiki/bin/view/CMS/DDTLinkExercising, and (NEW!) more details now visible again online at Nicolo's page: http://magini.web.cern.ch/magini/ddt.html. * LINKs: Computing meetings of the week: http://indico.cern.ch/conferenceDisplay.py?confId=31923 " - ATLAS - Other

11:20 → 11:35 ROC update

ROC manager update ************************* - There is a ROC manager's meeting today: http://indico.cern.ch/conferenceDisplay.py?confId=23758. -- Topics of interest: EGEEIII work breakdown structure & SLA vs SLD document definition. ops meeting update ********************* Lots on releases (extract below from ops meeting agenda): # gLite 3.1 Update18 went to production last Monday. The update contains: * NEW: glite-MON for SL4 * DPM 1.6.7-4 o fix for bug #33769: incorrect pool free space after dpm-drain o improved ACL management for srmMkdir command * UI/WN/VOBOX o lcg-tags non longer produces Globus warnings suppressed o voms-admin client 2.0.6-1 providing ACL support on command line * vdt_globus_essentials (affecting several services and notably the CE) o bug fix to prevent globus-job-manager processes to pile-up on a CE (big observed at CERN after SAM WMS?RB tests were enabled ) * voms-admin server (VOMS) o Refactored voms-admin-ping script o ACL management web service (compatible with client >= 2.0.6-1) o Registration web service. o many bug fixes Details in: http://glite.web.cern.ch/glite/packages/R3.1/updates.asp Now in pre-production # gLite3.1.0 PPS Update22 passed the pre-deployment tests and it is now being installed by the PPS sites. The release contains, among others, an update of yaim-core, so, technically, all services are concerned. The full list of patch deployed is: * glite-AMGA_oracle (initial release) * UI/WN/VOBOX o GFAL/lcg_util: many bug fixes o new lcg-ManageVOTAg version (solving bug 34245) o lcg-infosites: new option to query the wms and lb associated to a VO. -f option to filter based on the site name o [ YAIM ] glite-yaim-clients: bug fixes + configurable list of WMS and LB * R-GMA o Switch back to using MEMORY instead of DATABASE producer * YAIM (affecting all nodes) o new yaim-core with a consistent list of changes and bug fixes * CE o change to lcg-info-dynamic-scheduler to support DENY tags Details in: https://twiki.cern.ch/twiki/bin/view/EGEE/PPSReleaseNotes_310_PPS_Update22 # gLite3.1.0 PPS Update23 was released to PPS and it is currently in phase of pre-deployment testing. It contains: * WMS LB (SL4): first release to PPS * UI/WN/VOBOX o edg-gridftp-client-1.2.8 fixes bugs 33205, 27274 o DPM/LFC v1.6.10 * DPM/LFC o DICOM back-end service for DPM o re-buildable source RPMs o support for MacOSX o group writable directories when SRM started with umask 0 o bug fixes * CE o Patch to improve the performance of lcg CE * Several serivces affected o lcg-vomscerts-4.9.0 adds next cert for lcg-voms Details in: https://twiki.cern.ch/twiki/bin/view/EGEE/PPSReleaseNotes_310_PPS_Update23 # gLite3.0.2 PPS Update47 was released to PPS and it is currently in phase of pre-deployment testing. It contains: * FTS: o FTA Update: change the gridFTP session handling Details in: https://twiki.cern.ch/twiki/bin/view/EGEE/PPSReleaseNotes_302_PPS_Update47 Soon in production 2008-04-11(1): Task: gL3.1 U19 --> Production in preparation The update will contain: * UI/WN/VOBOX o may bug fixes, including the on epreventing to use aliases for WMS o new lcg-ManageVOTAg version * MON o R-GMA fix for forwards compatibility - 3.1.0 PPS Update 22 * Many services o lcg-vomscerts-4.9.0 adds next cert for lcg-voms - OPS test using lfc-lhcb.grid.sara.nl as LFC server for OPS and screwing up the monitoring was raised by several ROCs

11:35 → 11:40 Follow up on networking

- One site isolated TCP problems to a bad switch - what of the others? - What are the TCP vs UDP requirements - What is happening regarding SE tuning?

11:40 → 11:45 NGS - minimum stack feedback

- refer to email

document NGS_Software_Stack_v0.1.doc (Protected) NGS_Software_Stack_v0.1.pdf

11:45 → 11:50 Security update

- Preliminary look at site performance in SSC2 - Other security matters - Policy reminders

11:50 → 11:55 Tier-2 WLCG contributions for March

- Draft of the WLCG document was circulated last week - Comments and observations

more information tier2_reliability_March08.pdf

11:55 → 12:02 Round-the-team update

12:02 → 12:05 Actions review

12:05 → 12:10 AOB

- Do we need a UKIROC meeting this Thursday? -- Topics to be covered - Progress with the T2 reports for Q108 - Questions and issues for the workshop