HTCondor / ARC CE Workshop

Name: HTCondor / ARC CE Workshop
Start: 2016-02-29T13:00:00+01:00
End: 2016-03-04T14:00:00+01:00
Location: ALBA Synchrotron

29 Feb 2016, 13:00 → 4 Mar 2016, 14:00 Europe/Zurich

ALBA Synchrotron

Carrer de la Llum 2-26 08290 Cerdanyola del Vallès, Barcelona, Spain Tel: +34 93 592 43 00 Coordinates: +41°29'12.38", +2°6'35.74" (41.486773, 2.109929)

Brian Paul Bockelman (University of Nebraska (US)), Helge Meinhard (CERN), Jon Kerr Nilsen (University of Oslo (NO)), Jose Flix Molina (Centro de Investigaciones Energ. Medioambientales y Tecn. - (ES), Marc Rodriguez (ALBA), Mario Diaz-Laguardia (Alba Synchrotron), Todd Tannenbaum (Univ of Wisconsin-Madison, Wisconsin, USA)

Description

The workshop, second one after the successful workshop at CERN in December 2014, is an opportunity for novice and experienced users of HTCondor to learn, get help and have exchanges between them and with the HTCondor and ARC developers and experts. It is primarily addressed at users from EMEA. The workshop will consist of presentations, tutorials and "office hours" for consultancy. The HTCondor CE (Compute Element) as well as the ARC CE will be covered as well. ALBA Synchrotron Light facility located in Cerdanyola del Vallès (Barcelona) act as the event host, with the additional help of Port d'Informació Científica (PIC) for event preparation/coordination. There is a fee of 80€ (VAT included) which covers the lunches and all coffee breaks along the event.

Support

hepix-condorworkshop2016@cern.ch

Participants

64 View full list

Monday 29 February
- 13:00
  
  Registration
- Welcome and logistics
  
  2016_02_29_ALBA_HTCondor-1.pdf
  
  2016_02_29_ALBA_HTCondor-1.pptx
  
  AlbaHTCondorWorkshopLogistics.pdf
- HTCondor presentations and tutorials
  - 1
    
    Welcome
    
    Speaker: Miron Livny
    
    02-16-CWE.pdf
    
    02-16-CWE.pptx
  - 2
    
    HTcondor deployment & plans in PIC
    
    Short talk to explain about the HTcondor testbed and deployment plans at PIC Tier1.
    
    Speaker: Jose Flix Molina (Centro de Investigaciones Energ. Medioambientales y Tecn. - (ES)
    
    htcondor-workshop-pic.pdf
  - 3
    
    Data analysis in gravitational wave research
    
    This presentation describes a sketch of the gravitational wave detector LIGO. It introduces concepts of match filtering in analyzing experimental data and outlines the required computational resources and one realization - the Atlas computing cluster, respectively the HTCondor framework. Finally, results of the experiment are presented.
    
    Speaker: Henning Fehrmann (MPI für Gravitationsphysik)
    
    Barcelona.pdf
- 16:00
  
  Coffee Break
- 4
  
  PIC welcome
  
  Speaker: Manuel Delfino Reznicek (Universitat Autònoma de Barcelona (ES))
- HTCondor presentations and tutorials
  - 5
    
    HTCondor Pool Basics
    
    Speaker: John Knoeller
    
    3 TJs_Stats_Condor_Week_Barcelona_2016.pdf
    
    3 TJs_Stats_Condor_Week_Barcelona_2016.pptx
    
    Admin_Basics_Condor_Week_Barcelona_2016.pdf
    
    Admin_Basics_Condor_Week_Barcelona_2016.pptx
    
    Boxing_Users_Condor_Week_Barcelona_2016.pdf
    
    Boxing_Users_Condor_Week_Barcelona_2016.pdf
    
    Boxing_Users_Condor_Week_Barcelona_2016.pptx
    
    Pool_Management_Condor_Week_Barcelona_2016.pdf
    
    Pool_Management_Condor_Week_Barcelona_2016.pptx
  - 6
    
    Ask The Experts, Part I
    
    Speakers: Antonio Puertas Gallardo (European Commission DG JRC), Ben Jones (CERN), Jordi Peralta (Anima ITS), Jose Flix Molina (Centro de Investigaciones Energ. Medioambientales y Tecn. - (ES), Michel Jouvin (Laboratoire de l'Accelerateur Lineaire (FR)), Miron Livny (University of Wisconsin-Madison)
Tuesday 1 March
- HTCondor presentations and tutorials
  - 7
    
    HTCondor Policies: Job and Machines
    
    Speaker: Todd Tannenbaum (Univ of Wisconsin-Madison, Wisconsin, USA)
    
    TannenbaumT_Policy.pdf
    
    TannenbaumT_Policy.pptx
  - 8
    
    Configuring HTCondor for ATLAS Multicore
    
    Configuring ARC/Condor for ATLAS Multicore This talk will cover the specific measures used at the Liverpool grid site to provide multicore job slots for ATLAS using an ARC CE in front of a CONDOR batch system.
    
    Speaker: Mr Stephen Jones (Liverpool)
    
    mc.odp
    
    mc.pdf
- 10:10
  
  Coffee Break
- HTCondor presentations and tutorials
  - 9
    
    Matchmaker Policies: Users and Groups
    
    Speakers: James Frey (Unknown), Zach Miller
    
    BCN-Users_and_Groups.pdf
    
    BCN-Users_and_Groups.pptx
  - 10
    
    HTCondor at CERN
    
    Speaker: Ben Jones (CERN)
    
    cern_condor_barca.pdf
    
    cern_condor_barca.pptx
- 12:00
  
  Lunch
- HTCondor presentations and tutorials
  - 11
    
    HTCondor: CMS Experience
    
    Speaker: Farrukh Aftab Khan (National Centre for Physics (PK))
    
    HTCondorWorkshop_CMSExperience.pdf
  - 12
    
    Security Basics
    
    Speaker: Zach Miller (UW Madison)
    
    BCN_Security_Basics.pdf
    
    BCN_Security_Basics.pptx
  - 13
    
    HTCondor at DESY
    
    Speaker: Christoph Beyer (DESY HH)
    
    htcondor_barc_2016.pdf
  - 14
    
    Best Practices for Building Submit Nodes
    
    Speaker: Brian Paul Bockelman (University of Nebraska (US))
    
    SubmitMachineManagement - 2016.pdf
- 15:30
  
  Coffee Break
- HTCondor presentations and tutorials
  - 15
    
    Best Practices - Monitoring
    
    Speaker: Todd Tannenbaum (Univ of Wisconsin-Madison, Wisconsin, USA)
    
    mironView_gangliad_metrics
    
    TannenbaMonitoringBarcelona.pdf
    
    TannenbaMonitoringBarcelona.pptx
    
    view_Miron.json
  - 16
    
    Effective HTCondor-based monitoring system
    
    The CMS experiment at LHC relies on HTCondor and glideinWMS as its primary batch and pilot-based grid provisioning systems. Given the scale of the global queue in CMS, operators found it increasingly difficult to monitor the pool to find issues and fix them. Operators had to rely on several different webpages, with several different levels of information and sifting tirelessly through logs in order to monitor the pool completely. Therefore, coming up with a suitable monitoring system was one of the crucial items before the beginning of Run 2 to ensure early detection of issues and to give a good overview of the whole pool. Our new monitoring page (cms-gwmsmon.cern.ch) utilizes the condor classAd mechanism to provide a complete picture of the whole submission infrastructure in CMS. Monitoring page includes useful information from HTCondor schedulers, central manager, glideinWMS frontend and factory. It also incorporates information about users and tasks making it easy for operators to provide support and debug issues.
    
    Speaker: Justas Balcas (California Institute of Technology (US))
    
    Effective HTCondor-based monitoring system
    
    Monitoring page
  - 17
    
    Networking in HTCondor
    
    Learn how HTCondor navigates the twists and turns of modern networks, filled with firewalls, private networks, and multiple protocols. Topics will include CCB (Condor Connection Broker), shared port, and IPv4/6.
    
    Speaker: Jaime Frey
    
    Barcelona-Networking.pdf
    
    Barcelona-Networking.pptx
  - 18
    
    Ask The Experts, Part II
    
    Speaker: Todd Tannenbaum
Wednesday 2 March
- ARC CE office hours
- HTCondor presentations and tutorials
  - 19
    
    HTCondor Python Bindings
    
    Speakers: Brian Paul Bockelman (University of Nebraska (US)), John Knoeller
    
    HTCondor-Python.pdf
  - 20
    
    Putting your job in a box: HTCondor, Docker, and containers
    
    Speaker: John Knoeller
    
    Boxing_Users_Condor_Week_Barcelona_2016.pptx
- 10:00
  
  Coffee Break
- HTCondor presentations and tutorials
  - 21
    
    HTCondor Experiences at GRIF
    
    Speaker: Guillaume Philippon (Unknown)
    
    HTCondor 20160302.pdf
    
    HTCondor 20160302.pptx
  - 22
    
    Advanced Pool Management - Best Practices
    
    Speakers: John Knoeller, Zach Miller
    
    Pool_Management_Condor_Week_Barcelona_2016.pdf
    
    Pool_Management_Condor_Week_Barcelona_2016.pptx
    
    Pool_Management_Condor_Week_Barcelona_2016.pptx
  - 23
    
    What's New? What's Coming Up?
    
    Speakers: Todd Tannenbaum (Univ of Wisconsin-Madison, Wisconsin, USA), Todd Tannenbaum
    
    TannenbaumT_WhatsNew_Barcelona_2016.pdf
    
    TannenbaumT_WhatsNew_Barcelona_2016.pptx
- 12:00
  
  Lunch
- 12:50
  
  Facilities tour: ALBA Synchrotron
- ARC CE office hours
- HTCondor and the Grid presentations and tutorials
  - 24
    
    HTCondor-CE Overview and Architecture
    
    Speaker: Brian Paul Bockelman (University of Nebraska (US))
    
    HTCondor-CE-Overview.pdf
  - 25
    
    Integrating HTCondor with Arc
    
    Speaker: Andrew David Lahiff (STFC - Rutherford Appleton Lab. (GB))
    
    IntegratingHTCondorWithARC_v1.pdf
    
    IntegratingHTCondorWithARC_v1.ppt
  - 26
    
    Integration with EGI Middleware
    
    Speaker: Iain Bradford Steers (CERN)
    
    HTCondor-CE - EGI Integration.pdf
  - 27
    
    HTCondor Information System
    
    Speaker: Brian Paul Bockelman (University of Nebraska (US))
    
    HTCondor-Info-Services.pdf
- 15:30
  
  Coffee Break
- HTCondor and the Grid presentations and tutorials
  - 28
    
    Grid Universe Overview
    
    Speaker: Jaime Frey
    
    Barcelona-GridUniverse.pdf
    
    Barcelona-GridUniverse.pptx
  - 29
    
    HTCondor-CE and CERN
    
    Speaker: Iain Bradford Steers (CERN)
    
    HTCondor-CE - CERN Grid.pdf
  - 30
    
    HTCondorCE in DIRAC
    
    Recently support for the HTCondorCE computing elements was added to the DIRAC grid software. This presentation will discuss first experience of the ILC VO with running DIRAC pilots and payloads on HTCondorCE instances at CERN and various OSG grid sites, and will present issues and solved issues and possible improvements.
    
    Speaker: Andre Sailer (CERN)
    
    160302_sailer_HTCondorDirac.pdf
- 31
  
  Feedback session
Thursday 3 March
- ARC CE presentations and tutorials
  
  Going through different ways of running ARC CE for ATLAS computing; experiences in moving from Cream CE to ARC CE; setting up (1) a basic ARC CE, (2) ARC CE for WLCG with HTCondor, (3) non-gridftp ARC CE; ARC roadmap.
  - 32
    
    Different ways of running ARC for ATLAS computing
    
    Speaker: Jon Kerr Nilsen (University of Oslo (NO))
    
    ARC4ATLASComputing.pdf
    
    ARC4ATLASComputing.pptx
  - 33
    
    Concepts of the ARC CE - part 1
    
    Speaker: Jon Kerr Nilsen (University of Oslo (NO))
    
    ARCoverviewandkeyconcepts.pdf
    
    ARCoverviewandkeyconcepts.pptx
- HTCondor and HTCondor CE office hours
- 10:00
  
  Coffee break
- ARC CE presentations and tutorials
  
  Going through different ways of running ARC CE for ATLAS computing; experiences in moving from Cream CE to ARC CE; setting up (1) a basic ARC CE, (2) ARC CE for WLCG with HTCondor, (3) non-gridftp ARC CE; ARC roadmap.
  - 34
    
    Concepts of the ARC CE - part 2
    
    Speaker: Jon Kerr Nilsen (University of Oslo (NO))
  - 35
    
    Migration from a CREAM/Torque cluster to ARC/Condor at the Liverpool T2
    
    This talk will cover the specific measures used at the Liverpool grid site to provide multicore job slots for ATLAS using an ARC CE in front of a CONDOR batch system.
    
    Speaker: Stephen Jones (Liverpool University)
    
    arccondormc.odp
    
    arccondormc.pdf
- 12:00
  
  Lunch
- ARC CE presentations and tutorials
  
  Going through different ways of running ARC CE for ATLAS computing; experiences in moving from Cream CE to ARC CE; setting up (1) a basic ARC CE, (2) ARC CE for WLCG with HTCondor, (3) non-gridftp ARC CE; ARC roadmap.
  - 36
    
    Moving from Cream CE to ARC CE
    
    Speaker: Andrew David Lahiff (STFC - Rutherford Appleton Lab. (GB))
    
    MovingFromCREAMtoARC_v1.pdf
    
    MovingFromCREAMtoARC_v1.ppt
  - 37
    
    ARC roadmap
    
    Speaker: Jon Kerr Nilsen (University of Oslo (NO))
    
    ARC roadmap.pdf
    
    ARC roadmap.pptx
- HTCondor and HTCondor CE office hours
- 15:30
  
  Coffee break
- ARC CE presentations and tutorials
  
  Going through different ways of running ARC CE for ATLAS computing; experiences in moving from Cream CE to ARC CE; setting up (1) a basic ARC CE, (2) ARC CE for WLCG with HTCondor, (3) non-gridftp ARC CE; ARC roadmap.
  - 38
    
    Tech preview: Setting up ARC with Python-based backend
    
    Speaker: Martin Skou Andersen (University of Copenhagen (DK))
  - 39
    
    End-of-day discussion
- 18:20
  
  Facilities tour: PIC Tier-1 center
- 40
  
  PIC visit instructions
  
  https://goo.gl/maps/5zJUgw36c6G2
  
  Speaker: Jose Flix Molina (Centro de Investigaciones Energ. Medioambientales y Tecn. - (ES)
Friday 4 March
- ARC CE office hours
- HTCondor and HTCondor CE office hours
  - 09:00
    
    Coffee break
- 11:00
  
  Coffee Break
- ARC CE office hours
- HTCondor and HTCondor CE office hours

Choose timezone

HTCondor / ARC CE Workshop

ALBA Synchrotron