Focus on:  Hide Contributions
Back to Conference View
Login

HTCondor Workshop Autumn 2022

→ Europe/Paris
Confindustria Cuneo - Casa Betania

Confindustria Cuneo - Casa Betania

Via Vittorio Bersezio, 9 12100 Cuneo CN Italy
Helge Meinhard (CERN), Todd Tannenbaum (University of Wisconsin Madison (US)), Chris Brew (Science and Technology Facilities Council STFC (GB)), Christoph Beyer
Description

We are very pleased to announce that the 2022 European HTCondor Workshop will be held from Tuesday 11th October to Friday 14th October. Save the dates!  

Due to the current easing of restrictions on gathering and travel we are planning to return to an in-person format meeting. So, we are delighted to announce that INFN-Torino have offered to host the meeting in the beautiful Italian city of Cuneo near Torino. To allow ease of travel to and from the venue the meeting will start at on Tuesday morning and run until lunchtime on the Friday.  

The workshop will be an excellent occasion for learning from the sources (the developers!) about HTCondor, exchanging with your colleagues about experiences and plans and providing your feedback to the experts. The HTCondor Compute Entrypoint (CE) will be covered as well. Participation is open to all organisations (including companies) and persons interested in HTCondor (and by no means restricted to particle physics and/or academia!) If you know potentially interested persons, don't hesitate to make them aware of this opportunity. 

The workshop will cover both using and administering HTCondor; topics will be chosen to best match participants' interests. We would very much like to know about your use of HTCondor, in you project, your experience and your plans. Hence you are warmly encouraged to propose a short presentation. 

If you have any questions, please contact hepix-2022condorworkshop-support@hepix.org . 

We are looking forward to a rich, productive workshop, and we hope to meet many of you at our first in-person workshop for three years! 

Chris Brew (STFC - RAL) and Christoph Beyer (DESY) 
Co-Chairs of organising committee. 

Todd Tannenbaum 
HTCondor Technical Lead, U Wisconsin, Madison, USA

 
 
Registration
HTCondor Workshop 2022 Registration
32
Register
Participants
  • Ben Jones
  • Chris Brew
  • Christoph Beyer
  • Francesco Prelz
  • Gabriele Gaetano Fronze'
  • Gideon Bentum
  • Gregory Thain
  • Helge Meinhard
  • Ivan Glushkov
  • Jaime Frey
  • Jose Flix Molina
  • Marco Mascheroni
  • Matt Connell
  • Nikos Tsipinakis
  • Peter Couvares
  • Pradeep Jasal
  • Stefano Bagnasco
  • Stefano Dal Pra
  • Stefano Lusso
  • Thomas Hartmann
  • Todd Tannenbaum
  • +11
Support
  • Monday, October 10
    • 3:00 PM → 5:00 PM
      Closed Session for Organising Committee Meeting
      Conveners: Chris Brew (Science and Technology Facilities Council STFC (GB)), Christoph Beyer, Mr Gabriele Gaetano Fronze' (INFN Torino (IT) and LIGO-Virgo-Kagra Collaboration (US/IT/JP)), Gregory Thain, Miron Livny, Todd Tannenbaum (University of Wisconsin Madison (US))
  • Tuesday, October 11
    • 9:00 AM → 12:30 PM
      Workshop Session
      • 9:00 AM
        Welcome 10m
        Speaker: Christoph Beyer
      • 9:10 AM
        Introduction and House keeping 20m
        Speaker: Mr Gabriele Gaetano Fronze' (INFN Torino (IT) and LIGO-Virgo-Kagra Collaboration (US/IT/JP))
        Welcome.pdf
      • 9:30 AM
        Pointers to other HTCondor references 10m
        Speaker: Gregory Thain
        PointersToOtherInfo.pdf
        PointersToOtherInfo.pptx
      • 9:40 AM
        Whats new in HTCSS? Whats coming up? 20m
        Speaker: Todd Tannenbaum (University of Wisconsin Madison (US))
        Whats_New_Oct_2022_15min.pdf
        Whats_New_Oct_2022_15min.pptx
      • 10:00 AM
        Whirlwind Tour of all Condor Tools 30m
        Speaker: Gregory Thain
        whirlwind-tour.pdf
        whirlwind-tour.pptx
      • 10:30 AM
        Coffee Break 30m
      • 11:00 AM
        How OSG Increases Goodput with Elasticsearch 30m
        Speaker: Todd Tannenbaum (University of Wisconsin Madison (US))
        Increased_Goodput_Elasticsearch.pdf
        Increased_Goodput_Elasticsearch.pptx
      • 11:30 AM
        Exploitation of network-restricted resources at the Barcelona Supercomputer Center by CMS 20m

        This contribution describes a number of actions and novel solutions introduced by the Spanish CMS community in order to facilitate the inclusion of BSC resources into the CMS computing infrastructure for their use by the collaboration. In particular, new HTCondor developments that allow condor daemons to interact via shared filesystems.

        Speaker: Jose Flix Molina (CIEMAT - Centro de Investigaciones Energéticas Medioambientales y Tec. (ES))
        20221011_CMS_PIC_BSC_HTCondorWS (1).pdf
    • 12:30 PM → 2:00 PM
      Lunch 1h 30m
    • 2:00 PM → 4:50 PM
      The HTCondor CE
      Conveners: Gregory Thain, Todd Tannenbaum (University of Wisconsin Madison (US))
    • 4:50 PM → 5:20 PM
      Workshop Session
      • 4:50 PM
        Phasing Out GSI Authentication in the CMS Submission Infrastructure 20m

        The CMS Computing Submission Infrastructure group manages and exploits a set of HTCondor pools to satisfy the experiment computing needs. The biggest of those pools, the so-called CMS Global Pool, currently aggregates nearly 400k CPU cores dynamically from pledged and opportunistic WLCG resources.
        Historically, authentication among the diverse components of the infrastructure used the Grid Security Infrastructure (GSI), based on identities and X509 certificates. However, more modern authentication standards based on capabilities and tokens have emerged over the years. The CMS Submission Infrastructure group is in the process of phasing out the GSI part of its authentication layers in favor of IDTokens and Scitokens. In this contribution we will report on the current status of this migration, and our plans for the final GSI phase out.

        Speaker: Nikos Tsipinakis (CERN)
        HTCondor Week October 2022 Nikos Tsipinakis.pdf
  • Wednesday, October 12
    • 9:00 AM → 11:40 AM
      Workshop Session
      • 9:00 AM
        The HTC/HTCSS Data Story 25m
        Speaker: Gregory Thain
        data-story.pdf
        data-story.pptx
      • 9:25 AM
        Submitting sets of jobs 25m
        Speaker: Todd Tannenbaum (University of Wisconsin Madison (US))
        Submitting_Sets_of_Jobs.pdf
        Submitting_Sets_of_Jobs.pptx
      • 9:50 AM
        User data access and challenges 25m

        To complement grid production workflows, the National Analysis Facility (NAF) at DESY aims to provide individual users with an easy way without hurdles to process their data fast.
        Since users are used to plain file paths, a number of storages are mounted as network mounts on the NAF HTCondor nodes. But with users showing a wide range of access patterns, we encountered occassionally problems with mounts and file access affecting the cluster stability.
        To better understand the interplay between users, kernel and file system clients, we are now working on extending our monitoring. With the already available detailed event monitoring on the storage instances, we are going to add a complementary status overview of the execution side with the aim of a consolidated view the of storage and compute clusters.

        Speakers: Christoph Beyer, Thomas Hartmann (Deutsches Elektronen-Synchrotron (DE))
        DESY_NAF_HTC.pdf
        Job Monitoring plans @ DESY
      • 10:20 AM
        Coffee Break 30m
      • 10:50 AM
        Gravitational-Wave Computing - Needs and Trends 25m

        An Idiosyncratic Airing of Grievances, Condor-Style

        Speaker: Peter Couvares (LIGO Laboratory / Caltech)
        Couvares LIGO-G2200862-v2.pdf
      • 11:15 AM
        Multi tenancy with htcondor 25m

        At final we have a large unifed cluster.
        4 years ago there were three clusters sharing the same infrastructure (network and data center).
        A dedicated cluster per team.

        We decided to join forces and recreate a large cluster to better utilize the resources.

        Docker volumes able us to share compute resources and allow different storage access per team.

        In addition for that we are able to go further using docker networks and iptables. it allow us to share phisycal network without the risk of data leaks between teams.

        We will discuss about the challenges that we currently facing different job sizes, small pool for fast response and negotiation cycle.

        Speaker: David Handelman
    • 11:40 AM → 12:30 PM
      Office Hours

      One-to-one sessions with the HTCondor developers to discuss your setup and your issues

    • 12:30 PM → 2:00 PM
      Lunch 1h 30m
    • 2:00 PM → 5:30 PM
      Certificates, Tokens, and Identity: Past, Present, and Future
  • Thursday, October 13
    • 9:00 AM → 11:15 AM
      The HTCSS Access Point Story
    • 11:15 AM → 11:40 AM
      Workshop Session: Discussion on Future Challenges
    • 11:40 AM → 12:30 PM
      Office Hours

      One-to-one sessions with the HTCondor developers to discuss your setup and your issues

    • 12:30 PM → 2:00 PM
      Lunch 1h 30m
    • 2:00 PM → 2:40 PM
      Workshop Session
      • 2:00 PM
        HTCondor at CERN 20m
        Speaker: Ben Jones (CERN)
        cerncondoreu22.pdf
        cerncondoreu22.pptx
      • 2:20 PM
        Elemento's AtomOS and the path to the true Cloud 20m
        Speaker: Mr Gabriele Gaetano Fronze' (INFN Torino (IT) and LIGO-Virgo-Kagra Collaboration (US/IT/JP))
        Elemento Tech Pitch
    • 2:40 PM → 3:30 PM
      Show your Toolbox

      An experimental open session for attendees to briefly demonstrator any tools, tops or tricks they use to manage their use of HTCondor.

    • 3:30 PM → 5:30 PM
      Tour
    • 7:30 PM → 8:30 PM
      Social
  • Friday, October 14
Powered by Indico v3.3.5-pre