HTCondor Workshop Autumn 2022

10 Oct 2022, 14:00 → 14 Oct 2022, 14:00 Europe/Paris

Confindustria Cuneo - Casa Betania

Helge Meinhard (CERN), Todd Tannenbaum (University of Wisconsin Madison (US)), Chris Brew (Science and Technology Facilities Council STFC (GB)), Christoph Beyer

We are very pleased to announce that the 2022 European HTCondor Workshop will be held from Tuesday 11th October to Friday 14th October. Save the dates!   Due to the current easing of restrictions on gathering and travel we are planning to return to an in-person format meeting. So, we are delighted to announce that INFN-Torino have offered to host the meeting in the beautiful Italian city of Cuneo near Torino. To allow ease of travel to and from the venue the meeting will start at on Tuesday morning and run until lunchtime on the Friday.   The workshop will be an excellent occasion for learning from the sources (the developers!) about HTCondor, exchanging with your colleagues about experiences and plans and providing your feedback to the experts. The HTCondor Compute Entrypoint (CE) will be covered as well. Participation is open to all organisations (including companies) and persons interested in HTCondor (and by no means restricted to particle physics and/or academia!) If you know potentially interested persons, don't hesitate to make them aware of this opportunity.  The workshop will cover both using and administering HTCondor; topics will be chosen to best match participants' interests. We would very much like to know about your use of HTCondor, in you project, your experience and your plans. Hence you are warmly encouraged to propose a short presentation.  If you have any questions, please contact hepix-2022condorworkshop-support@hepix.org .  We are looking forward to a rich, productive workshop, and we hope to meet many of you at our first in-person workshop for three years!  Chris Brew (STFC - RAL) and Christoph Beyer (DESY)  Co-Chairs of organising committee.  Todd Tannenbaum  HTCondor Technical Lead, U Wisconsin, Madison, USA

Monday 10 October

15:00 → 17:00 Closed Session for Organising Committee Meeting

Conveners: Chris Brew (Science and Technology Facilities Council STFC (GB)), Christoph Beyer, Mr Gabriele Gaetano Fronze' (INFN Torino (IT) and LIGO-Virgo-Kagra Collaboration (US/IT/JP)), Gregory Thain, Miron Livny, Todd Tannenbaum (University of Wisconsin Madison (US))

Tuesday 11 October

09:00 → 12:30 Workshop Session

09:00 Welcome

Speaker: Christoph Beyer

09:10 Introduction and House keeping

Speaker: Mr Gabriele Gaetano Fronze' (INFN Torino (IT) and LIGO-Virgo-Kagra Collaboration (US/IT/JP))

Welcome.pdf

09:30 Pointers to other HTCondor references

Speaker: Gregory Thain

PointersToOtherInfo.pdf

PointersToOtherInfo.pptx

09:40 Whats new in HTCSS? Whats coming up?

Speaker: Todd Tannenbaum (University of Wisconsin Madison (US))

Whats_New_Oct_2022_15min.pdf

Whats_New_Oct_2022_15min.pptx

10:00 Whirlwind Tour of all Condor Tools

Speaker: Gregory Thain

whirlwind-tour.pdf

whirlwind-tour.pptx

10:30 Coffee Break

11:00 How OSG Increases Goodput with Elasticsearch

Speaker: Todd Tannenbaum (University of Wisconsin Madison (US))

Increased_Goodput_Elasticsearch.pdf

Increased_Goodput_Elasticsearch.pptx

11:30 Exploitation of network-restricted resources at the Barcelona Supercomputer Center by CMS

This contribution describes a number of actions and novel solutions introduced by the Spanish CMS community in order to facilitate the inclusion of BSC resources into the CMS computing infrastructure for their use by the collaboration. In particular, new HTCondor developments that allow condor daemons to interact via shared filesystems.

Speaker: Jose Flix Molina (CIEMAT - Centro de Investigaciones Energéticas Medioambientales y Tec. (ES))

20221011_CMS_PIC_BSC_HTCondorWS (1).pdf

12:30 → 14:00 Lunch

14:00 → 16:50 The HTCondor CE

Conveners: Gregory Thain, Todd Tannenbaum (University of Wisconsin Madison (US))

14:00 Introduction to the HTCondorCE

Speaker: Miron Livny

10 22 HTCondorCE.pdf

10 22 HTCondorCE.pptx

14:30 Token Transition Status and Discussion and CE future

Speaker: Brian Hua Lin (University of Wisconsin - Madison)

2022-10-11.token-update-ce-future.pdf

2022-10-11.token-update-ce-future.pptx

14:50 APEL Accounting and HEPScore

Speaker: Max Fischer (Karlsruhe Institute of Technology)

2022_10_HTCCE_Apel.pdf

client.cfg

parser.cfg

README.md

sender.cfg

15:10 Day to Day maintenance of a CE

Speaker: James Frey (University of Wisconsin Madison (US))

2022-10-11.ce-maintenance.pdf

15:30 Coffee Break

16:00 A Refreshing talk -- Tokens and the CE

Speaker: James Frey (University of Wisconsin Madison (US))

RefreshingTokensCE.pdf

RefreshingTokensCE.pptx

16:20 Session Wrap Up

Speaker: Miron Livny

16:50 → 17:20 Workshop Session

16:50 Phasing Out GSI Authentication in the CMS Submission Infrastructure

The CMS Computing Submission Infrastructure group manages and exploits a set of HTCondor pools to satisfy the experiment computing needs. The biggest of those pools, the so-called CMS Global Pool, currently aggregates nearly 400k CPU cores dynamically from pledged and opportunistic WLCG resources.
Historically, authentication among the diverse components of the infrastructure used the Grid Security Infrastructure (GSI), based on identities and X509 certificates. However, more modern authentication standards based on capabilities and tokens have emerged over the years. The CMS Submission Infrastructure group is in the process of phasing out the GSI part of its authentication layers in favor of IDTokens and Scitokens. In this contribution we will report on the current status of this migration, and our plans for the final GSI phase out.

Speaker: Nikos Tsipinakis (CERN)

HTCondor Week October 2022 Nikos Tsipinakis.pdf

Wednesday 12 October

09:00 → 11:40 Workshop Session

09:00 The HTC/HTCSS Data Story

Speaker: Gregory Thain

data-story.pdf

data-story.pptx

09:25 Submitting sets of jobs

Speaker: Todd Tannenbaum (University of Wisconsin Madison (US))

Submitting_Sets_of_Jobs.pdf

Submitting_Sets_of_Jobs.pptx

09:50 User data access and challenges

To complement grid production workflows, the National Analysis Facility (NAF) at DESY aims to provide individual users with an easy way without hurdles to process their data fast.
Since users are used to plain file paths, a number of storages are mounted as network mounts on the NAF HTCondor nodes. But with users showing a wide range of access patterns, we encountered occassionally problems with mounts and file access affecting the cluster stability.
To better understand the interplay between users, kernel and file system clients, we are now working on extending our monitoring. With the already available detailed event monitoring on the storage instances, we are going to add a complementary status overview of the execution side with the aim of a consolidated view the of storage and compute clusters.

Speakers: Christoph Beyer, Thomas Hartmann (Deutsches Elektronen-Synchrotron (DE))

DESY_NAF_HTC.pdf

Job Monitoring plans @ DESY

10:20 Coffee Break

10:50 Gravitational-Wave Computing - Needs and Trends

An Idiosyncratic Airing of Grievances, Condor-Style

Speaker: Peter Couvares (LIGO Laboratory / Caltech)

Couvares LIGO-G2200862-v2.pdf

11:15 Multi tenancy with htcondor

At final we have a large unifed cluster.
4 years ago there were three clusters sharing the same infrastructure (network and data center).
A dedicated cluster per team.

We decided to join forces and recreate a large cluster to better utilize the resources.

Docker volumes able us to share compute resources and allow different storage access per team.

In addition for that we are able to go further using docker networks and iptables. it allow us to share phisycal network without the risk of data leaks between teams.

We will discuss about the challenges that we currently facing different job sizes, small pool for fast response and negotiation cycle.

Speaker: David Handelman

11:40 → 12:30 Office Hours

One-to-one sessions with the HTCondor developers to discuss your setup and your issues

12:30 → 14:00 Lunch

14:00 → 17:30 Certificates, Tokens, and Identity: Past, Present, and Future

14:00 Introduction and Requirements / Goals for capabilities and security

Speaker: Miron Livny

10 22 Capabilities.pdf

10 22 Capabilities.pptx

14:30 WLCG Tokens / SciTokens Functionality in HTCSS

Speaker: James Frey (University of Wisconsin Madison (US))

WlcgAndSciTokens.pdf

WlcgAndSciTokens.pptx

15:00 TOFU - Trust on First Use Mechanism

Speaker: James Frey (University of Wisconsin Madison (US))

HTCondor-TOFU.pdf

15:20 HTCSS Threat Models - What are we protecting and why

Speaker: Gregory Thain

htcondor-threat-models.pdf

htcondor-threat-models.pptx

15:35 Coffee Break

16:05 Securing an HTCondor System with IDTOKENS

Speaker: Todd Tannenbaum (University of Wisconsin Madison (US))

Securing_with_IDTOKENS.pdf

Securing_with_IDTOKENS.pptx

16:45 Session Wrap up

Speaker: Miron Livny

Thursday 13 October

09:00 → 11:15 The HTCSS Access Point Story

09:00 Introduction and Story of the Access Point

Speaker: Miron Livny

09:20 Bring Your Own HPC Capacity Demo

Speaker: Todd Tannenbaum (University of Wisconsin Madison (US))

Bringing Your Own Capacity.pdf

Bringing Your Own Capacity.pptx

09:50 Direct Attachment of Execution Point to Access Point

Speaker: James Frey (University of Wisconsin Madison (US))

DirectAttachEPtoAP.pdf

DirectAttachEPtoAP.pptx

10:05 Scheduling at the Access Point

Speaker: Gregory Thain

schedling-at-the-ap.pdf

schedling-at-the-ap.pptx

10:25 Coffee Break

10:55 Wrap Up

Speaker: Miron Livny

11:15 → 11:40 Workshop Session: Discussion on Future Challenges

11:40 → 12:30 Office Hours

One-to-one sessions with the HTCondor developers to discuss your setup and your issues

12:30 → 14:00 Lunch

14:00 → 14:40 Workshop Session

14:00 HTCondor at CERN

Speaker: Ben Jones (CERN)

cerncondoreu22.pdf

cerncondoreu22.pptx

14:20 Elemento's AtomOS and the path to the true Cloud

Speaker: Mr Gabriele Gaetano Fronze' (INFN Torino (IT) and LIGO-Virgo-Kagra Collaboration (US/IT/JP))

Elemento Tech Pitch

14:40 → 15:30 Show your Toolbox

An experimental open session for attendees to briefly demonstrator any tools, tops or tricks they use to manage their use of HTCondor.

15:30 → 17:30 Tour

19:30 → 20:30 Social

Friday 14 October

09:00 → 12:30 The HTCSS Execution Point Story

09:00 Introduction and Story of the Execution Point

Speaker: Miron Livny

09:30 Why Wrappers are Evil

Speaker: Gregory Thain

Wrappers-gthain.pdf

Wrappers-gthain.pptx

10:00 Running jobs inside containers with the new Container Universe

Speaker: Gregory Thain

container-universe.pdf

container-universe.pptx

10:20 Job Isolation

Speaker: Gregory Thain

job-isolation.pdf

job-isolation.pptx

10:40 Coffee Break

11:10 Worker nodes with GPUs

Speaker: Todd Tannenbaum (University of Wisconsin Madison (US))

Using GPUs with HTCondor.pdf

Using GPUs with HTCondor.pptx

11:30 SSH to Job: How it works and why it sometimes doesn't

Speaker: James Frey (University of Wisconsin Madison (US))

SshToJob.pdf

SshToJob.pptx

11:50 Support for Job Checkpoints

Speaker: James Frey (University of Wisconsin Madison (US))

2022-self-checkpointing.pdf

2022-self-checkpointing.pptx

12:10 Wrap Up

Speaker: Miron Livny

12:30 → 12:50 Wrap up and Goodbye

Speaker: Chris Brew (Science and Technology Facilities Council STFC (GB))

HTCondor_Workshop_WrapUp.pdf

HTCondor_Workshop_WrapUp.pptx