Choose timezone
Your profile timezone:
U.S. CMS Tier-3 Weekly Operations Meeting
→
US/Central
,
Description
Weekly meeting of the U.S. CMS Tier-3 Operations team and coordinators.
Join Zoom Meeting
https://ucsd.zoom.us/j/656739403
Meeting ID: 656 739 403
One tap mobile
+16699006833,,656739403# US (San Jose)
+12133388477,,656739403# US (Los Angeles)
Dial by your location
+1 669 900 6833 US (San Jose)
+1 213 338 8477 US (Los Angeles)
+1 669 219 2599 US (San Jose)
+1 971 247 1195 US (Portland)
+1 346 248 7799 US (Houston)
+1 720 928 9299 US (Denver)
+1 646 558 8656 US (New York)
+1 651 372 8299 US
+1 786 635 1003 US (Miami)
+1 253 215 8782 US
+1 267 831 0333 US
+1 301 715 8592 US
+1 312 626 6799 US (Chicago)
+1 646 518 9805 US (New York)
Meeting ID: 656 739 403
Find your local number: https://ucsd.zoom.us/u/ach3kx1m3O
Minutes of the U.S. CMS Weekly Tier-3 operations meeting on Tuesday, April 21, 2020
Indico: https://indico.cern.ch/event/907542/
Zoom Coordinates: https://ucsd.zoom.us/j/656739403
Attending: James, Dave, Carl, Doug, Kenyi
Agenda Topics
Round table of operations
-
Doug:
-
No progress with Baylor SAM redirector tests. They have opened access for Doug. Config typo somewhere?
-
Colorado - easing up stay-at-home next week in some cities.
-
-
Carl:
-
Ticket 145598 - Minnesota squid upgrade. NTR. Carl will poke them.
-
-
Kenyi:
-
Tier-3 k8s deployment: Kenyi is starting to write documentation.
-
SLATE team is still writing the CMS xcache app, maybe ready next week.
-
CMS Connect:
-
Will be a guinea pig to token-based authentication, so need HTCondor 8.9.6. Got hit by the matchmaking failure, so he reverted to 8.8.8. Known problem will be discussed in tomorrow's HTcondor call: https://indico.cern.ch/event/911575/
-
ACTION ITEM: James reply to e-mail from Rob Gardner (about a SLATE k8s tutorial for Tier-2 admins) that we should assume that Tier-2 site admins know nothing about k8s packages or installations to start with, so would have not already installed OKD et al. - DONE
-
-
HTCondor Upgrade Status
The queries (below) are actually checking the glidein factory tarball startd, not the startd in the batch system at the site itself. This would need an instrumented pilot job to check.
Action items:
-
UMD: Doug will verify the HTCondor version is (see below) by logging in.
-
Dave & James will inquire at OSG Production meeting the upgrade plans for hosted-CE’s and tarballs.
Results of HTCondor CE schedd and glidein tarball Queries
condor_status -schedd -pool :9619 -af CondorVersion| sort | uniq -c
condor_status -pool :9619 -af CondorVersion GLIDEIN_Factory | sort | uniq -c
Summary of results:
-
About ½ of OSG pilots are running 8.6.3 tarballs and ½ 8.8.8. Upgrade cycling its way through?
-
TAMU CE’s schedd is running 8.6.13
-
UMD CE’s schedd is running
-
Hosted CE’s:
-
hosted-ce35 is running 8.8.7
-
hosted-ce34 is running 8.8.6
-
hosted-ce26 and hosted-ce16 are running 8.6.13
-
Detailed Results:
kodiak-ce.baylor.edu
schedd:
1 $CondorVersion: 8.8.8 Mar 20 2020 PackageID: 8.8.8-1 $
startd's:
44 $CondorVersion: 8.6.3 May 08 2017 BuildID: 404928 $ OSG
32 $CondorVersion: 8.8.8 Feb 17 2020 BuildID: 496171 $ CERN-Prod
30 $CondorVersion: 8.8.8 Feb 17 2020 BuildID: 496171 $ OSG
heposg01.colorado.edu
schedd:
1 $CondorVersion: 8.8.8 Mar 20 2020 PackageID: 8.8.8-1 $
startd's:
207 $CondorVersion: 8.6.3 May 08 2017 BuildID: 404928 $ OSG
287 $CondorVersion: 8.8.8 Feb 17 2020 BuildID: 496171 $ CERN-Prod
1134 $CondorVersion: 8.8.8 Feb 17 2020 BuildID: 496171 $ OSG
245 $CondorVersion: 8.9.6 Feb 17 2020 BuildID: 496172 $ OSG
cmslpc-ce.fnal.gov
schedd:
1 $CondorVersion: 8.8.8 Mar 19 2020 BuildID: 498525 PackageID: 8.8.8-1 $
startd's:
cmslpc-ce2.fnal.gov
schedd:
1 $CondorVersion: 8.8.8 Mar 19 2020 BuildID: 498525 PackageID: 8.8.8-1 $
startd's:
deepthought.crc.nd.edu
schedd:
1 $CondorVersion: 8.8.8 Feb 17 2020 BuildID: 496171 $
startd's:
15 $CondorVersion: 8.6.3 May 08 2017 BuildID: 404928 $ OSG
2 $CondorVersion: 8.8.8 Feb 17 2020 BuildID: 496171 $ CERN-Prod
hosted-ce35.grid.uchicago.edu
schedd:
1 $CondorVersion: 8.8.7 Jan 06 2020 PackageID: 8.8.7-1 $
startd's:
34 $CondorVersion: 8.9.6 Feb 17 2020 BuildID: 496172 $ OSG
cms-grid0.hep.uprm.edu
schedd:
1 $CondorVersion: 8.9.6 Mar 19 2020 PackageID: 8.9.6-1 $
startd's:
61 $CondorVersion: 8.6.3 May 08 2017 BuildID: 404928 $ OSG
145 $CondorVersion: 8.8.8 Feb 17 2020 BuildID: 496171 $ OSG
bonner06.rice.edu
schedd:
1 $CondorVersion: 8.8.8 Mar 20 2020 PackageID: 8.8.8-1 $
startd's:
ruhex-osgce.rutgers.edu
schedd:
1 $CondorVersion: 8.8.8 Feb 17 2020 BuildID: 496171 $
startd's:
38 $CondorVersion: 8.6.3 May 08 2017 BuildID: 404928 $ OSG
14 $CondorVersion: 8.8.8 Feb 17 2020 BuildID: 496171 $ CERN-Prod
12 $CondorVersion: 8.8.8 Feb 17 2020 BuildID: 496171 $ OSG
hosted-ce34.grid.uchicago.edu
schedd:
1 $CondorVersion: 8.8.6 Dec 03 2019 PackageID: 8.8.6-1.1 $
startd's:
ce01.brazos.tamu.edu
schedd:
1 $CondorVersion: 8.6.13 May 16 2019 $
startd's:
top.ucr.edu
schedd:
1 $CondorVersion: 8.8.8 Mar 20 2020 PackageID: 8.8.8-1 $
startd's:
hepcms-1.umd.edu
schedd:
1 $CondorVersion: 8.6.13 Dec 06 2018 $
startd's:
15 $CondorVersion: 8.8.8 Feb 17 2020 BuildID: 496171 $ CERN-Prod
hosted-ce26.grid.uchicago.edu
schedd:
1 $CondorVersion: 8.6.13 May 16 2019 $
startd's:
umiss001.hep.olemiss.edu
schedd:
1 $CondorVersion: 8.8.8 Mar 20 2020 PackageID: 8.8.8-1 $
startd's:
44 $CondorVersion: 8.6.3 May 08 2017 BuildID: 404928 $ OSG
5 $CondorVersion: 8.8.8 Feb 17 2020 BuildID: 496171 $ CERN-Prod
60 $CondorVersion: 8.8.8 Feb 17 2020 BuildID: 496171 $ OSG
hosted-ce16.grid.uchicago.edu
schedd:
1 $CondorVersion: 8.6.13 Oct 30 2018 $
startd's:
There are minutes attached to this event.
Show them.
The agenda of this meeting is empty
