We have updated all of our gatekeepers to the newest OSG release, 3.3.25.  In conjunction with the update we worked with an OSG team to establish the new lcmaps-voms mapping described at this URL:
https://twiki.opensciencegrid.org/bin/view/Documentation/Release3/InstallLcmapsVoms

This has been running at AGLT2 now for a week without any issues that we are aware of.

The released set of voms mappings, in /usr/share/osg/voms-mapfile-default, is not yet complete.  In particular the atlas mappings are not yet updated.  However, 2 over-ride files, and two "ban" files, located in /etc/grid-security, are searched first for account mappings prior to searching the default file.  Only the first mapping, based upon the First FQAN of the presented certificate, or the identity portion of the presented certificate, is used to do the account mapping.  We have put in place the following voms override file at AGLT2

[root@gate03 ~]# cat /etc/grid-security/voms-mapfile
"/atlas/usatlas/Role=production/Capability=NULL" usatlas1
"/atlas/usatlas/Role=software/Capability=NULL" usatlas2
"/atlas/usatlas/Role=lcgadmin/Capability=NULL" usatlas2
"/atlas/Role=lcgadmin/Capability=NULL" usatlas2
"/atlas/usatlas/Capability=NULL" usatlas3
"/atlas/Role=production/Capability=NULL" usatlas1
"/atlas/Capability=NULL" usatlas4
"/atlas/calib-muon/Role=NULL/Capability=NULL" muoncal
"/osg/ligo/Role=NULL/Capability=NULL" ligo
"/fermilab/*" fermilab
"/cms/*" uscms01

We also have a /etc/grid-security/grid-mapfile to map the specific DN in use at AGLT2 for the muon calibration effort.

[root@gate03 ~]# cat /etc/grid-security/grid-mapfile
"/DC=ch/DC=cern/OU=Organic Units/OU=Users/CN=diehl/CN=490810/CN=Edward Diehl" muoncal

Once AGLT2 has modified dCache to use a similar mechanism, we will no longer be dependent on GUMS and will turn off our service.  OSG in the future will deprecate and then entirely remove their GUMS support.

The dCache pgsql DBs crashed out over the past weekend, causing all jobs to fail.  It seems that the auto-vacuum is not working for our pgsql 9.5.7 instance.  This is under investigation, but as of 3pm Monday, we were back in business.  As a reminder we are running dCache 3.8.11.

AGLT2 will go offline at Noon Friday for a complete power outage in the UM server room.  We hope to be back up by day's end on Monday after multiple maintenance items are completed.  The switcher2 set our queues offline at Noon today.

singularity is installed on all AGLT2 WN.