US ATLAS Computing Integration and Operations

Wednesday 28 Oct 2015, 13:00 → 15:00 US/Eastern

virtual room (your office)

Notes and other material available in the US ATLAS Integration Program Twiki

13:00 → 13:15 Top of the Meeting

Speakers: Michael Ernst, Robert William Gardner Jr (University of Chicago (US))

13:15 → 13:25 Production

Speakers: Kaushik De (University of Texas at Arlington (US)), Mark Sosebee (University of Texas at Arlington (US))

shift-summary-10_21_15.pdf

shift-summary-10_28_15.pdf

13:25 → 13:30 Data Management

Speaker: Armen Vartapetian (University of Texas at Arlington (US))

151028_DataManagement_Armen.pdf

13:30 → 13:35 Data transfers

Speaker: Hironori Ito (Brookhaven National Laboratory (US))

13:35 → 13:40 Networks

Speaker: Dr Shawn McKee (University of Michigan ATLAS Group)

13:40 → 13:45 FAX

Speakers: Ilija Vukotic (University of Chicago (US)), Wei Yang (SLAC National Accelerator Laboratory (US))

A few points concerning access to the analytics clusters:

1. cluster to use at the moment: cl-analytics.mwt2.org. Other clusters are: CERN - aianalytics01.cern.ch and MWT2 - kibana.mwt2.org
2. kibana is on port 5601 and access to it is open from anywhere: http://cl-analytics.mwt2.org:5601
3. to see the state of cluster use HQ (http://cl-analytics.mwt2.org:9200/_plugin/HQ/) or Marvel (http://cl-analytics.mwt2.org:9200/_plugin/marvel/). Both pages should be directly accessible from CERN and UC. If you need them open for some other place, let me know.
4. to run a simple query (showing number of jobs per cloud) using curl:

curl -XPOST 'cl-analytics.mwt2.org:9200/jobs_archive*/_search?pretty' -d '
{
  "size": 0,
  "aggs": {
    "group_by_cloud": {
      "terms": {
        "field": "cloud"
      }
    }
  }
}'

5. to run a simple query using python (returns unique PerfSONAR IPs):

from elasticsearch import Elasticsearch
es = Elasticsearch([{'host':'cl-analytics.mwt2.org', 'port':9200}])
usrc={
    "size": 0,
    "aggregations": {
       "unique_vals": {
          "terms": {
             "field": "@message.src",
             "size":1000
          }
       }
    }
}
res = es.search(index="network_weather-2015-10-25", body=usrc, size=10000)
for tag in res['aggregations']['unique_vals']['buckets']:
    print tag['key’]

6. both curl and python access are possible only from CERN and UC.  Again, let me know if you need access from elsewhere.

13:55 → 14:55 Site Reports

13:55 BNL

Speaker: Michael Ernst

14:00 AGLT2

Speakers: Robert Ball (University of Michigan (US)), Dr Shawn McKee (University of Michigan ATLAS Group)

14:05 MWT2

Speakers: David Lesny (Univ. Illinois at Urbana-Champaign (US)), Lincoln Bryant (University of Chicago (US))

2015.28 MWT2 Site Report.pdf

14:10 NET2

Speaker: Prof. Saul Youssef (Boston University (US))

14:15 SWT2-OU

Speaker: Dr Horst Severini (University of Oklahoma (US))

14:20 SWT2-UTA

Speaker: Patrick Mcguigan (University of Texas at Arlington (US))

14:25 WT2

Speaker: Wei Yang (SLAC National Accelerator Laboratory (US))

14:55 → 15:00 AOB