CMS Big Data Science Project

Name: CMS Big Data Science Project
Start: 2017-08-09T16:00:00+02:00
End: 2017-08-09T17:30:00+02:00
Location: CERN

Wednesday 9 Aug 2017, 16:00 → 17:30 Europe/Berlin

513/R-068 (CERN)

Matteo Cremonesi (Fermi National Accelerator Lab. (US)), Oliver Gutsche (Fermi National Accelerator Lab. (US))

Description

FNAL room: Dark Side-WH6NW - Wilson Hall 2nd fl North East

CERN room: 513-R-068

Instructions to create a light-weight CERN account to join the meeting via Vidyo:

If not possible, people can join the meeting by the phone, call-in numbers are here:

The meeting id is hidden below the Videoconference Rooms link, but here it is again:

Hide

attendance: Vagg, David Lange, Thanat Jatuphattharachat, Lukasz, Saba, Siewan, Matteo, JimP, Igor, OLI, Jorge, Pratyush
ACAT
- spark-root
- new connector
- performance
presentations
- root4j/spark-root
  - clean separation almost done
  - Pratyush started classes, Victor will pick it up again
  - GROOT test case, but would be usable to have root like plotting functionality from JAVA and scala
- Distributed query service
  - Investigated: Apache Zookeeper, Mesos, Marathon
  - Apache Zookeeper was investigated more closely
    - Can build our workflow (FemtoCode) on top of Zookeeper
Discussion
- performance
  - dedicated hadoop cluster (3 nodes, 2 data nodes)
    - 10 Gbps network
    - 3x 32 cores
  - compare reading from HDFS on data nodes or from EOS over the network
    - HDFS replication factor is 2, data is always local
    - conclusion: connector does not add overhead
    - question: walltime investigation, interesting, will be done later
  - network performance
    - tried to put ROOT files on CERNBOX, no, maybe just a test
  - continue discussion offline
plan
- meet next week, August 16, to finalize ACAT content

There are minutes attached to this event. Show them.

Choose timezone