CMS Big Data Science Project

→ Europe/Berlin
513/R-068 (CERN)

513/R-068

CERN

19
Show room on map
Matteo Cremonesi (Fermi National Accelerator Lab. (US)), Oliver Gutsche (Fermi National Accelerator Lab. (US))
Description

FNAL room: Dark Side-WH6NW - Wilson Hall 2nd fl North East

CERN room: 513-R-068

Instructions to create a light-weight CERN account to join the meeting via Vidyo:

If not possible, people can join the meeting by the phone, call-in numbers are here:

The meeting id is hidden below the Videoconference Rooms link, but here it is again:

  • 10502145
Hide

  • attendance: Vagg, David Lange, Thanat Jatuphattharachat, Lukasz, Saba, Siewan, Matteo, JimP, Igor, OLI, Jorge, Pratyush

  • ACAT

    • spark-root
    • new connector
    • performance

  • presentations

    • root4j/spark-root
      • clean separation almost done
      • Pratyush started classes, Victor will pick it up again
      • GROOT test case, but would be usable to have root like plotting functionality from JAVA and scala
    • Distributed query service
      • Investigated: Apache Zookeeper, Mesos, Marathon
      • Apache Zookeeper was investigated more closely
        • Can build our workflow (FemtoCode) on top of Zookeeper

  • Discussion

    • performance
      • dedicated hadoop cluster (3 nodes, 2 data nodes)
        • 10 Gbps network
        • 3x 32 cores
      • compare reading from HDFS on data nodes or from EOS over the network
        • HDFS replication factor is 2, data is always local
        • conclusion: connector does not add overhead
        • question: walltime investigation, interesting, will be done later
      • network performance
        • tried to put ROOT files on CERNBOX, no, maybe just a test
      • continue discussion offline

  • plan

    • meet next week, August 16, to finalize ACAT content
There are minutes attached to this event. Show them.