CMS Big Data Science Project

Wednesday 11 May 2016, 10:00 → 11:00 US/Central

Fermilab room: DIR/ Snake Pit-WH2NE - Wilson Hall 2nd fl North East

CERM room:

 

Instructions to create a light-weight CERN account to join the meeting via Vidyo:

• https://account.cern.ch/account/Externals/

If not possible, people can join the meeting by the phone, call-in numbers are here:

• http://information-technology.web.cern.ch/services/fe/howto/users-join-vidyo-meeting-phone

The meeting id is hidden below the Videoconference Rooms link, but here it is again:

• 10502145

attendance: Alexey, Matteo, Cristina, Saba, JimP, KevinL, Nhan

 

• news
  • JimP
    • Report from Openlab Machine Learning and Data Analytics workshop April 29 (https://indico.cern.ch/event/514434/other-view?view=standard)
      • Goal was to get physics and industry together, physics presented needs and project proposals ➜ most of them said: we need what algorithms to apply, we need to do the faster
      • Industry message was: we have great algorithms, come use our stuff
      • CERN-IT has a spark cluster for purpose of analysis, could be a good second test bench for us
    • Report from HSF meeting May 2-4 (https://indico.cern.ch/event/496146/timetable/)
      • HEP Software Foundation 
      • 3rd annual meeting, ramping up
      • large discussion if HSF should be a foundation like the apache foundation to hold copy rights for for example ROOT
      • packaging systems was a big topic ➜ spack 
      • no big data talks, seems nobody is trying what we are working on
      • there was a lot of talk about industry talk
      • talks about tensor flow for machine learning
  • Scala workshop today, 1 P, roundtable, same vidyo as for this meeting ➜ there is a slack channel #tutorial1-scala-spark
  • Unrelated: Fermilab machine learning slack forum: https://hepmachinelearning.slack.com/signup
  • google cloud big data blog: https://cloud.google.com/blog/big-data/
  • Introduced Nhan ➜ invited Nhan to slack

 

• discussion status
  • FY16: feasibility studies of open source big data technology applicability to HEP analysis and comparison to traditional ROOT based analysis
    • formed team with experts and scientists, meeting biweekly
    • identified use case -> CMS monoTop dark matter analysis in the hadronic channel
    • identified first technology: Apache Spark
    • working on two thrusts
      • general feasibility study using Apache Spark -> first parts of the analysis migrated to Spark, running on Spark cluster at Princeton and producing plots
      • studying use case on HPC installations using NERSC's SparkHDF5 API's and SparkR -> first prototype functional

 

• Next steps and action items:
  • Milestone 1: define completely distributed skimming that reads root files and writes out to distributed parquet files
    • implement simplified multi-sample plot comparing data with MC using python
    • organize Scala workshop DONE
      • JimP suggests two separate parts, one as a Scala introduction and Root2Scala tools, one specifically for the migration of the analysis code
    • everyone is checking out Alexey’s workflow in github
  • Milestone 2: get the data from CERN to the Princeton Big Data cluster and setup xrootd server
    • convert the files into AVRO format using snappy (turn it on, it’s a ROOT2AVRO switch)
      • 40% larger than root files
      • Want to do the conversion and upload to HDFS in one step
      • Alexey will run the conversion and store output on HDFS
    • xrootd server installation at Princeton
  • NERSC
    • next step: finish the python converter
    • Saba is not yet talking with Matteo and Cristina about analysis code ➜ wants to attend JimP’s Scala workshop to migrate C++ analysis code to Scala

 

• Progress:
  • Milestone 1: define completely distributed skimming that reads root files and writes out to distributed parquet files
    • final goal is to use EDM files (MINIAOD) ➜ Nhan will work with JimP
      • our ROOT 2 Avro converter is not able right now to convert MINIAOD into Avro
      • best is to write a small Analyzer to write out the AVRO converter ➜ avoid to reimplement code in Scala/Java
        • we produce the same data structure but in a different file format
      • there will be an external dependency to Avro-C
    • Matteo & Cristina workflow still stand alone
      • JSON input, python to produce parquet files
      • python script to read parquet files and use matplotlib to make plots
      • stacked plots need matplotlib update, could also be done manually in matplotlib ➜ 
    • Alexey’s workflow on Princeton cluster
      • input JSON files
        • ROOT2AVRO converter produces broken JSON (NAN) ➜ Jimp and Alexey will investigate
      • output parquet files
      • plot with matplotlib
      • tries to use JimP’s histogrammer, saving into JSON fragment does not seem to work (JimP needs to investigate)
    • concern: not running the same workflow
      • need to converge as soon as possible
      • want to have a clear plan by the end of this week
    •  goal is to move to princeton and converge into one workflow
      • AVRO input
      • Scala code
      • store parquet files in HDFS
      • then we need to figure out how to access the parquet files and make plots
      • goal of this week is to sit down after the introductory Scala workshop to move the code and everything over to Scala and Princeton
      • Saba will also work at Princeton 
  • Milestone 2: get the data from CERN to the Princeton Big Data cluster and setup xrootd server
    • Data is at Princeton 
    • AVRO conversion is done, but without snappy ➜ needs to be redone
    • xrootd server setup needs support from Brian ➜ postponed, also because problems in direct ROOT 2 Scala need to be solved
  • Milestone 3: implement analysis using SparkHDF5 API and SparkR
    • python code to read ROOT files and write HDF5 files is evolving
      • could look into same approach using EDAnalyzer to write out HDF5 files later
    • currently reading RDD out of HDF5 file, talking to NERSC people
  • Role assignment
    • Matteo is delivering the physics side and Cristina and Nhan are working for him
    • JimP is developing the need software elements for Spark and other technologies and Alexey and Saba are helping
    • Alexey is integration at Princeton, pulling everything together and making it work on the cluster
    • Saba is responsible for the SparkHDF5/SparkR use case
      • is also contributing to the Princeton use case and execute code
      • will get help from the team for the NERSC development

 

• Next steps and action items
  • want to have a clear plan by the end of this week how to converge into one workflow at Princeton
  • want to have stacked plots done at Princeton
  • Saba wants to write some of the analysis code in Scala to read HDF5 and analyze it in SparkR