CMS Big Data Science Project

Tuesday 7 Jun 2016, 11:00 → 12:00 US/Central

FNAL room: Bullpen-WH7XE - Wilson Hall 7th fl Crossover East

CERM room:

 

Instructions to create a light-weight CERN account to join the meeting via Vidyo:

• https://account.cern.ch/account/Externals/

If not possible, people can join the meeting by the phone, call-in numbers are here:

• http://information-technology.web.cern.ch/services/fe/howto/users-join-vidyo-meeting-phone

The meeting id is hidden below the Videoconference Rooms link, but here it is again:

• 10502145

attendance: Alexey, Bo, Mike Hildreth, OLI, Kevin Lannon, Matthew Link, Saba, Cristina, OLI, Jason Slaunwhite, JimP, Nhan

 

agenda: https://indico.cern.ch/event/539648/

 

• news
  • Welcome Illia Cremer from Intel
    • Intel through Openlab is working on a project together with CMS to investigate the Dark Matter use case (and maybe others) on Spark.
    • Illia is starting to join our meetings to see how this would fit into the openlab project, already sent him a summary with github pointers
  • Matthew Link, student of Kevin Lannon from Notre Dame, is giving a short introduction of using Vertica for HEP
  • Then we go through the projects

 

• Vertica for HEP
  • overall time schedule
    • the week, install Vertica and get it running
    • program ends End of July
  • What do you need?
    • Pointed Jason to tree converter github project
      • looks very useful
      • nothing specific
  • JimP
    • Vertica and Spark very similar, except that files can be skipped in Vertica ➜ database is indexed so that whole files can be skipped
    • Does Vertica have some special key setup strategy?
      • Vertica does not have fully multi-dimensional indexing, but the main variable is indexed
      • Spark is not good with table joins 
  • general question: what about using a database store for Spark ➜ continue on slack
  • other high speed SQL engines: Impala, Drill, 
    • if Vertica specializes in joining tables on the fly each of them sorted by a different variable ➜ could be uniquely suited for HEP
  • JimP
    • if they plan to interface with ROOT, thinks they can use the histogrammer, maybe collaborate on that to use histogrammer in between Vertica and ROOT

 

• Princeton workflow
  • Cristina will work on Scala code conversion (cannot be completed in 2 weeks)
    • Jim helped with TLorentzVectors, dealing with problems of vectors and list, plan to finish a big part this week
    • able to use histogrammer
  • work on histogramming, need plotting software in more useable state ➜ JimP and Alexey
    • almost finished the scala to bokah implementation
    • switching to python to bokah
    • possible to do stack plots, first implementation
  • follow up CMSSW to AVRO by following with Nhan to make it happen at Princeton and LPC ➜ Matteo
    • useful discussion
    • Different approaches for CHEP and after
      • for CHEP: 
        • small proof-of-concept CMSSW conversion at Princeton
        • conversion of full dataset: large scale test and then load AVRO into Princeton cluster
      • after CHEP: Nhan suggested to find format in between BACON and flat-flat ntuple
        • going directly flat-flat from CMSSW directly is not the way to go, we at least want to have vectors
        • idea is not to stick with the BACON format for that step
        • hybrid facility concepts
        • maybe also investigating different formats than AVRO
      • even farther out: some discussion with Chris Jones: drive CMSSW from Spark
        • would allow analyst to do everything in Spark, even the CMSSW run
  • Issues
    • Cristina does not see all files in HDFS at Princeton, couldn’t find the data, but could find all MC
    • JimP is looking into HDFS libraries linked into CMSSW
• NERSC workflow
  • following Cristina’s Scala work implementing analysis code
  • rework HDF5 conversion process
    • slow performance with python code, found problem (unnecessary loop per leaf)
      • reduced form 53s to 4s to convert 800 kB file
      • probably still too slow
      • rootpy (instead of pyroot) might be faster and could help first step of filling numpy arrays
    • two layouts:
      • column based: taking more space, 1.2 MB for the 800k ROOT input, no compression or cleverness
      • matrix oriented
        • form matrix of all the floats and all the integers: 937 kB ➜ uncompressed, significantly better than ROOT
    • third format: group variables that are accessed together 
      • similar to bucket structure in ROOT file
  • plan: use larger datasets and continue tests/analysis code implementation
    • plan to test on larger files, see what gives better reading performance of two HDF5 format implementation
      • need to coordinate with analysis needs
    • also will look at performance to read HDF5 files into Spark
    • needs more data at NERSC, ROOT files
    • in the end, wants to look at SparkR
  • OLI: get disk space at Fermilab
  • Saba listened to webinar introducing Spark 2, they are thinking of using R

 

• Conference
  • Grace-Hopper conference: presentation was accepted, it is a presentation ➜ Saba
  • IEEE Big Data is in December in Washington D.C. ➜ we could also write an abstract for the whole use case ➜ JimP is going
  • SuperComputing in November: workshop paper, poster and demo
  • CHEP in October
  • Strange Loop (WebCS conference in September in St. Louis) ➜ accepted for a talk

 

• next meeting in two weeks, June 22
  • OLI not available, Matteo will run the meeting
  • goals:
    • Princeton workflow: have first version of analysis code ready in Scala for people to run (Nhan, Illia and OLI are all interested in running the code)
    • NERSC workflow: performance tests and tuning on larger files

11:00 → 11:10 Introduction: Vertica for HEP

Speakers: Kevin Patrick Lannon (University of Notre Dame (US)), Matthew Link

VerticaIntro.pdf