Hadoop User forum

Wednesday 6 Dec 2017, 14:00 → 15:00 Europe/Zurich

31/S-028 (CERN)

14:00 → 14:40 HEP Data Processing with Apache Spark

This talk will cover recent developments on the use of Apache Spark for HEP Data Processing. In particular the progress related to the use of spark-root to read and process ROOT data using Spark will be covered with relevant examples coming from the work on the CMS BigData project.

Speaker: Viktor Khristenko (CERN)

Convert Features preengineered to actual images

Data Reduction Example: Take CMS Open Data AOD / extract Collection of Muons / Save dimuon mass

Examples of Collecting several histograms in one go

Feature Engineering

public CMS Open Data Muonia Simple invariant mass calculation and histograming

Spark Application (used w/ spark-submit) on CMS Open Data

sparkforhep_cern_06122017_template2.pdf