IML Machine Learning Working Group - Parallelized/Distributed Machine Learning

Friday 24 Feb 2017, 15:00 → 17:35 Europe/Zurich

40/S2-C01 - Salle Marie Sklodowska-Curie (CERN)

1 News and group updates

Speakers: Lorenzo Moneta (CERN), Michele Floris (CERN), Paul Seyfert (Universita & INFN, Milano-Bicocca (IT)), Dr Sergei Gleyzer (University of Florida (US)), Steven Randolph Schramm (Universite de Geneve (CH))

StevenSchraamm-IML-news.pdf

2 Internally-Parallelized Boosted Decision Trees

Speaker: Andrew Mathew Carnes (University of Florida (US))

BDT_Parallel_IML.pdf

3 Rapid development platforms for machine learning

Speaker: Dr Andrew Lowe (Hungarian Academy of Sciences (HU))

lowe_tools2.pdf

4 Distributed Deep Learning using Apache Spark and Keras (see materials)

Data parallelism is an inherently different methodology of optimizing parameters. The general idea is to reduce the training time by having n workers optimizing a central model by processing n different shards (partitions) of the dataset in parallel. In this setting we distribute n model replicas over n processing nodes, i.e., every node (or process) holds one model replica. Then, the workers train their local replica using the assigned data shard. However, it is possible to coordinate the workers in such a way that, together, they will optimize a single objective during training and as a result, reduce the wall clock training time. There are several approaches to achieve this, and these will be discussed in greater detail in the materials below.

Speaker: Joeri Hermans (Maastricht University (NL))

Distributed Deep Learning with Apache Spark and Keras

MNIST example and code

Theory and MNIST application

5 Parallelization in Machine Learning with Multiple Processes

Speakers: Gerardo gutierrez (ITM), Omar Andres Zapata Mesa (University of Antioquia & Metropolitan Institute of Technology)

TMVA_ROOTMpi.pdf

6 Minutes

IML-2017-02-24-minutes.txt