Big Data processing and analysis challenges in mega-science experiments

29 Jan 2015, 09:00 → 31 Jan 2015, 15:00 Europe/Moscow

NRC KI and JINR

Alexei Klimentov (Brookhaven National Laboratory (US)), Vladimir Korenkov (Joint Inst. for Nuclear Research (RU))

How to come to NRC KI NRC_KI_scheme_eng_.doc NRC_KI_scheme_eng_.pdf

how to find LIT JINR map_JINR.bmp

link to hotel aerostar http://www.aerostar.ru

link to hotel Pekin http://hotelpeking.ru/en/

link to Mariott (Tverskaya Yamskaya) http://marriottmoscowtverskaya.ru

Схема проезда к НИЦ КИ Схема_расположения_Курчатовского_центра_культуры_и_проходной_НИЦ_«Курчатовский_институт».docx Схема_расположения_Курчатовского_центра_культуры_и_проходной_НИЦ_«Курчатовский_институт».pdf

Thursday 29 January

09:45 → 10:00 Welcome. Dr.V.Velikhov, NRC-KI vice-director; Dr.V.Demin, NBICS vice-director

Introduction and logistics. A.Klimentov NRC-KI-workshop_Jan29_2015.pdf NRC-KI-workshop_Jan29_2015.pptx

10:00 → 11:50 Database Technologies

Convener: Dr Maria Grigorieva (NRC KI)

slides

• database_discussion.pdf
• Gancho_Dimitrov_PanDA_DB_implementation_Moscow_workshop_29thJan2015.pdf
• HStorage_development_state.pdf
• lassnig_db_teg.pdf
• Our_experience_in_NoSQL_integration.pdf
• Our_experience_in_NoSQL_integration.pptx
• swc20120614_lassnig.pdf

Topics for discussion:

1. Our experience of the developing of Hybrid SQL/NoSQL Storage, and NoSQL database integration in BigPanDA monitor

2. Database performance tests : SQL - NoSQL, NoSQL - NoSQL

3. Technology evaluation tests results for NoSQL databases: MongoDB, HBase, Cassandra, Dremel, CouchDB, MariaDB

4. Experience of using Hadoop/Spark/MapReduce in PanDA Infrasctucture. Use cases.

5. Foreseen performance and possible changes in PanDA Oracle archived database schema during/after the LHC Run2

6. Query routing strategy in BigPanDA applications (BigPanDA Monitor in particular)

7. How to implement cross database requests in heterogeneous architecture

8. Strategies of the data modelling for NoSQL databases 

10:00 → 11:00 EOS and Data Store

Convener: Eygene Ryabinkin (National Research Centre Kurchatov Institute (RU))

10:00 → 11:50 PanDA@NRC-KI

Convener: Ruslan Mashinistov (Russian Academy of Sciences (RU))

10:00 PanDA @ NRC-KI

PanDA instance installation and commissioning status PanDA for ATLAS PanDA beyond LHC and HEP

Speaker: Ruslan Mashinistov (Russian Academy of Sciences (RU))

Slides PanDA_instance_@_RRC-KI_-_Jan29.pdf

10:50 Discussion

PanDA beyond ATLAS : ALICE Do we need version w/o VOMS, Grid Services… Do we need Light weight DDM Staging and I/O matters

10:00 → 11:00 webFTS

Convener: Mr Andrey Kiryanov (B.P. Konstantinov Petersburg Nuclear Physics Institute - PNPI ()

10:00 Introduction: current status of FTS3/WebFTS

Speaker: Oliver Keeble (CERN)

Slides Presentation1.pdf

10:10 FTS readiness for transfers to/from non-Grid resources (VOMS/non-VOMS credentials, etc.)

10:20 Impact of Federated Identity support in WebFTS for non-Grid users

Slides http://cern.ch/kiryanov/WebFTS-IdF.pdf

10:30 FTS & HPC: old/new APIs, lightweight clients, Python client library for PanDA integration

10:40 Usage scenario for Titan: requirements from both sides

Speakers: Mr Andrey Kiryanov (B.P. Konstantinov Petersburg Nuclear Physics Institute - PNPI (), Danila Oleynik (Joint Inst. for Nuclear Research (RU))

10:50 Feature requests for FTS/WebFTS

11:00 → 11:50 Meeting with Big Data Lab PhD students (round table)

Conveners: Alberto Pace (CERN), Dr Markus Schulz (CERN), Massimo Lamanna (CERN), Prof. Shantenu Jha (Rutgers U), Simon C. LIN (Academia Sinica)

round table panel and participants list Students_and_PhD_students_list_for_round_table-2.docx Students_and_PhD_students_list_for_round_table-2.pdf

11:50 → 13:30 BigPanDA

Convener: Dr Alexei Klimentov (Brookhaven National Laboratory (US))

11:50 Introduction

11:55 webFTS highlights

Speakers: Mr Andrey Kiryanov (B.P. Konstantinov Petersburg Nuclear Physics Institute - PNPI (), Dr Markus Schulz (CERN), Oliver Keeble (CERN)

12:05 Database session highlights

Speakers: Gancho Dimitrov (CERN), Dr Maria Grigorieva (NRC KI), Ms Marina Golosova (NRC KI), Dr Mario Lassnig (CERN)

Slides Status_report.pdf Status_report.pptx

12:15 PanDA session highlights

Speakers: Danila Oleynik (Joint Inst. for Nuclear Research (RU)), Kaushik De (University of Texas at Arlington (US)), Ruslan Mashinistov (Russian Academy of Sciences (RU))

Slides panda-session-summary-jan15.pdf panda-session-summary-jan15.pptx

12:25 PanDA HPC pilot

Speakers: Danila Oleynik (Joint Inst. for Nuclear Research (RU)), Prof. Shantenu Jha (Rutgers U)

Slides PanDA_Pilot_HPC_(KI).pdf PanDA_Pilot_HPC_(KI).pptx

12:45 webFTS integration with PanDA

Speakers: Mr Andrey Kiryanov (B.P. Konstantinov Petersburg Nuclear Physics Institute - PNPI (), Danila Oleynik (Joint Inst. for Nuclear Research (RU)), Kaushik De (University of Texas at Arlington (US)), Oliver Keeble (CERN)

13:30 → 14:30 Lunch

14:30 → 16:50 Modeling Distributed Computing Systems

Conveners: Dr Eugene Burnaev (IPPI RAS), Kaushik De (University of Texas at Arlington (US)), Prof. Shantenu Jha (Rutgers U)

14:30 → 16:30 visit to I.V.Kurchatov museum and NRC KI Supercomputing center

doodle poll to visit I.V.Kurchatov museum http://doodle.com/hmwa3377raash3ne

doodle poll to visit NRC KI Tier1 and supercomputi http://doodle.com/bmksyczw8hye5waw

14:30 → 16:00 Nano-,bio-,information and cognitive technologies Institute seminar

14:30 "PanDA, A New Paradigm for Computing in HEP"

Speaker: Kaushik De (University of Texas at Arlington (US))

Slides panda-ki-jan15.pdf panda-ki-jan15.pptx

15:10 CERN and Computing

Speaker: Alberto Pace (CERN)

Slides 2015-01-29-Alberto-Pace-Moscow-Big_Data_challenges_in_mega-science-V03.pdf 2015-01-29-Alberto-Pace-Moscow-Big_Data_challenges_in_mega-science-V03.pptx

17:00 → 17:20 Leave to JINR by bus from NRC KI

Friday 30 January

10:00 → 10:15 Welcome from Prof.N.Rusakovich, JINR Chief Scientific Secretary

10:15 → 11:10 Modeling in mega-projects

10:15 NICA mega-project

Speaker: Prof. O.Rohachevsky (JINR)

10:35 Modeling of grid-cloud systems

Speaker: Prof. Gennady Ososkov (JINR)

11:10 → 11:25 coffee break

11:25 → 12:25 BigPanDA project

11:25 Project overview : Growing PanDA ecosystem, PanDA beyond ATLAS, prospects for ALICE, COMPASS, NICA.

Speaker: Kaushik De (University of Texas at Arlington (US))

Slides panda-dubna-jan15.pdf panda-dubna-jan15.pptx

11:40 Towards an Abstractions-driven and Standards-based Next-generation of Distributed Cyberinfrastructure

Speaker: Prof. Shantenu Jha (Rutgers University)

12:05 Network awareness

Speaker: Artem Petrosyan (Joint Inst. for Nuclear Research (RU))

Slides network_awareness_2015.01.30.pdf network_awareness_2015.01.30.pptx

12:25 → 14:00 Technology trends

12:25 webFTS and FTS3

Speaker: Oliver Keeble (CERN)

Slides FTS3_Trends_v3.pdf

12:45 EOS

Speakers: Alberto Pace (CERN), Massimo Lamanna (CERN)

Slides DUBNA20150129.pdf

13:05 ORACLE and Hadoop/HBASE in ATLAS

Speakers: Gancho Dimitrov (CERN), Dr Mario Lassnig (CERN)

Slides Gancho_Dimitrov_Oracle_in_ATLAS_Dubna_Workshop30thJan2015_2.pdf hadoop_atlas.pdf

13:40 CLAVIRE Highlights

Speakers: Prof. Alexander Boukhanovsky (ITMO), Dr Denis Nasonov (ITMO)

Slides CLAVIRE_2015.pdf CLAVIRE_2015.pptx

14:00 → 15:00 lunch

15:00 → 15:30 Monitoring

15:00 NoSQL for the storage of monitoring data:

Speaker: Ivan Kadochnikov (Joint Inst. for Nuclear Research (RU))

15:15 Monitoring in DIRAC environment for the BES3 experiment

Speaker: Igor Pelevanyuk (Joint Inst. for Nuclear Research (RU))

Slides

• JINR_Meeting_BESIII_Dark.pdf
• JINR_Meeting_BESIII_Light.pdf
• JINR_Meeting_BES_Light.pptx

15:30 → 16:20 Reports on Tier1

15:30 ASGC TW Tier-1

Speaker: Simon Lin (Academia Sinica (TW))

Slides 20150130-ASGC-SimonLin.pdf 20150130-ASGC-SimonLin.pptx

15:50 NRC KI RF Tier-1

Speaker: Dr Vassily Velikhov (NRC KI)

16:05 JINR Tier-1

Speaker: Dr Tatiana Strizh (JINR)

16:20 → 16:35 tea break

16:35 → 16:55 Presentation of the heterogeneous cluster of LIT

Speaker: Dr D.Podgainy (JINR)

16:55 → 17:30 Excursion in LIT

18:00 → 21:00 Beer party

doodle poll http://doodle.com/b7qnir6hw4h2epwk#table

Saturday 31 January

09:30 → 13:00 Technical splinter meetings

09:30 Distributed Modeling splinter meeting