Big Data processing and analysis challenges in mega-science experiments

29 Jan 2015, 09:00 → 31 Jan 2015, 15:00 Europe/Moscow

NRC KI and JINR

Alexei Klimentov (Brookhaven National Laboratory (US)), Vladimir Korenkov (Joint Inst. for Nuclear Research (RU))

How to come to NRC KI NRC_KI_scheme_eng_.doc NRC_KI_scheme_eng_.pdf

how to find LIT JINR map_JINR.bmp

link to hotel aerostar http://www.aerostar.ru

link to hotel Pekin http://hotelpeking.ru/en/

link to Mariott (Tverskaya Yamskaya) http://marriottmoscowtverskaya.ru

Схема проезда к НИЦ КИ Схема_расположения_Курчатовского_центра_культуры_и_проходной_НИЦ_«Курчатовский_институт».docx Схема_расположения_Курчатовского_центра_культуры_и_проходной_НИЦ_«Курчатовский_институт».pdf

Thursday 29 January

Welcome. Dr.V.Velikhov, NRC-KI vice-director; Dr.V.Demin, NBICS vice-director

Introduction and logistics. A.Klimentov NRC-KI-workshop_Jan29_2015.pdf NRC-KI-workshop_Jan29_2015.pptx

Database Technologies

Convener: Dr Maria Grigorieva (NRC KI)

slides

• database_discussion.pdf
• Gancho_Dimitrov_PanDA_DB_implementation_Moscow_workshop_29thJan2015.pdf
• HStorage_development_state.pdf
• lassnig_db_teg.pdf
• Our_experience_in_NoSQL_integration.pdf
• Our_experience_in_NoSQL_integration.pptx
• swc20120614_lassnig.pdf

Topics for discussion:

1. Our experience of the developing of Hybrid SQL/NoSQL Storage, and NoSQL database integration in BigPanDA monitor

2. Database performance tests : SQL - NoSQL, NoSQL - NoSQL

3. Technology evaluation tests results for NoSQL databases: MongoDB, HBase, Cassandra, Dremel, CouchDB, MariaDB

4. Experience of using Hadoop/Spark/MapReduce in PanDA Infrasctucture. Use cases.

5. Foreseen performance and possible changes in PanDA Oracle archived database schema during/after the LHC Run2

6. Query routing strategy in BigPanDA applications (BigPanDA Monitor in particular)

7. How to implement cross database requests in heterogeneous architecture

8. Strategies of the data modelling for NoSQL databases 

EOS and Data Store

Convener: Eygene Ryabinkin (National Research Centre Kurchatov Institute (RU))

PanDA@NRC-KI

Convener: Ruslan Mashinistov (Russian Academy of Sciences (RU))

1 PanDA @ NRC-KI

PanDA instance installation and commissioning status PanDA for ATLAS PanDA beyond LHC and HEP

Speaker: Ruslan Mashinistov (Russian Academy of Sciences (RU))

Slides PanDA_instance_@_RRC-KI_-_Jan29.pdf

2 Discussion

PanDA beyond ATLAS : ALICE Do we need version w/o VOMS, Grid Services… Do we need Light weight DDM Staging and I/O matters

webFTS

Convener: Mr Andrey Kiryanov (B.P. Konstantinov Petersburg Nuclear Physics Institute - PNPI ()

3 Introduction: current status of FTS3/WebFTS

Speaker: Oliver Keeble (CERN)

Slides Presentation1.pdf

4 FTS readiness for transfers to/from non-Grid resources (VOMS/non-VOMS credentials, etc.)

5 Impact of Federated Identity support in WebFTS for non-Grid users

Slides http://cern.ch/kiryanov/WebFTS-IdF.pdf

6 FTS & HPC: old/new APIs, lightweight clients, Python client library for PanDA integration

7 Usage scenario for Titan: requirements from both sides

Speakers: Mr Andrey Kiryanov (B.P. Konstantinov Petersburg Nuclear Physics Institute - PNPI (), Danila Oleynik (Joint Inst. for Nuclear Research (RU))

8 Feature requests for FTS/WebFTS

Meeting with Big Data Lab PhD students (round table)

Conveners: Alberto Pace (CERN), Dr Markus Schulz (CERN), Massimo Lamanna (CERN), Prof. Shantenu Jha (Rutgers U), Simon C. LIN (Academia Sinica)

round table panel and participants list Students_and_PhD_students_list_for_round_table-2.docx Students_and_PhD_students_list_for_round_table-2.pdf

BigPanDA

Convener: Dr Alexei Klimentov (Brookhaven National Laboratory (US))

9 Introduction

10 webFTS highlights

Speakers: Mr Andrey Kiryanov (B.P. Konstantinov Petersburg Nuclear Physics Institute - PNPI (), Dr Markus Schulz (CERN), Oliver Keeble (CERN)

11 Database session highlights

Speakers: Gancho Dimitrov (CERN), Dr Maria Grigorieva (NRC KI), Ms Marina Golosova (NRC KI), Dr Mario Lassnig (CERN)

Slides Status_report.pdf Status_report.pptx

12 PanDA session highlights

Speakers: Danila Oleynik (Joint Inst. for Nuclear Research (RU)), Kaushik De (University of Texas at Arlington (US)), Ruslan Mashinistov (Russian Academy of Sciences (RU))

Slides panda-session-summary-jan15.pdf panda-session-summary-jan15.pptx

13 PanDA HPC pilot

Speakers: Danila Oleynik (Joint Inst. for Nuclear Research (RU)), Prof. Shantenu Jha (Rutgers U)

Slides PanDA_Pilot_HPC_(KI).pdf PanDA_Pilot_HPC_(KI).pptx

14 webFTS integration with PanDA

Speakers: Mr Andrey Kiryanov (B.P. Konstantinov Petersburg Nuclear Physics Institute - PNPI (), Danila Oleynik (Joint Inst. for Nuclear Research (RU)), Kaushik De (University of Texas at Arlington (US)), Oliver Keeble (CERN)

13:30 Lunch

Modeling Distributed Computing Systems

Conveners: Dr Eugene Burnaev (IPPI RAS), Kaushik De (University of Texas at Arlington (US)), Prof. Shantenu Jha (Rutgers U)

visit to I.V.Kurchatov museum and NRC KI Supercomputing center

doodle poll to visit I.V.Kurchatov museum http://doodle.com/hmwa3377raash3ne

doodle poll to visit NRC KI Tier1 and supercomputi http://doodle.com/bmksyczw8hye5waw

Nano-,bio-,information and cognitive technologies Institute seminar

15 "PanDA, A New Paradigm for Computing in HEP"

Speaker: Kaushik De (University of Texas at Arlington (US))

Slides panda-ki-jan15.pdf panda-ki-jan15.pptx

16 CERN and Computing

Speaker: Alberto Pace (CERN)

Slides 2015-01-29-Alberto-Pace-Moscow-Big_Data_challenges_in_mega-science-V03.pdf 2015-01-29-Alberto-Pace-Moscow-Big_Data_challenges_in_mega-science-V03.pptx

17:00 Leave to JINR by bus from NRC KI

Friday 30 January

17 Welcome from Prof.N.Rusakovich, JINR Chief Scientific Secretary

Modeling in mega-projects

18 NICA mega-project

Speaker: Prof. O.Rohachevsky (JINR)

19 Modeling of grid-cloud systems

Speaker: Prof. Gennady Ososkov (JINR)

11:10 coffee break

BigPanDA project

20 Project overview : Growing PanDA ecosystem, PanDA beyond ATLAS, prospects for ALICE, COMPASS, NICA.

Speaker: Kaushik De (University of Texas at Arlington (US))

Slides panda-dubna-jan15.pdf panda-dubna-jan15.pptx

21 Towards an Abstractions-driven and Standards-based Next-generation of Distributed Cyberinfrastructure

Speaker: Prof. Shantenu Jha (Rutgers University)

22 Network awareness

Speaker: Artem Petrosyan (Joint Inst. for Nuclear Research (RU))

Slides network_awareness_2015.01.30.pdf network_awareness_2015.01.30.pptx

Technology trends

23 webFTS and FTS3

Speaker: Oliver Keeble (CERN)

Slides FTS3_Trends_v3.pdf

24 EOS

Speakers: Alberto Pace (CERN), Massimo Lamanna (CERN)

Slides DUBNA20150129.pdf

25 ORACLE and Hadoop/HBASE in ATLAS

Speakers: Gancho Dimitrov (CERN), Dr Mario Lassnig (CERN)

Slides Gancho_Dimitrov_Oracle_in_ATLAS_Dubna_Workshop30thJan2015_2.pdf hadoop_atlas.pdf

26 CLAVIRE Highlights

Speakers: Prof. Alexander Boukhanovsky (ITMO), Dr Denis Nasonov (ITMO)

Slides CLAVIRE_2015.pdf CLAVIRE_2015.pptx

14:00 lunch

Monitoring

27 NoSQL for the storage of monitoring data:

Speaker: Ivan Kadochnikov (Joint Inst. for Nuclear Research (RU))

28 Monitoring in DIRAC environment for the BES3 experiment

Speaker: Igor Pelevanyuk (Joint Inst. for Nuclear Research (RU))

Slides

• JINR_Meeting_BESIII_Dark.pdf
• JINR_Meeting_BESIII_Light.pdf
• JINR_Meeting_BES_Light.pptx

Reports on Tier1

29 ASGC TW Tier-1

Speaker: Simon Lin (Academia Sinica (TW))

Slides 20150130-ASGC-SimonLin.pdf 20150130-ASGC-SimonLin.pptx

30 NRC KI RF Tier-1

Speaker: Dr Vassily Velikhov (NRC KI)

31 JINR Tier-1

Speaker: Dr Tatiana Strizh (JINR)

16:20 tea break

32 Presentation of the heterogeneous cluster of LIT

Speaker: Dr D.Podgainy (JINR)

Excursion in LIT

Beer party

doodle poll http://doodle.com/b7qnir6hw4h2epwk#table

Saturday 31 January

Technical splinter meetings

33 Distributed Modeling splinter meeting