CERN’s accelerator complex is an extreme data generator, every second a large amount of comprehensively heterogeneous data coming from control equipment and monitoring agents is persisted and needs to be analyzed. Over the decades, CERN’s researching and engineering teams have applied different approaches, techniques and technologies. This situation has minimized the necessary collaboration and more relevantly, restricted the cross data analytics over different domains. These two factors are essential to unlock hidden insights and correlations between the underlying processes, which enable better and more efficient daily-based accelerators operations and more informed decisions.
The proposed Big Data Analytics as a Service Infrastructure aims to: (i) Integrate the existing developments. (ii) Centralize and standardize the complex data analytics needs for the wide CERN research and engineering community. (iii) Deliver real time and batch data analytics capabilities and (iv) provide transparent access and extraction-transformation-load, ETL, mechanisms to the different and critical-mission existing data repositories.
This talk will reflect the desired properties resulting from the analysis on CERN’s data analytics requirements; the main challenges: technological, collaborative and educational.