XLDB 2013 Workshop Europe

Evaluation of some LSST Queries: Preliminary Results

29 May 2013, 11:10

10m

60/6-015 - Room Georges Charpak (Room F) (CERN)

Big Data in Astronomy / Astrophysics

Speaker

Prof. Mohand-Saïd Hacid (LIRIS, Lyon 1 University)

Description

In many scientific fields, such as physics, astronomy, biology or environmental science, the rapid evolution of data acquisition tools (e.g., sensors, satellites, cameras, telescopes) as well as the extensive use of computer simulations have led in recent years to an important production of data. Modern scientific applications are then facing with new problems that are primarily related to the management and use of such data. In addition to the growing volume of data to handle, their complex nature (e.g., images, uncertain data, multi scale,...), the heterogeneity of their formats and the various processing to which they are subject are the main sources of difficulties. The problems are such as scientific data management was recognized as a real bottleneck which slows down scientific research since it relies more and more on the analysis of massive data. In this context, the role of the computer as a direct way to improve the discovery process in science is important. As part of its mission, CNRS, in the framework of PetaSky project, we conducted experiments on PT1.1 data set in order to compare the performances of both centralized and distributed database management systems. Regarding centralized systems, we have deployed three different DBMSs: Mysql, Postgresql and DBMS-X (a commercial relational database). Regarding distributed systems, we have deployed HadoopDB. The goal of these experiments is to report on the ability of these systems to support LSST requirements from data management perspective. We mainly analyzed issues related to performance, speed up, fault tolerance and latency.