Oct 12 – 16, 2015
Brookhaven National Laboratory
America/New_York timezone

Document oriented database infrastructure for monitoring HEP data systems applications

Oct 15, 2015, 2:00 PM
Bldg. 510 - Physics Department Large Seminar Room (Brookhaven National Laboratory)

Bldg. 510 - Physics Department Large Seminar Room

Brookhaven National Laboratory

Upton, NY 11973
Basic IT Services Basic IT Services


Carlos Fernando Gamboa (Brookhaven National Laboratory (US))


Within the High Energy Physics (HEP) software infrastructure a diverse Data storage and Distribution software technologies are used. Despite their heterogeneity they all provide a capability to trace application event data to be able to troubleshoot problems related to the software stack or software usage. The subsequent data is written once and stored in the majority of cases in a file, commonly known as log file, for posterior analysis. An open source infrastructure to collect, store in a document oriented type database, and visualize data processed from the application log files is explored in this presentation. Experience in deploying the ELK (Elasticsearch, Logstash and Kibana) framework applied to a data storage technology is summarized.
Length of presentation (max. 20 minutes) 20

Primary author

Carlos Fernando Gamboa (Brookhaven National Laboratory (US))

Presentation materials