19–25 Oct 2024
Europe/Zurich timezone

HPSS Batch Application

21 Oct 2024, 15:18
57m
Ground floor lobby

Ground floor lobby

Poster Track 1 - Data and Metadata Organization, Management and Access Poster session

Speaker

Justin Spradley

Description

How to effectively and efficiently stage a large number of requests from an IBM HPSS environment using a MariaDB database to keep track of requests and use Python for all business logic and to consume the HPSS API. The goal is to be able to scale to handle a large number of requests and to meet different needs of different experiments, and to make the program adaptable enough to allow for each experiment to have its own unique business logic. This update will take advantage of features of the newest versions of HPSS, as well as MariaDB, Python, and Linux. Furthermore, the hope is that the application will be able to log and handle a wider array of errors and exceptions, and allow for more in depth monitoring as the status of each request will be stored in a database which allows for easy querying. Furthermore this may allow for additional enhancements such as staging requests by priority.

Primary author

Justin Spradley

Presentation materials

There are no materials yet.