In a large scale storage system which consists of hundreds of servers, tens of thousands of clients, a variety of devices, anomaly detection is a nontrivial task. Traditional solutions which are still working in our cluster operations include setting static thresholds on KPIs, searching key words in system logs and so on. These methods highly depend on experience of system administrators,...
Storage architectures are becoming larger and deeper. This leads to a quadratic increasing of systems complexity. To address the issue more ambitious monitoring schemes are arriving to the market. A similar trend is observed for both HPC and general purpose data centers.
In this talk we will position the DDN monitoring efforts in this context, and open a more generic discussion on the...
The Pawsey Supercomputing Centre is an unincorporated joint venture to support Australian researchers, funded by the four major universities in Western Australia, local governament, and the CSIRO. The centre works with many different types of filesystems such as Lustre, GPFS, NFS, Mediaflux and Ceph, that for legacy reasons do not talk very well with each other. Recent trends in research are...
Over five years storage team at DESY provides a reliable data-cloud service. While service is still officially in a pilot phase, it’s has the same support and priority level as any other production services provided by the IT group. Our choice of technology used to provide data-cloud service is nextCloud, as user facing front-end, and dCache, as back-end storage system.
The latest...
The aim of this presentation is to show how Cynny Space cloud object storage solution integrates with Vegoia, a new Network Intelligence platform.
The presentation will outline the following topics:
1) Cynny Space cloud object storage
The object storage solution is the first storage built on fully-equipped ARM® based micro-servers. Thanks to 1:1 micro-server to storage unit ratio and the...
Safespring are currently building a new version of our ceph service, and in this session, I will give an overview of the architecture of the load balancer for our S3 service, which we are building using open source tools. In our design, we use internet as a design pattern for the datacenter cluster network ensure scalability and predictable performance. That means we use BGP everywhere, even...
Data management has historically started at the point of ingest where users manually placed data into a system. While this process was sufficient for a while, the volume and velocity of automatically created data coming from sequencers, satellites, and microscopes have overwhelmed existing systems. In order to meet these new requirements, the point of ingest must be moved closer to the point...
Onedata [1] is a transparent, high-performance data management system, which provides transparent access to globally distributed storage resources and supports a wide range of use cases from personal data management to data-intensive scientific computations. Due to its fully distributed architecture, Onedata enables the creation of complex hybrid-cloud infrastructure deployments, including...
