Speaker
Description
In our talk at CS3 2025 we will discuss the (r)evolution in the storage and network back-ends for cloud-based data centric services with a special focus on the sync & share service as a storage killer application.
We will review various storage back-ends for cloud platforms along with their performance, scalability and maintenance complexity and economical aspects. We will also touch on the new trends in the cloud platforms that convert into hyper-convergent, fully software-defined and open source-based systems, where compute, storage and network components act together to provide flexible, functional, scalable, reliable and high-performance platform that is cost-effective, easy to implement, maintain, automate, monitor and optimize.
In face of the AI revolution and overall shift of economy, science, research and education towards data-centric applications and services we observe increasing requirements vs cloud platforms that help dealing with large data sets. At the very infrastructure level this means a growing pressure on performance (IOPS, GB/s) and economical scalability. We can also observe that ‘classical’ storage technologies: disk arrays, Fibre Channel-based SAN networks etc. become obsolete and legacy as they lack flexibility, do not support programmatic orchestration widely, and are relatively costly to purchase, implement and maintain. Their usage requires specialised knowledge and can lead to vendor lock-in situations. On the other end of technology spectrum, commodity technologies are rapidly developed, breaking the next barriers of reliability and performance. Growing popularity and improving economic affordability of flash storage systems (SSD, NVMe) enables advancing the cloud platforms and applications performance so that they address today’s IO requirements. Also recent development of Ethernet enables using this protocol as a carrier for I/O traffic. RoCE is in particular useful to implement RDMA in Ethernet networks that are widely spread in today’s IT infrastructure.
These two advancements in storage and network technology support for reliability, scalability and economical efficiency, re-use of existing knowledge as well as compute, storage and network platform integration and simplification. Among the new technologies NVMeoF gains the momentum, with increasing number of vendors offering NVMeoF products. Also the increasing adoption of software defined networks (SDN) and cloud software stacks supporting SDN facilitates compute, storage and network resources provisioning, orchestration and automation.
In our presentation we will discuss the PSNC cloud computing, storage and network platform with a special focus on usage of SSD, NVM, RoCE technologies and usage of SDN. PSNC platform is based of Openstack, OKD compute components and both specialised (storage arrays, NAS appliances, HDDs, SSD/NVMe) and software-defined storage components (Ceph on HDD and SSD/NVMe), software defined network (SDN) as well as legacy and modern storage networks (Fibre Channel, iSCSI, Ethernet, NVMeoF). The platform provides storage, compute and network resources for our sync & share systems including country-wide box.pionier.net.pl service offered to academic and research community in Poland since 2015, based on Seafile software as well as our EOSC EU Node ownCloud/Kiteworks OCIS-based sync & share service devoted to EOSC users.
Seafile and ownCloud/Kiteworks OCIS are interesting and challenging benchmark applications to storage systems, back-end networks and compute infrastructure components due to their extreme IO requirements. In our presentation we will share information on current setup of these applications implemented at PSNC as well as make projections on future improvements of our sync & share services compute, storage and network back-ends.
