28–30 Jan 2019
CNR
Europe/Zurich timezone

Building a Scalable, Flexible, Interoperable Federated Cloud to Power Australian Research

29 Jan 2019, 15:20
20m
CNR

CNR

National Research Council - Piazzale Aldo Moro 7, 00185 Roma, Italy
Presentation Open Data Ecosystems and CS3 Open Data Ecosystem and CS3

Speaker

Mrs Joanne Morris (Australian Research Data Commons)

Description

The Nectar Research Cloud provides a self-service OpenStack cloud for Australia’s academic researchers. Since its inception in 2012 it has had rapid growth, with approximately 37,000 virtual CPUs now being used at any given time (more than 95% of the available resource), with over 7,000 virtual machines being run by approximately 2,000 researchers and used by thousands more. The number of registered cloud users is now over 14,000 and continues to grow at over 200 per month. In the last year, over 4,000 researchers ran a virtual machine in the Nectar cloud. Over 2,500 project allocations have been approved, with about 50 new applications per month. More than 700 allocations are to support multi-institutional collaborations, and over 300 support national research grants.
The Nectar Cloud is different to many clouds in being a federation across seven organisations, each of which runs cloud infrastructure in one or more data centres and contributes to the operation of a distributed help desk and user support. A Nectar core services team runs centralised cloud services. The Nectar Cloud federation is continuing to be extended with the inclusion other institutional cells and also the recent partnering with Auckland University. The offering of services that Nectar Research Cloud continues to grow with most recently releasing a Simple Container Service (Docker) and an Container Orchestration Service (Kubernetes). The Nectar Research Cloud is very focused on enhancing the user experience through integration with other services and providing simplified user interfaces.
Nectar recently merged with the Australian National Data Service (ANDS) and Research Data Services (RDS) to form the Australian Research Data Commons (ARDC). ARDC is funded by the Australian Government through the National Collaborative Research Infrastructure Strategy (NCRIS) and is currently developing its five year strategy and capital plans. As part of the strategic planning cycle ARDC is engaging with a range of resource providers, institutions and research organisations, e-infrastructure providers, peak bodies and government. ARDC’s vision is “Transforming Australian Research together; Towards an Agile, Interoperable & Sustainable eResearch Infrastructure Ecosystem for Australia”.
One of the motivations and challenges of the merger of Nectar, ANDS, and RDS into the ARDC is to provide more seamless integration of the national cloud and compute services with data and storage services. One example is recent work to make it easier for users of the Nectar cloud to access their data from Cloudstor, a national sync-and-share storage service.
This presentation will give an overview of the experiences, challenges (both past and present), and benefits of running a federated OpenStack cloud and an overview of the some of the integrations we have implemented to improve the user’s experience. It will also describe future plans to extend the infrastructure and services, integrate with other services and data storage providers, as well as extending interoperability with the growing number of international science and research clouds through initiatives such as the Open Research Cloud, along with improving access for international researchers to assist in international collaborative research.

Primary author

Mrs Joanne Morris (Australian Research Data Commons)

Co-author

Dr Paul Coddington (Australian Research Data Commons)

Presentation materials