SDU eScience Center doubles storage capacity with new scalable S3 system

We have successfully replaced our existing Ceph cluster with a more compact, energy-efficient and scalable solution. The new storage system more than doubles our storage capacity, expanding it from 4 petabytes (PB) to 10 PB; an upgrade that will secure the storage needs for data-intensive research at SDU in the coming years.

The new hardware is provided by Supermicro, while the software layer is powered by Quantum ActiveScale, a robust object storage solution. The system is based on S3 (Simple Storage Service) — a protocol originally developed by Amazon for storing and retrieving large volumes of unstructured data as objects. S3 has become a widely adopted standard in cloud and enterprise storage environments.

“The new system offers a seamless integration with the existing Weka file system that we acquired a few years ago. Users will not notice any changes in their workflows, as Weka transparently manages the movement of data to the new system, which now serves as a second-tier storage layer. What users will notice is that new system offers faster access to cold-tier data, improving performance for retrieving archived datasets,” says Martin Lundquist Hansen, team leader for infrastructure at the SDU eScience Center.

The new storage system is funded by University of Southern Denmark (SDU), meaning that SDU researchers will directly benefit as they are able to apply for and utilize more storage resources. A limited portion of the capacity is also available through DeiC.

Sustainable and scalable environment

Compared to the old Ceph cluster, the new system is more compact and consumes less power, aligning with SDU’s goals for sustainability and operational efficiency.

“The architecture is also highly scalable, allowing us to easily expand the system in the coming years, according to researchers’ needs, by adding more disks,” says Martin Lundquist Hansen.