Lessons Learned at SC14

(article written for the Scality blog)

Attending a Super Computing Conference is always a humbling experience. It’s the event where you meet scientists that are helping to solve the hardest problems in the world, such as the discovery of new drugs, genomics research to better discover and treat diseases, finding new energy sources or the precise prediction of the next superstorm. The SC Conference is the place where innovations that are making these life-changing discoveries possible are shared.

What started as an informal get-together of a couple hundred computer scientists 25 years ago has become a major tradeshow, attended by thousands of researchers and technologists. This is not an audience of trade show tire-kickers but one of supercomputing specialists who continuously push technology to its limits to solve some of the greatest scientific challenges.

SC14 was Scality’s second participation in the show. Last year’s event led – amongst others – to Los Alamos National Laboratory (LANL) selecting the Scality RING as an active archive for complex simulations used to monitor the health of the US nuclear stockpile.

Lesson 1: a half-exabyte customer reference in this industry gets your technology to go viral quickly.

The two key use cases Scality supports for the supercomputing industry are Active Archives and Distributed Computing. Scality RING for Active Archives enables HPC customers to offload data off their expensive Tier 1 storage to free cycles for other high-performance workloads. The Distributed Computing use case focuses on the RING’s ability to run highly parallel and independent computing jobs.

Both use cases are very distinct, each with specific characteristics, but at the same time, they go very closely together: LANL is using the Scality RING both as scale-out archive storage and high-performance storage for simulations.

Lesson 2: the HPC industry is very actively investigating how they can lower costs by running compute jobs on industry-standard servers.

Challenged by Amdahl’s law, HPC storage architects are working hard to shift to more parallel and distributed architectures in diverse areas such as genomics simulations, biology research and electronic design problems. In many areas of research, the quantity of data being manipulated is growing exponentially. This is certainly true of numerical simulations that typically store short term results on very high performance “scratch” file-systems before storing the data more permanently on researchers “home” directories or on tape. It’s also true in all fields using increasingly accurate physical sensors to acquire data, such as oil and gas seismic surveys, radar and satellite imaging and electron microscopy.

Interest is growing in archiving these massive amounts of data. Having it readily accessible for further study increases its value and furthers the objectives of the research being performed.

A key benefit of the Scality RING is the ability to seamlessly integrate with existing high-performance storage systems. Scality is the only software-defined storage platform that supports mixed environments of GPFS, Lustre and object storage, with native support for NFS and SMB file access. HPC customers are leveraging Scality’s open source block driver to build data movers for their GPFS infrastructures and several technology partners are integrating their Lustre systems with the RING, using the Lustre HSM functionality.

Lesson 3: the RING is a perfect solution for “home” storage, but also supports some Scratch storage requirements. Think of Home Storage with Scratch performance capabilities

The Scality RING offers unique flexibility in this industry. Using industry-standard servers, customers can build Exabyte-scale storage systems that can be used for active archives, home storage and to some extent, scratch. Thanks to some highly visible customer successes in the HPC industry, the result is that Scality has very quickly become the hottest storage technology in this market. One visitor, who had been doing his homework before visiting us at the show described us as “Scality? Well that’s like Ceph that works, right?”

It’s a good start – we can live with that.


~ by tomleyden on December 16, 2014.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: