In this podcast Shawn Rosemarin, vice-president of customer engineering in research and development at Pure Storage, argues that spinning disk hard drives (HDDs) will face commercial extinction in five years’ time.
We talk about pressures on data storage from rising energy costs as well as rapidly increasing volumes of data.
Rosemarin also talks about the inherent inefficiencies of mechanical hard drives and how manufacturers’ efforts to upgrade the technology are the “tail end” of their evolution. He cites also the backdrop that solid-state storage is dominant when considering the consumer market, and that this helps drive down the cost of flash.
The question is posed, when will the HDD and flash curves converge?
Antony Adshead: Pure says we’re at an inflection point and that HDDs will be extinct in five years. So, what is it that’s becoming more challenging with HDDs?
Shawn Rosemarin: Let’s cut to the chase here. Energy costs have gone up significantly, and what we know is that once energy costs have gone up, they rarely, if ever, come down. And so, it’s become more and more of a burning issue for folks to think about how they are going to manage this increased energy cost. Whether on-prem or in the cloud, the costs have gone up.
The fact is, to make that matter worse, we’re seeing an explosion of data growth. We already projected data growth in five years would be 10 times what it is today, but that was before all the excitement that’s happened with ChatGPT and all the AI-generated content, and so I would argue that 10 time growth is super-conservative versus where we’ll be.
Adshead: What can flash drives do that HDDs can’t?
Rosemarin: If you think about the fundamental physics, a hard drive spins and flash doesn’t. And therein lies the crux of why energy utilisation is so much less. When you think about the spinning of those hard drives and even some of the latest technology using microwaves or even lasers to speed those drives up, the ultimate reality is it consumes more electricity.
So, the answer to your question is, not only are flash drives significantly more reliable and significantly higher performing, but to the crux of our conversation today, they’re much more power efficient.
Adshead: Pure claims there’ll be no hard disks sold in five years’ time, so how can flash drives become dominant?
Rosemarin: What I would tell you today is that they are already dominant in everywhere but the datacentre. If we look at our smartphones, at our smart appliances, our automobiles and we look at all other elements of technology, storage has moved to flash.
However, in the datacentre based on total capacity and capacity per gigabyte, many consumers of hard drives have stayed there because there has been a significant cost delta.
But high-performance workloads have already moved to flash.
The reality was, though, how quickly would those two cost curves converge? And I’m talking about acquisition cost here. How quickly would the cost of a hard drive and the cost of a flash drive, in Pure’s terms a Direct Flash Module and the rest of the industry’s terms an SSD [solid-state drive], how close or how quickly would those two curves come together?
We’ve seen a dramatic drop in the cost of flash over the last few years. Why? Because as supply goes up price comes down. And when you look at the mass of consumption of flash around the world compared to hard drives – more and more fabs, more and more capability, drop in price – that’s put us in a position now where we are very confident that the cost of flash, acquisition and the long-term cost of flash drives will exceed and overtake the cost of hard drives over the next five years.
Adshead: But HDDs are of higher capacity and are much less costly per gigabyte?
Rosemarin: Well, this is the most interesting part. As Pure Storage, we’re in a very unique position. We don’t leverage industry-standard SSDs. We create our own flash drives, called Direct Flash Modules, that leverage raw NAND. That gives us the ability to control the roadmap and drive our own innovation agenda as it relates to the density of these drives.
We did it in TLC [triple-level cell], going up to 52TB [terabytes]. We did it most recently in QLC [quad-level cell] across our portfolio, which has allowed us to get even denser and going up to 48TB. And, we most recently, just a couple of months ago, told the world that we will have a 300TB flash drive within a couple of years.
When we look at the hard drive world as a whole, we see that we are at the tail end of that innovation cycle. The traditional hard drive manufacturers continue to try to drive more and more density, the latest being HAMR or leveraging lasers to speed up the head of the drive, but they are at the end of that curve. It’s a 67-year-old technology, it has pretty much run its course, and when we look at 300TB per drive within the same footprint as today’s 48TB drive, there really is no comparison between the innovation curve of flash from a Pure Storage side and hard drives.