What Would 10 Petabytes Look Like?

Posted on by

Sometimes I stop and think about what Coraid storage can really do for folks. Over the years we have sold between 10,000 and 15,000 systems. First, they were 15 bay systems, then 24, then 36. The cost of our arrays per slot keeps going down. We keep making the system less expensive and the disk folks keep putting more and more room on a spindle. This morning I asked myself the following question.

What would 10 PB of Coraid storage look like?

Imagine three 19" racks. Each rack has ten SRX/3641 arrays each with 36 10 TB drives. Three such racks add up to 1,080 disk drives at 10 TB each, or 10.8 PB of disk space.

Ten PB in six linear feet of floor space. Not bad.

The ten SRX/3641's fit nicely in a standard rack. They leave 2U for switching at the top. Half of the ports go to the SRX/3641s and the other half goes to the hosts that will use the storage located in different racks.

The 2U is leftover for a couple of fast switches. The SRX doesn't really care what brand of switch is used, as long as it allows jumbo frames. With two switches, you naturally get redundant pathways to the storage, and with ATA-over-Ethernet, none of this has to be configured. Multi-pathing and bonding come for free in this technology.

We use a couple of Arista 52 port 10GbE switches. Each SRX can have up to 6 10GbE ports, which can easily pump data in both directions at the same time. This means we have potentially 120 Gb of bandwidth on each SRX. That amounts to 1.2 TB of bits flying into and out of a single rack. Three racks equates to simply three times as much bandwidth. Truly scale-out.

How much would 10 PB cost?

Each SRX/3641 is $2,995. Three years of software license is $995 for each year which totals to $2,995, about the same as the basic VMware, Windows, or RedHat license. About $350 will get you a 10 TB disk drive, so that's a total of $12,600 for all the disk drives. A couple of dual 10GbE NICs in a SRX/3641 are $299 each, so $600 for that. That totals to $19,190. Your prices on disk drives might bobble up or down a bit, but you are still around $20,000 for 360 TB of storage in a single box.

Ten of those is $200,000. Three of those racks, giving you the 10 PB I was thinking about, is $600,000.

That's less than an order of magnitude cheaper than the closest competitor that I know of. You can get something small at this price point, but will quickly run out of performance, capacity, or bandwidth.

Now, imagine you don't have to pay all of that at one time.

You can buy a single chassis to start with. You don't even have to buy all the disks - just add them as you need them.

The whole system can expand at the rate you want. Have a burst in your business and need more storage? Just buy more disks. When you fill a chassis, just buy another one. When you fill a rack, start on the next one.

It all scales-out at any rate you want, one drive at a time.

You don't need to stop at 10 PB either. The system will expand to 23,400 PB. You can have 65,535 SRXs in the same network. But I'm guessing you don't need that much disk space.

We're both cheaper and better than using the cloud. Your storage is safe and sound on your own premises. With Cloud you have to pay not only for the storage itself, but for accessing it. These charges for network traffic make your costs variable for what should be a fixed asset. This often times creates surprising bills. Normally a business wants to nail down fixed costs while leaving the variable costs to things related to revenue.

With Coraid storage, you have a fixed expense. No surprises in your email box at the end of the month.

And it's cheaper too. The way I designed it turns out to be more affordable than the cloud. The cost for any of these configurations, the smallest to the largest, is only about one tenth of a penny per GB per month if you use the storage for five years. That's $0.001 per GB per month.

To figure that out for yourself, just take the price for a single SRX/3641 with all it's disks and NICs and software, about $20,000, and divide it by 360,000, which gives you the cost per GB, about $0.055. Now, divide that by 60 months, and you come up with $0.000952.

There are Coraid customers who have had boxes running that long without rebooting, so running our stuff for five years is a no brainer.

It's nice to see how we can go fast with terabytes of bandwidth, and truly scale-out storage. Each SRX adds more disks, more processors, more networking and more RAM.

About the Author

Brantley CoileInventor, coder, and entrepreneur, Brantley Coile invented Stateful packet inspection, network address translation, and Web load balancing used in the Cisco LocalDirector. He went on to create the Coraid line of storage appliances, a product he continues to improve today.

Affordable, reliable, logical storage.

Coraid Etherdrive SRX™ software transforms simple hardware into high-performance Ethernet block storage at 10x price/performance advantage over Fibre Channel and iSCSI. Now shipping with new hardware, Coraid EtherDrive SRX now supports text/email notifications, 4K Advanced Format disk drives, VMware ESX 6.0/6.5 and more.

Get Started Today

←Previous | Blog Archive | Next →