What Would 10 Petabytes Look Like?

Sometimes I stop and think about what Coraid EtherDrive SAN System can really do for folks. Over the years we have sold between 10,000 and 15,000 systems. First, they were 15 bay systems, then 24, then 36. The cost of our arrays per slot keeps going down. We keep making the system less expensive and the disk folks keep putting more and more room on a spindle. This morning I asked myself the following question.

What would 10 PB of Coraid EtherDrive SAN System look like?

Imagine three 19" racks. Each rack has ten SRX3641 Media Arrays each with 36 10 TB drives. Three such racks add upto 1,080 disk drives at 10 TB each, or 10.8 PB of disk space.

Ten PB in six linear feet of floor space. Not bad.

The ten SRX3641 Media Arrays fit nicely in a standard rack. They leave 2U for switching at the top. Half of the ports go to the SRX3641 Media Arrays and the other half goes to the hosts that will use the storage located in different racks.

The 2U is left over for a couple of fast switches. The SRX Media Array(s) doesn’t really care what brand of switch is used, as long as it allows jumbo frames. With two switches, you naturally get redundant pathways to the media arrays, and with ATA-over-Ethernet, none of this has to be configured. Multi-pathing and port-bonding come for free with this technology.

We use a couple of Arista 52 port 10GbE switches. Each SRX Media Array can have upto 6 10GbE ports, which can easily pump data in both directions at the same time. This means we have potentially 120 Gb of bandwidth on each SRX Media Array. That amounts to 1.2 TB of bits flying into and out of a single rack. Three racks equate to simply three times as much bandwidth. Truly scale-out.

How much would 10 PB cost?

Each SRX3641 Media Array is $2,995. Three years of software license is $995 for each year which totals to $2,995, about the same as the basic VMware, Windows, or RedHat license. About $350 will get you a 10 TB disk drive, so that’s a total of $12,600 for all the disk drives. A couple of dual 10GbE NICs in an SRX3641 Media Array are $299 each, so $600 for that. That totals to $19,190.Your prices on disk drives might bobble up or down a bit, but you’restill around $20,000 for 360 TB of storage in a single box.

Ten of those is $200,000. Three of those racks, giving you the 10 PB I was thinking about, is $600,000.

That’s less than an order of magnitude cheaper than the closest competitor that I know of. You can get something small at this price point, but will quickly run out of performance, capacity, or bandwidth.

Now, imagine you don’t have to pay all of that at one time.

You can buy a single chassis to start with. You don’t even have to buy all the disks - just add them as you need them.

The whole system can expand at the rate you want. Have a burst in your business and need more storage? Just buy more disks. When you fill a chassis, just buy another one. When you fill a rack, start on the next one.

It all scales-out at any rate you want, one drive at a time.

You don’t need to stop at 10 PB either. The Coraid EtherDrive SAN System will expand to 23,400 PB. You can have 65,535 SRX Media Arrays in the same network. But I’m guessing you don’t need that much disk space.

We’re both cheaper and better than using the cloud. Your storage is safe and sound on your own premises. With cloud you have to pay not only for the storage itself, but for accessing it. These charges for network traffic make your costs variable for what should be fixed. This often times creates surprising bills. Normally a business wants to nail down fixed costs while leaving the variable costs to things related to revenue.

With Coraid EtherDrive SAN System, you have a fixed expense. No surprises in your email box at the end of the month.

And it’s cheaper too. The way I designed it turns out to be more affordable than the cloud. The cost for any of these configurations, the smallest to the largest, is only about one tenth of a penny per GB per month if you use the storage for five years. That’s $0.001 per GB per month.

To figure that out for yourself, just take the price for a single SRX3641 Media Array with all its disks and NICs and software, about $20,000, and divide it by 360,000 GB (36 bays x approx. 10000 GB per disk), which gives you the cost per GB, about $0.055. Now, divide that by 60 months, and you come up with $0.000952.

There are Coraid customers who have had boxes running that long without rebooting, so running our stuff for five years is a no brainer.

It’s nice to see how we can go fast with terabytes of bandwidth, and truly scale-out storage. Each SRX Media Array adds more disks, more processors, more networking and more RAM.