COPAN Systems Enterprise MAID uses disk power off to massively reduce power consumption by switching disks off, rather than spinning down the drives but keeping them electrically active. The Enterprise MAID power consumption feature allows for the highest disk density packaging in the industry of 896 disk drives in a single cabinet. At any one time no more than 25% of user data disks in the system are powered on (although all disk devices always appear available). By limiting the maximum spin-up ratio, the total maximum power requirement is permanently and always reduced to a minimum. This gives not only utility power savings but also gives savings in associated cooling. This saving is further enhanced by the reduction in data-centre overhead power costs (UPS, batteries etc), we could amount to a further 40% of actual load split across all the services in the datacentre.

COPAN Systems uses an Always on region™ (AOR) to cache data destined for MAID storage, that is frequently accessed (for read or write). This ensures that application metadata is given a fast response, just as in any traditional storage environment. The AOR is used to map specific portions of disk LUNs to disk that are always spinning. This simplifies locating a persistent application on to an Enterprise MAID subsystem, as metadata can also be located in the same array.

Some estimates suggest that up to 80% of all data stored in enterprise applications is persistent or inactive, not currently supporting business operations, but essential for compliance, audit and future use. If 80% of the data in an organization was able to be identified and migrated to a COPAN Systems MAID platform, the power savings could far exceed that of write-offloading alone. If COPAN Systems MAID was deployed for persistent data, whilst write-offloading was deployed for transactional storage (potentially with SSDs as the offload target), the overall total benefit, in power cooling and infrastructure costs could be truly huge.

COPAN Systems offers solutions today with 896TB of raw storage in a single footprint with a total power consumption of 7.3KwH. This equates to 8.14 watts per TB. Compare this 12w for a 300GB cheetah (40 watts per TB), COPAN Systems MAID platform is approximately 5 x more power efficient). This includes all of the controllers necessary to drive large disk pools, which are excluded from the comparison for DAS disks It should be RAID protection differences would have to be considered too, smaller RAID groups equate to better power management, larger RAID groups lessen the benefit.

Power savings in storage, can only be achieved by maximizing disk sizes and switching disks fully off when possible. Identifying persistent data and removing it from the transactional storage subsystems to a MAID platform (COPAN Systems is often up to 6 x more dense than transactional storage), and then maximizing efficiency in transactional storage through caching and write-offloading would seem to be a good strategy for dealing with massive data growth.

In addition, there have been many studies of power managed disk drives (e.g. http://csl.cse.psu.edu/publications/ispass03.pdf ) such that the focus of these studies are the access patterns to power-managed disk arrays. In the research paper referenced above, Penn State and IBM research, determined that the use of power managed disk drives for Tier1/Tier2 applications would actually consume more power. This is noted by the fact that it takes more power (18-20w) to spin up a disk drive. If the frequency of the spin up/down is too high, the drives consumes more power.

Write-offloading is a very interesting technique for minimizing disk spin-up in transactional environments. COPAN Systems believes that minimizing the data in transactional environments, by relocating data written once, occasionally accessed, and never changed, to a purpose-built platform could deliver even further benefits before write-offloading is considered.

Disk spin-down cannot be implemented in isolation. Thought must be given to data access patterns, data protection schemes, and disaster recovery. Any environment where spin-down is used as opposed to imposing a hard limit on disk use, requires an infrastructure to deliver at full load, with datacentre overhead. Spin down does not address these issues, which may come to represent a far higher cost than the savings in drive spin-down alone. When used in a persistent environment, massive savings can be accrued with MAID platforms with a hard power budget.

An alternative approach might be to use very large disks with multiple copies of the data. Then only spin up enough copies to support the current I/O demand requirement. Of course that will build up a whole lot of write-back operations which will be required when the devices are eventually “spun up”. However, a suitable storage array with non-volatile cache and dirty data markers could deal with this and do a “lazy-resync” to flush out cache data).

In effect there would be a series of mirrored disks, but it would be possible (in principle) to just have one of these online at any time due to the persistence in the array of the write-back cache. From time-to-time the array logic could spin the drives up and stage-out the data to avoid cache exhaustion. Of course if there’s a huge data churn then that approach won’t work, but then simply the array will power up all the drives anyway. It’s during “quiet” periods tha this would save power.

There is the objection that there are many copies of data held, but as we already have to do that for hardware redundancy reasons, then we already have to allow for some of that. Of course this solution envisages (potentially) many more copies, but as many of us know to our costs, the reducing cost per GB of storage is not matched by an increas in IOP capability. With 1.5TB (and shortly 2TB) disks available, then this might be a way of being able to exploit these huge empty spaces.

Another thing to note, is that queuing theory tells you that a single queue, multiple server, model (such as you’d get with this principle) then you can drive the back-end devices to a much higher level of utilisation without adversely affecting service times. The effect of that is that you can eke more IOPs out of a given set of disks by having multiple mirrors and stil get good service time (at the expense of space utilisation due to the multiple copies). Such a design would work best for workload patterns characterised by high read-to-write ratios.

However, I’m not wholly convinced off all this – I think something more dramatic in the form of SSD is going to be required as, for high-perforance storage, rotating disks are proving to be a major bottleneck. That also goes for the power too, although I would be interested to know how much of the power in an Enterprise array comes from the rotating disks, and how much from the (necessarily) very powerful controllers and electronics.

As for “green IT” not being a real priority, I think you’ll find it’s another thing filtering down from HPC into other markets. The guys who run the really big computers are already very well aware that they will probably spend more on power and cooling than on the equipment itself. These guys care about performance as much as anyone, and they know that to get better performance they need to overcome current limits on system size. Why do you think Roadrunner or Blue Gene are based on power-efficient processors instead of Intel heat pumps?

Sure, there are a lot of people paying lip service to “green IT” and there’s little upside to providing the same functionality with the same physical plant in a greener fashion, but there’s very signficant upside in meeting *new* needs without having to build a new data center. I’ve seen customers base purchase decisions on these factors. It’s a real priority for some real people, even if they’re not the people where you hang out.

When order processing backs up on the last day of the quarter who is going to give a hoot about IT’s power conservation program? There is very little upside for IT to go green.

Robin

Even better, use commodity 2.5″ drives to cut power requirements by closer to 90% of what those ‘enterprise’ drives use while retaining not all that much less than 50% of the throughput – more, if you use a few more of them to spread random accesses across.

Not to mention not having to explain to your users why some of their requests take so long to complete.

If increasing data center storage power efficiency was the goal, framing the question as “How do we do this using enterprise disks?” seems to have been far too narrow. And once current enterprise disks have been replaced by far more efficient ones, whether taking the same kind of drastic throughput hit just to reduce power consumption *another* 60% would seem questionable – though letting half of each mirrored pair of RAID-1 drives spin down during periods of low demand might be worthwhile, since that wouldn’t expose readers to the kind of 10-second delays that the described approach does.

- bill

DRPM [12] and Hibernator [32] are recently proposed approaches to save energy by using multi-speed disks (standard enterprise disks spin at a fixed rate of 10,000 or 15,000 rpm). They propose using lower spin speeds when load is low, which decreases power consumption while increasing access latency. However, multi-speed disks are not widely deployed today in the enterprise, and we do not believe their use is likely to become widespread in the near future.

Maybe the disk vendors are getting ready to do this, but I think they need their OEMs to buy in. That will take a couple of years.

Wes, the paper noted they didn’t investigate SSDs but that there was no reason, in principle, that they wouldn’t work. The advantage of a network cache is that unlike a server-based cache it can still be accessed if the server goes down. With a network cache you can afford to build in all the HA features. Other thoughts?

Robin

I would imagine most of the power used by an HDD is used by the motors, not by the electronics, right?