http://www.virtual-strategy.com/Features/Abstract-Microsoft-and-Google-Cloud-Computing-Dominance-Through-Renewable-Energy.html

As a neutral third party, I can see the basis of Robin’s opinion in his remark to you. I’ve been on the net since 87 or so in one form or another. Robin is well-advised to keep even hints of unnecessary rancor from his communities. It’s never fun when things turn personal, I think.

Anyway, this is a bit of an industry joke for you:

“For file system overhead being generally low, I see that you unaquainted with NetApp”.

*ba-da-bing*

On to the next subject, and for COPAN, you are correct. You need a VTL like use case. “Store a great deal, access seldom”. The interface need not be a VTL interface, but the use case ought to be.

We use “per usable GB” measures of our merits in our TCO analyses for a very good reason. Various vendors with storage appliances (NAS based ones, notably) have variable and often high file system overhead. There are also limits in aggregate sizes that make RAID overhead high with some vendors and not others.

Joe.

I want StorageMojo comments to be a place were people who know less than you feel comfortable asking questions and venturing opinions. Right now, I’m pretty sure that isn’t the case.

You know how to express yourself politely. Please do so from now on.

Sincerely,

Robin

My main concern was whether somehow the raw disks were consuming far more power than Seagate claims they do, because $0.25/GB for just the raw-disk power over a 5-year life *would* be significant to people who elect to use software solutions for managing their redundancy (hence don’t have the added expense associated with high-end hardware controllers to drive aggregate power costs back down into the relatively unimportant category - though management costs could still eclipse up-front and power costs in many cases).

The COPAN number is impressive, though I suspect assumes something much closer to backup/archival than to primary-storage use (allowing them to keep most disks spun down most of the time). With primary disk storage, there’ll always be a trade-off between spreading out the hot data across as many spindles as possible for performance and being able to spin down idle disks effectively, I’m afraid.

Incidentally, I just checked our local utility’s power rates, including transmission and distribution charges of various kinds: they are (as you found with yours) flat for large businesses (over 1,000 KW demand) regardless of amount of use (save for cases in which the business’s demands are peaky rather than stable, where they wind up paying a bit extra to account for the additional *potential* capacity that typically goes unused in whole or in part). Small businesses (under 100 KW demand) pay less per KWH the more they use, again with a penalty for peakiness. Intermediate sized businesses pay rates between those for large and small busineses, and less per KWH used the more they use (so basically all businesses pay less per KWH the more they use up to the large business category where the rate flattens out at the lowest figure).

Interestingly, the base energy price is the same for all users and regardless of demand: it’s the distribution and transmission prices that vary. Residential and large business users also can get price breaks for (metered) off-peak use.

Finally, there’s that matter of moderation constipation again. While Robin managed to unstick his process for some of my posts, one remains hung (and another recent one that asks why as well: since he was unresponsive to email and suggests on his “Contact & About” page that comments are another means of communicating with him, I tried that). The posts in question are well within the ‘moderation’ bounds that he describes on the “Contact & About” page, and I’m being forced to the conclusion that he’s just censoring me in a surprisingly (to me, not knowing him save for my contact here) arbitrary and petty manner.

So this constitutes another probe to see whether whatever criteria he’s using may relate to a post’s content in its entirety. If you’re curious what the fuss is about, I’d be happy to email the posts to you so that you can form your own conclusions (as I’ll do if this post also gets stuck in the queue).

- bill

There are multiple components to a utility charge, but generally speaking the costs of the energy that the utility supplier must overcome in order to provide electricity service profitably come in two flavors: a fixed capital cost and the variable costs associated with supplying the energy. The former refers to the capital cost of constructing the power facilities, which are amortized by the utility supplier over a number of years (the exact # depends upon the terms of the loans used to construct power facilities). In order to offset the amortized debt cost, the utility company assesses its largest customers a “demand charge” that is established based upon peak usage that’s often determined over a rolling multiple-month average. Penalties are paid if a customer exceeds this peak usage, often times at 2x the rate that is charged at usage below peak. Bottom line is that it is crucial that electricity usage stays below the peak amount that a company has indicated its need would be.

The second component to the utility charge, of course, is variable based upon usage. The costs to fuel, maintain, and operate the power facility are often passed-through to customers who use a small amount of electricity. In many cases, however, higher usage corresponds to higher per-kWh energy charges. In fact, the per-kWh rate charged to commercial busineses often stairsteps, depending upon the amount of electricity consumed. Therefore, a data center that has to pay for the added costs of cooling can easily find itself in a higher usage bracket, as was the case with the example I mentioned. Also, I should mention that many utilities charge higher rates in the summer vs. the winter, indicative of the higher costs associated with cooling infrastructure at the data center and the added burden placed on the power supply at the local utility.

Anyway, all this to say that it’s not always a “one charge fits all” in the case of electricity. If anything, power plant economics center around the ideas that a) unexpected spikes in demand are costly and must be adequately offset through necessary charges, and b) the more power that’s consumed, often times, the more that must be charged on a per-unit basis, particularly when electricity supply is constrained.

By the way, as part of the feedback from this article, I was made aware that Microsoft charges its customers that subscribe to its data center services not on a square-footage basis as many co-lo’s do but rather on a per-kWh basis. Mr. Todd, I thought you’d find this interesting. Go do a search and you’ll find it, I believe it was mentioned by Microsoft about a month ago. I would guess more will change their pricing metrics to Microsoft’s methodology in the months ahead.

If you would, gentlemen, please don’t parse each and every word, I am not able to visit this site all that often so let’s cover this in more general terms. We can argue this for weeks, which is undesirable. I just want to get the crux of these main points across. Thanks.

–Steve

The costs are per net usable GB, yes (although we do not mirror). The power TCO costs are for all power, including the controllers, etc… and less we forget: JBOD systems and their power supplies do not necessarily pull at the stated idle/average times of the drives within them. And of course are 5 years of all those things.

To be clear, I do not “see” drives or their power requirements. I see multipetabyte proposed storage solutions, with min/avg/peak power ratings for full trays of storage and the control infrastructure and the like, at the system level.

Those are where those numbers come from.

You will find yourself not dismissing power costs in a full TCO calculation, but also finding that the most important things to concentrate on are $/GB CAPEX and labor. Things like power, space utilization, and disk replacement charges will fall into a more holistic assessment.

BTW, I’m seeing 25-30kW/PB typically from a survey of 14 nearline offerings (net usable, not mirrored: RAID-6 or equiv, and after file system subtractions). It would be worse in Tier-1 of course.

BTW, looking over my figures, $0.24/GB is actually the hottest vendor. Some fare better; $0.20 is pretty typical. Some of the cooler vendors, $0.14. COPAN $0.04.

Joe

It looks as if you wrote your last response to me before Robin got around to posting the addendum which I added to the post to which you were responding, describing the fairly consistent decrease in *per-platter* power requirements over time within a single Seagate drive family - a progression which helps drive down power/GB even faster than the simple increases in areal density do (and thus helps these reductions equal or even exceed the $/GB price reductions - the point of my original response to you, though I’d still like to know more details about the isolated example to the contrary which you came up with).

So while I agree that first instincts can be misleading (in this or almost any area), I’m not sure I agree that the situation is particularly complex: while it’s certainly necessary to go through very specific calculations to generate a funding proposal, merely characterizing the general trends (and how relatively important they are to costs) is pretty straight-forward (though still a matter of elementary-school arithmetic rather than mere ‘instinct’).

In that vein, I’m interested in the specifics of how your TCO analysis arrived at the conclusion that the 5-year power & cooling costs of Tier-2 enterprise SATA storage were $0.25/GB (or $250/TB), since at the $0.08/KWH pricing that you referred to earlier the 5-year power cost (not including cooling, which you earlier WAGged at an additional 30%) of a contemporary 1 TB Seagate Tier-2 enterprise SATA drive is only $40 (or $0.04/GB) at the highest rate of power consumption that they tested for. Even their previous 750 GB model would have required less than $47 to power it for 5 years (or about $0.062/GB - still less than 1/3 of the number that your analysis came up with even after adding another 30% for cooling load).

Are you by any chance quoting costs per *net* GB (e.g., that would require 2 GB of raw storage in a mirrored system)? Do controllers and other SAN hardware add significantly to the power use that you’re including? Are any costs that are not directly proportional to power costs included (e.g., costs of power or cooling equipment)?

Please understand that I’m not asking for any information that might be considered proprietary: since you felt free to volunteer a number, I’d just like some general idea of what it actually reflects.

Thanks,

- bill

I also suspect that he’s talking about a data center that comes nowhere near your own experience that “Power to the cooling is typically 50% or less of the power to the equipment being cooled (generally, I wag 30%)” - and hence that the facility he’s describing could be doing a *lot* to decrease their electric load required for cooling. If so, the fact that they are not taking such steps reflects the relative unimportance of even such inflated energy costs to them (contrary, as usual, to Steve’s basic premise that electricity is too expensive and scarce for storage growth to continue).

So when we toss in your own additional experience that a) “A major data center we support pays $.08/kWhr, no matter how much they use” and b) “power costs of both power and cooling, as well as the capital costs of the power and cooling equipment, associated with storage are all only a small part of the TCO of storage”, I find it somewhat difficult to reconcile your stated position with Steve’s assessment of it (”I’m glad to hear you, who is clearly an expert in this area, agree that this cost makes utilities a meaningful consideration in the storage TCO equation”): once again he appears to be confusing your *conditional* agreement that *if* electric rates actually *were* $0.20/KWH *then* the cost of powering both the storage and the required cooling equipment would become a “fairly notable line item” with agreement that it already *is* a fairly notable line item.

Then again, your last statement (”The real problem is supply and base energy costs”) makes me wonder: are you agreeing that some ‘real problem’ (as contrasted with a relatively minor line item at worst, as you seem to have been saying up until now) with storage energy use actually exists, or just the more mundane observation that conservation measures by definition have limits after which the only way to further contain costs (ignoring the question of whether they are of such a magnitude that such containment is important) is to reduce prices?

(Given the moderation constipation which my own posts encountered there a couple more things I’d like to discuss, but I’ll make them separate posts to try to keep the points separated.)

- bill

If you’re argument is that the “green” fixes are mere low hanging fruit patches, that are nothing more than stopgaps, I generally agree. We’ll see a big boost from MAID-technology variants (for those who have relatively inactive nearline use cases), another boost from SSD’s (assuming that they can get past the unrecoverable read incident rate problem), and then for the most part we’ll be back to sucking the local municipal grids dry.

The real problem is supply and base energy costs.

Joe.

Usage certainly depends upon region, you’re right. I guess it all depends upon the overall supply of the energy in the region being considered. Those utilities who increase the charges for higher degrees of use, you have to assume, are in regions of tigher energy supply. Which goes back to my original thesis: we need more energy supply, over the long run, to keep usage costs down. These higher usage pricing thresholds are a killer. Redesign of compute infrastructure to accommodate lower supplies of energy is a short-term fix. At the rate of growth of data center computing, these “green” fixes will be quickly offset. This is the crux of my argument.

Usage is one thing, categorizing it is another. I’m sure this will matter entirely based on region. A major data center we support pays $.08/kWhr, no matter how much they use, and since the entire data center is for the powering and cooling of IT equipment, that’s that.

You understand, though, that unless your power is metered separately, there’s no way to know what’’s what, except to say you exceeded your budget and are paying penalty fees.

Joe.

p.s., as an aside, $0.20/kWh would start to make the five year TCO for powering storage, at least in nearline, a fairly notable line item (particularly if you are like us, where you’re nearline storage is costing more towards $1 than not).

This response is also something of a test, since several of my recent posts have been ‘awaiting moderation’ for quite a while now, and Robin hasn’t been responding to emails asking why.

- bill