It gushes money
Gartner’s business model is genius. They gather information from vendors and users – for large fees from both – and then sell that information back to them for even more money. Bliss.
They own a toll booth on the user/vendor information highway. And collect $1.3 billion a year from the traffic – over $300,000 per employee. Drool.
But the best is the Magic Quadrant, Gartner’s money-spinning qualitative graphic. You’ve seen it, but here’s a blank version.
The “magic” is how it gets tech execs leaping, like spawning salmon, for the upper right corner.
The little engine that couldn’t
Everyone toes Gartner’s line. Until now.
Beth Pariseau posted on ZL Technologies’ suit against Gartner . ZLT hopes to force Gartner to provide:
- Fair Disclosure on Conflicts of Interest – Gartner generates its revenues from payments made by the same vendors whose products it evaluates. Similar to the new rules now being imposed on financial ratings agencies on Wall Street, Gartner should be required to disclose the revenues received from the vendors it ranks.
- Fair Disclosure on Evaluation Scores – The tech industry would benefit if Gartner were required to disclose more data in its evaluation process and disclose component scores so vendors know exactly where they are lacking and by how much and take corrective action. Currently, there is zero disclosure, which can lead to arbitrary placement, with no recourse and no basis for appeal.
- Better Oversight – Gartner currently has an employee act as ombudsman to handle disagreements. The conflict of interest is self-evident in the way ZL’s concerns were summarily dismissed with little supporting evidence. There is a crying need to establish an impartial ombudsman similar to those found in public media, in order to ensure purchasers that they are receiving impartial analysis.
Sour grapes?
In its court filings ZLT talks about the harm it suffers caused by customer reliance on the MQ. I’m not surprised.
As Geoffrey Moore noted in Inside the Tornado IT managers are herd animals. They know there is safety in numbers – in keeping their jobs and getting problems solved – so they like to stick together.
The power of the MQ is that it captures this mentality and gives it a graphic form that comforts even the most technophobic CFO/CEO in a few seconds. IT may not have the foggiest notion about the firm’s 5 year requirements or what implementation will entail, but by going with the big guy they have an alibi and an escape plan all in one.
MQ criteria
Gartner’s lumps all the MQ verbiage under 2 headings:
- Ability to execute
- Completeness of vision
Ability to execute
Ability to execute favors the large. If you’ve got 5,000 engineers and free cash flow you can, eventually, execute anything. Never mind that a team with 10 smart engineers and a clear vision will move faster and smarter to solve a particular problem: tiny CDC built the first supercomputer, not IBM.
Many of the criteria explicitly favor size: global presence in large markets; “viability;” market share; marketing and sales to drive acceptance; and more. You’re a smaller company? Tough. It’s Gartner’s quadrant and you may not live in it.
Completeness of vision
The large company bias here is the way markets are defined. For example, the mid to high-end NAS MQ excludes vendors whose focus, today, is not “primary file systems storage, instead of storage that narrowly targets backup or archive data.”
But the available research finds that most enterprise files are created and accessed only a few times. What then is the difference between “primary” and “archive” storage? Shouldn’t Gartner raise their enterprise customer’s awareness of this and other file storage issues?
Gartner analysts read the research. Why doesn’t the MQ reflect the best information in major use cases, instead of reinforcing popular prejudice? Doesn’t Moore’s Law steadily move the bar upward for what “narrowly” focused systems can do?
Mid to high-end NAS that includes both scale-up and scale-out – as Gartner’s market definition does – is lumping 2 very different markets together and blurring the distinctions. Rather than accepting vendor market definitions, Gartner analysts should be in the forefront of defining new market segments.
For $100,000+ per year a CIO should expect no less.
The StorageMojo take
The Magic Quadrant has the analytical rigor of a beauty contest. Implicit and explicit assumptions about customers, markets, technologies, use cases and suppliers obscures more than it reveals. The MQ seeks to rank vendors not only by what their products do, but by what Gartner presumes an enterprise customer should want. They presume too much.
Marketers often disagree about what constitutes a “product:” is it the widget itself; the widget + services; or the widget + services + ?? In truth, customer perception of what constitutes a product changes with time and experience. But Gartner is stuck: if someone has a better widget – as ZLT says it does – there is no way that the MQ will tell you that.
Enterprise IT staffs abhor change, so Gartner could argue they are meeting customer needs by rigging the MQ to favor incumbents. But the current crisis and the need for greater efficiency and cost-effectiveness means decision makers need better data and informed opinion.
If ZLT, for example, can search archived emails 1,000x faster than Symantec – as they claim – then Gartner should disclose that for the customers for whom performance is important. Get feedback from actual customers – what Gartner does today – about ZLT and pass it on. Don’t just ding them because they’re small.
Customers aren’t idiots; they can see that a company isn’t very big. What they don’t know is how well their products work.
Gartner needs to start earning that $1.3 billion, not just collecting it. If the FTC can require lowly bloggers to report vendor freebies and payments, perhaps the day isn’t far off when mighty IT consulting shops will have to do likewise. Kudos to ZLT for noting the emperor’s scanty attire.
Courteous comments welcome, of course.
Related posts:
- Let Gartner tell your story? Looking at ZL Technologies suit against Gartner (seeGartner’s magic hydrant)...
Related posts brought to you by Yet Another Related Posts Plugin.
{ 11 comments… read them below or add one }
People that take Gartner’s word for anything deserve what they get. If a coin operated intelligence company is what you want then save yourself the money and go with what your confirmation bias tells you to do. If you are an IT professional that needs “an alibi and an escape plan” you probably aren’t very professional anyway. Gartner’s only real use is to provide examples of where not to look for cost effective solutions.
It is about time someone wake up with these analysts who have been not so great predicting the future for some times. I include all large analysts firms in the pack.
What is their predictions success ratio over long period?
Have they been wrong predicting iSCSI or other technologies success? Even with there massive content gathered at big price?
Is there a conflict of interest here? They should disclose how much they received per vendors.
If it is true for blogger and comments posted on the internet it should be also true for analysts.
Between paying Gartner to think for them and vendors to do the infrastructure, it’s no wonder business execs want to adopt technologies that get rid of no-content vendor pass-throughs that walk the halls calling themselves “IT staff”. Filling out purchase orders and plugging in ethernet cables do not provide competitive advantages and if you cant get a competitive advantage from your IT, then why not outsource it?
It’s sad that so many people think that technology decisions can be broken down into a 400×400 pixel psudo-graph and a 20 page pdf, 5 pages of which contain no content, and the remaining 15 of which contain marketing drivel and buzzword Tourette’s. It’s a indictment on the industry that ideas from these puff pieces can get so ingrained in peoples minds that it takes 10 or 15 years to undo the damage.
My company has an informal policy — Don’t ever cite Gartner as an information source, because their analysis tends to be biased toward where the money is.
I got interviewed by Gartner about a specific storage vendor once. There were a bunch of stupid 1-10 question. When asked to rate vendor’s service and other things, I answered them with: if EMC is 5, these guys are a 10. The person on the other side grew increasingly frustrated by my answers. Then they asked me another set of equally retarded questions one of which was if I were concerned about vendor’s financials. It is not about rating the product or service or anything important. It is about providing a false sense of security to the incompetent. Anyone stupid enough to either pay Gartner or base any of their decisions on the snake oil quadrant deserves what they get.
Gartner is a tool to be used by knowledgeable professionals to help them understand the market, narrow focus and choose product. Nobody buys right off a list. Although you and others may not like the 2 x 2 chart, it provides a good shorthand of the overall market in ways that are easier to talk about than a stack of thousands of pages of product information. If a IT pro or a CIO is just looking for a defensible position, they could spend a lot more money getting a lot less value by hiring some of the consulting organizations I have dealt with.
As for ZL’s product, its a niche that is not covered adequately by Gartner say ZL. Big deal – they need to get to the decision makers and make the sale. The lack of proper placement on the Gartner MQ is a great rallying cry but does not really solve their sales problem. The lawsuit is great bit of theater but it will not redefine the size of their market so that Gartner or anyone else will judge it separately.
Disclosure – 20 years ago, I used to work for a Gartner-like company. Although there were issues with some of the work we did, the value proposition was a fairly robust one. As a user, they spent $10 to 15K per year with us to get information that would take five full time people to compile. Not unbiased, not perfect but a lot of information boiled down and analyzed for pretty small fee.
Little known fact: Gartner doesn’t install and use the software they recommend. If they did, their choices would be very different.
Gartner simply ranks software by feature lists and how big/viable the company is. This encourages big enterprise companies to just add features to move up and to the right of the MC. Not create good software.
To be fair – the guys who invented the Magic Quadrant have a blog and they themselves warn against the “we only evaluate in the leader quadrant” thinking.
Anyone who calls themselves an IT expert and then relies solely on the Quadrant to make decisions is just plain lazy. Sometimes the extra information below the graph adds more than the graph.
It really comes down to how much work the CIO does when assessing a solution. In some cases availability is important and speed of change is not important – like storage – you want a stable company but features are not so important. In some cases features are all important and stability not so much. It may be worth finding a “visionary”. In some cases you may be looking for a “Niche Player” who will worship your business and deliver the exact solution that you need and not confuse matters with many features that just get in the way.
A bit of pre-planning makes all the difference.
Now excuse me while I drink this kool-ade.
Gartner’s “analysts” are largely inferior, most of whom add virtually no insights to the discussion. Like IDC and Forrester, Gartner sells content and preferred placement in the Magic Quadrant to the highest bidder. In the software space it’s Microsoft, Oracle, SAP, etc. In the hardware space it’s IBM, HP, etc. In the profesional services space, it’s Accenture, Deloitte, blah, blah.
Any fool who trusts Gartner’s analysis – to make any decision – is simply ignorant to the industry’s sleezy pay-for-play practices.
Even the non-technical CTO/CFO should know that Gartner is to IT what Moody’s or Standard & Poor’s are to the financial sector; pay-for-play scam artists who perpetuate the fraud in collusion with these lazy and ill-informed decision makers – all at the expense of innocent IT staff that has to make these sub-par products/solutions work in an ever complicated operational environment.