In the IT Analysis world, IDC is almost as much of a gorilla as Gartner. It is the statistical gorilla. Gartner also has a healthy stats business through Dataquest, but most of the vendors I have worked with view IDC as number one in stats. And even if IDC wasn’t number one, most vendors would give them preference over Gartner for stats, just to keep Gartner in check. However, if Dataquest concludes that you have a bigger market share than IDC does, their stats clearly have great merit.

The Problem of Accuracy

Technical analysts like myself have a tendency to look down on the statistical analysis business, but gathering market stats and sizing markets is not a trivial activity. I know this because I’m a trained statistician (I even trained to be an actuary after I left University, but please don’t tell anyone). The problem that IDC (and any other analyst company doing market sizing) has, is that vendors lie through there back teeth about sales. Why would they do anything else?

If a vendor tries to hoodwink me about sales by pretending, say, that it has four large banks as customers, it doesn’t matter a damn. I care whether a vendor has customers because it’s a validator of whether a technology works and I like to talk to technology users to get a flavor of how a technology is being used, what the negatives are and so on. It’s vitally important to do this to validate vendor benefit claims. So I will ask to talk to customers when I want to probe the technology, but I’ll rarely be interested in market penetration.

Start-ups will often give their technology away or sell it at a deep discount in order to get solid reference sites, but that doesn’t particularly matter much to me. But it would if I were trying to size a market. Imagine that a new market rises up and it consists of say 5 relatively new players. Let’s say a couple of them were involved in some other market and they’ve repurposed their products for the new area of application and the others have built something from scratch. The new guys all claim to have strong growth rates and the “repurposed” players can show you customer lists of 30 or 40 customers. How do you work out what’s really going on?

It could all be smoke and mirrors. It may be that no-one is actually selling much to anyone; the new vendors having planted their products with a few customers for no charge and the older ones are pretending that their existing customer base is actually using the repurposed product, when it isn’t. (This is a real example I know about, by the way, I just haven’t named the market). If you’re IDC you get hired to try to unravel situations like that, and it takes skill. If you’re one of the 5 vendors and you get an analyst or journalist to believe you’re the market leader selling lots of licenses and they write it down, then you grab hold of that and show it to everyone you meet, even your grandmother and the local vicar.

The Stats Are A Big Deal

The second point to make about the IDC business is that vendors need the stats and they use them to plan their business, so accuracy matters. Consider the typical VC situation. A team with a technology wants to borrow a shed-load of money to establish a product in the market. It’s an in-memory database. So how big is the market for in-memory databases; now, in the next 3 years and in the next 5 years? As a VC, aside from being convinced that the product is better than anything out there, I’d want to know who the competitors are, how big the market is, how much of the market you think you can get and how you’re going to execute. Getting that information is not a simple task, so who ya gonna call?

Actually, as a start-up you may not call anyone, or you may simply hire an analyst who has some knowledge of the market to produce a very rough sizing. It depends on context and the need for accuracy, but if you need really accurate market figures, you definitely won’t be calling an analyst like me, it’ll be IDC or Dataquest or possibly a smaller statistical analyst company that really knows your market.

And if you’re Dell Computer trying to plan your approach to the ink jet printer market, there’s only the big stats analysts you’ll even think of calling. Gathering data on such markets involves whole teams of people making calls of various kinds and mapping them against geographical areas. It also involves understanding the relationship between the business market and the consumer market and the tracking of trends in all adjacent markets. Data is maintained year-on-year and refined whenever possible. And marketing budgets and sales campaigns are based on the stats. In such situations the quality of the data really matters.

Other Survey Work

Quite a few smaller analyst operations do survey work. Hurwitz & Associates, whom I work with in the US, for example, does qualitative survey work. In the UK, both Freeform Dynamics and QuoCirca carry out primary research, which can involve extensive surveys – even global surveys. There are also analyst companies like Evans Data Corp, which specialize on a particular focus area (software developers). Survey work can involve working directly for customers (vendors) or simply gather stats and selling (usually high priced) reports that give a detailed breakdown of a market.

Vendors may quibble about the high price of some of these reports, but the simple fact is that the cost of assembling the data is not always recouped. It’s a publishing business and like all publishing businesses, it’s really easy to make a loss, especially in the Internet age.

Information wants to be free? No it doesn’t, it wants to command a high price. (But music, yeah that desperately wants to be free).

IDC and Technology Analysis

I’ve heard analysts refer to IDC as a “bunch of bean counters” and that’s not fair either. IDC is not just a statistical analysis company, it does technology analysis too and it has some good technology analysts. I know this for a fact because my old associate, Rob Hailstone, worked for IDC in EMEA for a while and he knows as much about software development as any other analyst I’ve come across (he’s now with the Butler Group). He is also a very good presenter. Nevertheless IDC is not in the big league as far as technology analysis is concerned. For IDC, technology analysis is more of a sideline.

Note: This posting is one in a series of postings that deals with the topic of dealing with analysts. Click here for links to other postings in the series.