I ruffled a few feathers with a posting on AV Testing corruption earlier this year. The posting provoked a big reaction, with almost every major AV company commenting in their blogs, with the exception of Symantec and Microsoft.

I never responded to any of these postings. Why?

Well, as a number of vendors deduced correctly, that there was a mole right in their midst, who was blowing on a whistle. There was thus a desperate rush by some of them to “shoot the whistle blower”, so I thought it best not to stir the pot again, for a while. A few months later, my mole gave me the following report on the AV vendors reaction to the corruption of AV testing:

1. There are several AV vendors that were outraged at the idea that AV testing could be skewed. However, in conversations with their labs and other “reputable” testing companies, their concerns were confirmed.
2. Others knew very well what was going down and thus were anxious to know “who was the one to give Robin all the info”? This confirms that this has been known and recognized by several vendors. (I just listened and didn’t say boo).
3. Some of them took exception to your blog, but since you had so many details they figured you had to have had an “inside source”
4. Some people wished dearly that you had named names in the blog. (Well names could be named, but I’d rather let the industry have a chance to clean itself up first – RB).
5. I attended a session with representatives from Microsoft, Symantec, and several others. The corrupt AV Testing issue came up and most were upset about the blog – for two reasons 1) It casts doubts on the validity of all existing tests (including some that might show them favorably) and 2) because they now knew that they weren’t given the same “advantage” as the others had. (It’s funny….they’re mad because they didn’t have a chance to cheat like the others did).
6. The ones attending this meeting weren’t the ones that did well in the published tests, so it’s obvious to see who was gaming the system and who wasn’t.

I might add another point to all of this. AV Testing is pretty meaningless when you have products like whitelisting products which work in such a way that they will always score 100 percent on any such tests. As whitelisting is now becoming mainstream (Symantec went over to it recently) AV Testing is set to become a declining activity.