I was briefed by Endeca recently. It wasn’t the first time. I’d run into the company a few years ago when I was doing a broad sweep through search companies to try to get a grip on how good the technology was. Endeca stood out for me then, simply because they had—as far as I could tell—bridged the gap between search and navigation. In the last two years they’ve improved the technology considerably.

So here’s the problem:

Searching is not (or shouldn’t be) a normal activity. In our normal activity we only search for something in exceptional circumstances—when it gets lost. Ideally we have a place for everything and everything is kept in its place. We find things by ‘navigating’ to where they are and then using them. Consequently it’s more natural for us to have a structured organization of things (including information) and to find what we need by navigating through the structure.

However, out there in cyberspace, there is a vast amount of information and not much of it is organized according to any kind of standard. Some sites have good internal organization (like Wikipedia, or eBay or Amazon), but there is no common taxonomy in any of this. That’s why the price comparison web sites are so clunky.

So if you want to find something in cyberspace, you don’t navigate to it, you search. At least that’s the assumption—and the interesting thing about Endeca is that it challenges that assumption. Before you leap to the wrong conclusion, lets first state that Endeca doesn’t (at the moment anyway) try to provide any kind of web search capability and, as far as I know it has no intention of doing so in the near future. The data volumes are far too large—it involve would involve boiling the ocean. Better to let global warming do that.

Endeca is in the unfortunate position of getting compared to other products that are not really like it. What Endeca does is build a kind of taxonomy, from the ground up over a set of files which could be databases (Oracle, or Access even) or could be text or HTML content. When you build a taxonomy of this kind you are, in effect, building a metadata structure—simple to do with a database, but actually quite possible with text or indeed anything that is tagged.

What Endeca then does with this metadata/constructed-taxonomy is build a kind of Excel pivot-table over the data. If you cannot visualize that then I’m not surprised, you’ve probably not messed with pivot-tables. So, to put it another way, it builds a series of navigation paths based on the metadata. Now without trying to explain why (because to be honest, there’s a great deal more that Endeca does than just mess with metadata), what you get is a very easy to understand navigation structure that has its claws firmly lodged in the whole data pool. And it works.

Endeca dynamically builds a navigation structure that works and, remarkably, is very easy to comprehend.

Endeca has been selling like hot cakes into certain application areas. One such area is web sites. The leading D-I-Y store in the USA, Home Depot, installed Endeca to provide the navigation for its ecommerce operation—replacing its previous solution and the outcome was a 30% increase in web sales simply because web site users were able to find what they were looking for.

And that’s what’s impressive about this technology. It’s not that it creates a navigation structure—there are quite a few ways to do that—it happens to create navigation structures that work very well, on the fly. If you’re looking at search technology, you should take a look at Endeca. It could be what you need.