What’s “Best” About the Best Search Algorithms?

If a search tool is any good, people are going to use it to make decisions, many of them involving money. And as soon as that happens, there’s a monetary incentive to game the search engine. The best search engines, then, aren’t necessarily the ones with the best initial results: they’re the ones most resistant to gaming. Or, more accurately, search engines that can most easily be gamed by helping users of that search engine.

The early search engine algorithms weren’t as naive as they look now: merely counting the words in a document, especially the words the author used to describe that document should have worked well. If nobody knew what search engines were, it would still work. Few academics or judges game searches for academic papers and legal opinions, which allows naive algorithms to persist in those areas.

But the naive result is easy to game, and the gaming is at best neutral for the user’s experience, but most likely to be harmful. Pagerank elegantly sidesteps this problem by demanding that users get links, and especially that they get links from people who get linked to often.

That’s still a system people can game. But the path of least resistance is, at some level, to contribute interesting content to the web. If you want to outrank someone for keyword X, you generally need to build a page that is even better than the previous best page on the topic of keyword X. Granted, that means some combination of “Better content” and “Better self-promotion.” But self-promotion is just Google outsourcing the discovery of good content to the people who create it.

When search engines make a decision, ask yourself: does this make gaming more or less benign? If it’s the latter, don’t expect that decision to stick.