lucene and nutch developer doug cutting for sure has some search tales to tell: "web search is a commodity"

Word has it that huge advances in web search are just around the corner, and that the big web search engines have lots of fancy secrets that make their search better. Folks speak of web search as a specialty product, not a commodity.

Bah! Web search is a commodity. Google hasn't fundamentally altered its search engine in over five years. If companies have big new web search tricks up their sleeves, don't you think we'd have seen them by now? And Yahoo! was able to more-or-less clone Google's web search in fairly short order.

Google is more like Coca-Cola than like Eminen ... (finding at cutting's blog free speech)

a historical tale by now: the anatomy of a large-scale hypertextual web search engine, the work of sergey brin and lawrence page at stanford

In this paper, we present Google, a prototype of a large-scale search engine which makes heavy use of the structure present in hypertext. Google is designed to crawl and index the Web efficiently and produce much more satisfying search results than existing systems ...

maybe an ant algorithm can help? artificial ant colonies in digital image habitats - a mass behaviour effect study on pattern recognition.

maybe a place to find some answers: the edge question: "what do you believe is true even though you cannot prove it?" (thanks for the pointer godany!)

We are in the age of "searchculture", in which Google and other search engines are leading us into a future rich with an abundance of correct answers along with an accompanying naïve sense of certainty. In the future, we will be able to answer the question, but will we be bright enough to ask it?

another place to screen: search engine watch