Powerset Redux

In a previous XODP Blog post, I commented on the fact that Powerset had succeeded where Google had failed in wooing natural language processing (NLP) guru Ronald Kaplan to spearhead its efforts at developing a semantic search engine. Unlike most of the commentators that I've read on the blogosphere, I was willing to suspend judgment on Powerset's technology until it came out of stealth mode. However, I remained titillated by the prospect of semantic search finally coming of age, so I did a little more digging, and I found some more information about what Powerset is planning to debut:

"There are two key things here: the use of NLP and the disruption to the search interface. Finally, information retrieval will actually mean information retrieval, not document retrieval. One of the fundamental models of search that may be challenged . . . is . . . that search engines are designed to take people to pages. The more we can understand and summarize the information on those pages, the weaker this model becomes. . . ."

[The above summary comes from the blog of Matthew Hurst, a close friend of Powerset's CEO Barney Pell.]

As promising as this vision of search is, it fails to take into account the cost of search computations:

"The computation for each query has quantifiable costs and benefits. The costs include R&D, computing infrastructure amortization, energy, rent, maintenance. The direct benefit is advertising revenue from the query response; indirect benefits such as market share gained from greater search quality are harder to measure, but can still be estimated. Search engine success is ultimately given by the efficiency with which it delivers advertising revenue given these factors."

This sort of cost/benefit analysis gives rise to an inherent conflict of interest in search engine technology as we know it. To wit, if a search engine's natural search results are too good, there's no incentive to click on paid search results, and the business model breaks down. At the other extreme is an information resource like Wikipedia, with a business model that is based primarily upon a gifting economy. That sort of business model also breaks down when the quality of information it provides gets too good, as it attracts all sorts of people whose primary interest is in getting rich off of those who work for free.

To quote the late Keith Moon, "Sometimes you do alright to steer clear of quality." (A quote that I found misattributed to Pete Townshend.) Nowhere is this more true than in it is when it comes to technology, where a successful disruptive technology (i.e,. blogs) usually starts out as an inferior technology. Assuming, arguendo, that Powerset's technology truly is better than Google's, this is the real challenge that Powerset's evangelists will have to address.