Monday, March 19, 2007

Upgrading to Web 3.0

The term "Web 2.0" was coined sometime in 2004 by Dale Dougherty of O'Reilly Media. Far from a new technical standard, Web 2.0 began as a marketing gimmick for the eponymous Web 2.0 Summit, an annual high tech trade show that will be in its fourth year as of October 2007. As Tim O'Reilly wrote in September of 2005:
"The bursting of the dot-com bubble in the fall of 2001 marked a turning point for the web. . . .

" . . . Far from having 'crashed,' the web was more important than ever, with exciting new applications and sites popping up with surprising regularity. What's more, the companies that had survived the collapse seemed to have some things in common. . . ."
Far from a universal or even ubiquitous technical meme, Web 2.0 has nonetheless come to mean many things to many people in the high tech industry while meaning absolutely nothing to others. But this hasn't stopped many people in the know from calling for a Web 3.0 upgrade. The first person worthy of note to call for such an upgrade was Jeffrey Zeldman, who did so tongue-in-cheek as part of a rant against the Web 2.0 meme:
"When I started designing websites, if the guy on the plane next to me asked what I did, I had to say something like 'digital marketing' if I wanted to avoid the uncomprehending stare.

"A few years later, if I told the passenger beside me I was a web designer, he or she would regard me with a reverence typically reserved for Stanley-Cup-winning Nobel Laureate rock stars.

"Then the bubble burst, and the same answer to the same question provoked looks of pity and barely concealed disgust. . . .

"[ . . . ]

"Eventually . . . [t]he web was "back" even though it had never left. . . .

"[ . . . ]

"But how to persuade the other sharks in the tank that this blood feast was different from the previous boom-and-bust? Easy: Dismiss everything that came before as 'Web 1.0.'"
As a technical meme, Web 3.0 assumes the validity of a still emerging Web 2.0 meme, and then asks, "Who and what will be left standing when the next economic bust and boom cycle has finished ravaging the Internet?" I'd like to think that sexy applications like natural language processing, artificial intelligence, and the Semantic Web will rise from the ashes. However, I've been through the cycle of disruptive technologies too many times to expect this sort of purposeful progress.

Saturday, March 10, 2007

The Waning Relevance of Search Engines

When the Web was young -- i.e., before the advent of Yahoo! and Google -- USENET[sic] was the primary medium for fielding informational queries on the Internet, and USENET FAQs were the primary medium by which such queries were answered. Incidental to such FAQs were pointers to particularly useful URLs that people had discovered and vetted by following trails of informational bread crumbs. Most of these trails started on USENET. However, over time, more and more Home Pages on the World Wide Web became populated with links to interesting and useful websites, and then along came the spiders.

Spiders (now more commonly known as web crawlers) created the first indexes[sic] of the World Wide Web by following links from one URL to another and making a record of what they found along the way. An individual spider or collection of spiders working together can produce a somewhat comprehensive database of URLs on the World Wide Web. However, that database is (at best) a collection of snapshots from some time in the recent past rather than a live feed that includes recent changes, and it will not include URLs found on "The Invisible Web." These distinctions were once lost on most people, who assumed that a "'Net search" would magically provide up to the minute information from the Web; this distinction is still lost on some people who have no idea how a search engine works.

The term search engine predates the Web. The earliest reference to the term that I've found was by Professor Lee A. Hollaar of Utah University back in March of 1985. (The Utah Text Search Engine: Implementation Experiences and Future Plans, Proceedings of the Fourth International Workshop on Database Machines.) But the term really caught on with the Internet community in the early 1990s. By that time, spiders were already cataloging an enormous amount of content found on the Visible Web. Even so, very few people otherwise in the know appreciated just how important search engines would soon become, erroneously assuming that anything more complex than the Unix "grep" query was overkill.

A former colleague of mine was a product manager for Netscape before the company was acquired by America Online, and he once recounted to me how popular the search function was on the Netscape website from the very beginning. Meanwhile, almost everyone at Netscape was trying to figure out how to sell the company's Web browsers and its easily forgotten line of related software products, all of which became completely irrelevant when Microsoft started giving away the Internet Explorer Web browser. Even Yahoo! failed to recognize the importance of search-related products, quickly diversifying from a searchable Web directory into a Web portal and outsourcing its search services to Google until a few years ago.

For all its flaws, the Google algorithm is still the standard by which all other search engine algorithms and post-Google information retrieval mechanisms are judged. The only post-Google innovation in content indexing and retrieval that even comes to close to being such a standard is Wikipedia, and most of those who consider Wikipedia a successful innovation do so because of Wikipedia's prominence and visibility in Google's search results. Even the blogosphere's importance is largely validated by the impact that it has on Google's search results. However, the importance of search engines reached its zenith a while back, and their relevance is slowly waning.

As I reported previously, Wikipedia recently announced that outbound links from Wikipedia to other websites would include a "nofollow" attribute. The rationale for this decision is that it will reduce the incentive for "black hat" search engine optimizers (SEOs) to spam Wikipedia, as a link from an important website like Wikipedia will normally inflate the PageRank that Google assigns to that link's destination URL. However, I think the end result will be something quite different. To wit, to the extent that Google actually ignores outbound links from Wikipedia, Wikipedia will actually offer something unique and different from Google to its end users. I don't think this will put Google out of business, but it will eventually diminish the relevance of search engines as the primary medium for fielding online informational queries.

In a vein similar to Wikipedia, sites like Digg, Fark, Technorati, reddit, and del.icio.us are emerging as places for the purveyors of information to hook up with their audiences with less and less concern for the impact that such sites have upon a(n) URL's Google PageRank. Moreover, like Wikipedia, certain online portals are so prominent that the only reason anyone uses a search engine to find them is because many end users are not in the habit of using bookmarks to pull them up -- e.g., eBay and MySpace. Even people who choose to use a search engine to find websites with the products, services, or information that they want sometimes complain that highly commercialized search results require a serious researcher to dig to the second or third page of search results to find something useful. In sum, while far from obsolete, the relevance of search engines is slowly waning, and commercial interests are slowly turning their attention to other channels of online marketing.

Friday, March 09, 2007

Jimbo Wales Feeds Wikipedians and the Press a Red Herring

According to Brian Bergstein of the Associated Press:
"Following revelations that a high-ranking member of Wikipedia's bureaucracy used his cloak of anonymity to lie about being a professor of religion, the free Internet encyclopedia plans to ask contributors who claim such credentials to identify themselves."
Discussions regarding this proposal are still underway at Wikipedia, but when news of the above-quoted article reached Jimbo Wales Wikipedia User Page space, it became quite clear to me that Jimbo had already made up his mind about moving forward with this plan, and that the discussions he had initiated on his user page were simply a way of placating the opposition by giving them an opportunity to speak their mind.

As someone who has repeatedly asserted Jimbo's lack of culpability in the recent Essjay scandal, I am very disappointed to see that Jimbo has offered up the red herring of verified credentials rather than address his [Jimbo's] failure to properly vet a pseudonymous individual whom he appointed to ArbCom, Wikipedia's court of final appeal. Superficially, the Essjay case was about falsified credentials; at its core, it was about an elaborate deception that Essjay rationalized as being necessary because he held positions of trust at Wikipedia.

Verifying credentials will not address the core issue of deception by Wikipedia administrators. All it will do is validate a Larry Sanger-esque type of credentialism at Wikipedia and create an attractive nuisance for credentialists and impostors. Essjay outed himself as an impostor because he felt that he had to so when accepting his paid position at Wikia. As such, the best remedy for the sort of deception that Essjay perpetrated would be a policy that forced people seeking administrator privileges at Wikipedia to provide the powers that be at Wikimedia with their true identity; if Wikipedia administrators wish to remain pseudonymous to the rest of the world, that's their prerogative.

The Digg Effect

The Digg website was launched just over two years ago on December 5th, 2004, and now boasts having some one million users. For those of you who have never heard of Digg, it's a community-based website where members can submit news stories for the community's consideration, vote for the stories that they consider newsworthy, and "bury" the stories that they don't like. In close conjunction with the advent of the blogosphere, Digg has emerged as an alternative news source where stories that would otherwise remain obscure become headline news.

While following the trail of bread crumbs that is the blogosphere, I started noticing that many of the stories I was following were becoming very prominent on Digg. I also noticed that many prominent bloggers attributed their blogging success to "being Dugg," and I immediately thought of how quickly a story could rise to prominence on the World Wide Web by being "Slashdotted." The two concepts share much in common, but the initial seed for a Slashdot story must be planted by the powers that be at Slashdot, and there is no way to formally vote on a Slashdot story to make it more popular.

After voting on a number of stories on Digg, I finally decided to post my own. As of this writing, my story has 16 Diggs and is ranked third in popularity out of some 260 "up and coming" stories under the category political opinion, and (based on my observations of the Digg system) it still has over 3 hours to become the most popular up and coming story. Now the question is whether the powers that be at Digg will decide to "make it popular." If they do, then my ISP will have to make sure that I do not become a victim of the Digg effect.

Wednesday, March 07, 2007

Putting Essjay's Fraud into Perspective

I have done my level best to avoid posting yet another entry on the XODP Blog about the Wikipedian impostor known as Essjay, limiting myself to commentary on other blogs and on Wikipedia itself. However, I was very impressed with a post on Miland Brown's World History Blog that seemed to put the larger issue of Wikipedia's reliability into perspective:
". . . [T]his kind of deception is not new and is not limited to Wikipedia. Does anyone remember Jayson Blair? He both plagiarized and fabricated articles at the New York Times for several years. I do not think Blair has proven that the New York Times is a bad resource though despite his fraud.

"And let's not forget about Stephen Glass at the New Republic. 27 of 41 stories written by Glass for the magazine contained fabricated material. He wrote such fake gems as a 15 year old at national hacker convention and a Church of George Herbert Walker Bush, Jesus Christ. I still like the New Republic as a resource despite the Glass incident.

"People fabricating degrees is not new either. The Chronicle of Higher Education frequently exposes people in higher education with diploma mill degrees. For example, history adjunct Fred Ruhlman at the University of Tennessee was reported at Cliopatria, '(His) 'doctorate' is from 'the American University of London,' a notorious diploma mill and his book, Captain Henry Wirz and Andersonville Prison, was withdrawn from publication by the University of Tennessee Press because of its plagiarism from William Marvel's Andersonville: The Last Depot.' While embarrassing for the University of Tennessee, it hardly means UT degrees are now worthless.

"I can find those fake Blair articles in my library right now in the microfilm versions of the New York Times. Those fake Glass articles from the New Republic are still in the bound periodical section of the library too. However, every edit Essjay has made is being examined and altered if it is found to be problematic. Unlike the mainstream press who have their mistakes archived forever in libraries, Wikipedia can be fixed when the fraud is discovered."
In a previous blog post, I qualified my hope that I would like to see Essjay make a fresh start at Wikipedia under a new anonymous pseudonym with the disclaimer that such a fresh start was not possible unless Essjay recants his assertion that he was offered compensation by Stacy Schiff of The New Yorker. However, I am inclined to reconsider that disclaimer in light of new facts brought to light by the Wikipedia Signpost:
"Based on earlier statements he made, it is possible that [Essjay] used calling cards to phone Schiff for the interview, and his claim may have referred to an offer to reimburse the cost of those calls. . . . Schiff's response [i.e., "That is nonsense."] did not address whether this alternate interpretation was correct."
As a (hopefully) final note, I encountered a comment on one blog that explained to me why some people seem to be judging Wikipedia's credibility so harshly in light of Essjay's fraud:
". . . [T]rusting Wikipedia is . . . a greater personal risk than trusting the New Yorker, regardless of whether the New Yorker is actually more trustworthy than Wikipedia. . . [T]he New Yorker’s puffery about its fact-checking effectively indemnifies [someone] against the risk of looking foolish, in a way that Wikipedia does not. . . .

"This sort of indemnification is precisely what professional journalists are defending when they rudely dismiss blogs–or Wikipedia–as inferior. Rather than make serious, careful comparisons between themselves and alternative information sources–comparisons that would inevitably show themselves to be seriously flawed in their own right–they emphasize the process by which they gather and vet their content."

Monday, March 05, 2007

Cooperative Research and Citizen Journalism

Viewer-supported Link TV, recently broadcast a program entitled 9/11 Press For Truth as part one of a two part special entitled Truth, Lies, and the Press. This program was based on the work of Internet researcher Paul Thompson, author of The Terror Timeline and the website that inspired the book The Complete 9/11 Timeline, which is now part of the Cooperative Research website. I was very impressed with the content of the Link TV special, and I hope to discuss that at some point on my Internet Esquire Blog; what I hope to cover here on the XODP Blog is the Cooperative Research website, also known as the Center for Cooperative Research.

The Center for Cooperative Research is a non-profit enterprise sponsored by The Global Center, a 501(c)3 non-profit corporation, and its content is openly licensed under the Creative Commons Attribution-NonCommercial-ShareAlike license. As open content licenses go, the Creative Commons is not one of my favorites. However, the larger mission of the Center for Cooperative Research is one that I consider worthwhile. To wit:
"The website is an experiment in open-content civic journalism. It allows people to investigate important issues by providing a space where people can collaborate on the documentation of past and current events, as well as the entities associated with those events. The website can be used to investigate topics at the local, regional, or global level. The data is displayed on the website in the form of dynamic timelines and entity profiles, and is exportable into XML so it can be shared with others for non-commercial purposes."
The site's growth potential is somewhat limited because it relies upon qualified professional volunteers to act as editors for the content submitted by member contributors. However, if the Complete 9/11 Timeline is any indication, what the site lacks in quantity of research, it more than makes up for in terms of quality. To wit, I was quite astonished to see just how many news items that should have been headlines were reported by the mainstream press but buried on page 11. Once these news items are put into a timeline format, they make a compelling case for concluding that Pakistan's ISI Intelligence Service was a sponsor of Al Qaeda and the 9/11 hijackers.

Saturday, March 03, 2007

Closure Still Lacking in Essjay Scandal

The beleagured Essjay "retired" from Wikipedia today, and I was prepared to let sleeping dogs lie until I read a post at Andrew Lih's blog:
"Paying a source for a story is an absolute no-no in the normal practice of print journalism. And it struck me immediately how incredible it was [Essjay] would accuse Stacy Schiff, a Pulitzer Prize winning author writing for The New Yorker, of this crime. We either have a serious breach of ethics with Ms. Schiff or another dubious statement claim from Essjay (nee Ryan Jordan)."
Notwithstanding the hyperbole of comparing a breach of professional ethics with a crime, a survey of Google search results for ''paying a source' journalism' indicates that paying a source for a story is, indeed, an ethically murky area. As such, there can little doubt that Essjay was lying, once again, adding a truly bizarre spin to an already bizarre story.

In a previous blog post, I stated that I would like to see Essjay make a fresh start at Wikipedia under the protection of a new, anonymous pseudonym. However, that's not possible unless Essjay retracts this rather bizarre claim about Stacy Schiff. And given the fact that a substantial number of Wikipedians remain stuck in denial and steadfast in their support of Essjay, it is highly unlikely that he will ever offer a retraction or an apology.

Credentialists and Impostors

The recent intrigue over Wikipedian Essjay's phony credentials has inspired Larry Sanger to reconsider Citizendium's registration policy:
". . . . We are very concerned about the credibility of the Citizendium as a reference work. . . . We simply do not want to wake up in five years, to find that someone has done a study of the Citizendium and demonstrated that in fact 25% of all of our contributors are using neither their real names nor pre-approved pseudonyms. In short, we've reluctantly concluded that the honor principle, even coupled with a willingness instantly to ban people like Essjay who are exposed for using false personas, really isn't due diligence."
Larry goes on to narrate the various methods that Citizendium hopes to use to validate the identities of people who wish to contribute to the project. To the unitiated observer, strong security measures like this probably seem to make sense. However, my experience has been that such security measures serve as an attractive nuisance for both impostors and credentialists, and I'm not sure which is worse.

Conventional wisdom would have us believe that credentials are good and that impostors (in this context, someone who pretends to have credentials that he or she does not actually possess) are bad. However, the underlying issue is expertise, and credentials don't always translate to expertise. As such, when someone without expertise uses credentials to win a debate, this is arguably much worse than someone with expertise pretending to have credentials. The problem with both of these situations is that credentials, or lack thereof, become more important than expertise.

While I'm a big fan of expertise, I'm not a big fan of credentials, and I'm even less fond of credentialists, as my experience has been that someone who truly has expertise will seldom feel the need to fall back on his or her credentials and will generally feel contempt for those who make a habit of doing so. Indeed, those with true expertise are usually most conspicuous to me through their modesty and silence, and what concerns me most about Essjay's misguided actions on and off Wikipedia is that they have provided an opportunity for credentialists without expertise to stifle people without credentials who actually have expertise.

Prior to Jimbo Wales withdrawing his support for Essjay and asking him to resign from his positions of trust at Wikipedia, it was my sincere hope that Essjay would see the light and do this on his own. As it is now stands, Jimbo's statement has only partially resolved the situation. To wit, there are a number of Wikipedians who are urging Essjay's critics to back off, just as there are quite a few people who think that Essjay should still retain his privileges as a Wikipedia administrator. Personally, I would like to see Essjay make a fresh start at Wikipedia, and the only way that he can do that is if he resigns all privileges that he acquired under his false persona and creates a new account at Wikipedia under the protection of a new, anonymous pseudonym.

Correction: In the comments section of the above-cited Citizendium Blog post, Larry Sanger has pointed out my erroneous assertion that Citizendium's recent policy change was (1) inspired by the intrigue at Wikipedia and (2) made by him and him alone. I stand corrected. -- DFP,Jr.

Thursday, March 01, 2007

Never Underestimate the Power of Denial

In following up on my previous XODP Blog post about the fraud perpetrated by the Wikipedian known pseudonymously as "Essjay," I came upon a post at Kelly Martin's Nonbovine Ruminations blog entitled, Larry Sanger proven right about Wikipedia. I followed a link to the post from Google Blogsearch expecting to find a cheap shot leveled against Wikipedia by a Langer Sanger-esque Citizendium-loving credentialist, but that's not what I found:
"Larry Sanger has long argued that Wikipedia is anti-intellectual. And, sadly, he has been proven right, by Jimbo's recent fiat appointment of Essjay to the English Wikipedia's Arbitration Committee. . . ."
While I agree with most of what Kelly has to say about Essjay, I don't agree with her characterization of Jimbo's misguided loyalty to Essjay as being proof of Wikipedia's alleged anti-intellectual bias. Rather, I see it as being motivated by recalcitrant denial on the part of Jimbo in re Essjay's recalcitrant denial of his [Essjay's] culpability in perpetrating such a fraud. On this note, an update at the end of Kelly's blog post points to a recent "apology" on Essjay's user talk page at enwiki:
"I would like to clear up an oversight on my part. I was, until this morning, under the impression that in my initial post on this subject (in response to a question from Dev920 made some weeks ago) I had made an apology for anyone who felt they were hurt by my decision to use misinformation. In speaking to various different people, including Jimbo, I did make it known that I was sorry that anyone felt hurt by my actions, and I believed I had done so in my initial statement. On re-reading that, I find I did not; it was a rather lengthy statement I had been thinking about for some time, and I seem to have left out a rather critical element of it. So, I rectify that now, with further apologies that it was not included originally, as I pointed people back to that statement in the belief it was complete.

"I *am* sorry if anyone in the Wikipedia community has been hurt by my decision to use disinformation to protect myself. I'm not sorry that I protected myself; I believed, and continue to believe, that I was right to protect myself, in light of the problems encountered on the internet in these trying times. I have spoken to all of my close friends here about this, and have heard resoundingly that they understand my position, and they support me. . . . I'm also sorry the New Yorker chose to print what they did about me; there seems to be a belief that I knew they were going to print it, and that is not the case. . . .

"For two years, I have poured my life into making this site a better place. That many people feel hurt by my decision pains me greatly, and to them I am genuinely sorry. To the stalkers, the trolls, and the vandals, I am not sorry; they are abusive, hateful people, and they have done far worse things than those whole of the Wikipeida Community, myself included, have ever thought about doing. . . . I have no intention of going anywhere, because to do so would be to let the vandals, trolls, and stalkers win."
I wouldn't go so far as to say that Essjay is paranoid and delusional, but this "apology" makes it quite clear that Essjay is unapologetic about perpetrating this fraud. He even goes so far as to blame two of his victims (i.e., Stacy Schiff and her fact checker from The New Yorker) for not being more skeptical about his credentials, credentials to which he frequently averred when attempting to boost his credibility as an expert in theology. Essjay's earlier statements on this matter left some room for reasonable doubt as to whether he was actually bragging about being able to perpetrate this fraud, but what he just doesn't get is that his earlier excuse that such a fraud was necessary, justifiable and/or excusable as a way of protecting himself from online stalkers just does not hold water.