Search Takes a New Shape

Back in the old days – or the ’90s as some call them – we utilized the Internet as an information resource. What’s that phone number, where is that address, where can I buy that product – you had concrete questions and were no longer required to speak to a human to get answers. Sure, there were bulletin boards and Usenet forums for discussion but they primarily involved coding arguments and game walkthroughs. The Internet wasn’t truly upended into a community, and all that that entails, until just a couple of years ago. It was then that the inundation of bloggers collided with social networking and lifestreaming to produce a perfect storm of content. (And when I say lifestreaming, I mean the trend of putting as many pieces of our life online as possible – books we’re reading, music we like, etc.) We’ve now backed ourselves into a corner online, raging against the indundation of content even as we scroll through our fifth page of FriendFeed updates. We recommend well-written articles about navigating through the noise, right after sharing 25 items in Google Reader.

The logical next step in this technological journey is to therefore prune, to make our time online more meaningful and relevent, no matter how small the nugget of information. Whether I’m setting out to qualify findings in a drug discovery experiment or wondering when Amy Winehouse was last arrested, I want the most reliable, relevant answer in the shortest amount of time. The problem is no longer whether the information is out there but rather how we can get to it quickly and accurately.

It’s against this background that I’m seeing a gradual evolution of the semantic search market. What was once viewed as a sector attempting to “kill” Google has become something different in recent months. It turns out that what Google does is 1) pretty hard to do well and 2) pretty sufficient for most people. Creating an all-purpose search engine that answers any query in any form and delivers a more targeted result will take time to get right. I don’t think we’re going to wake up one day to find suddenly filling all our search needs.

Several companies I’ve met with lately have innovative takes on making the Web more meaningful for users. But I’ve noticed that my reaction has gone from, “Wow, I’ve never seen that before” to “Oh, that’s an interesting take on what so-and-so is doing.” In other words, I believe smarter search and discovery have reached a new stage in their evolution. The problem has been well defined and the angles from which it can be approached have been established (at least for the foreseeable future.) It’s now time to focus on the niggling details: how to identify and amass relevant information, present it to users in an easily consumable yet rich visual format, and, perhaps most importantly, establish a user base that will consistently turn to your product as a valued information resource.

I’ve written previously about companies that I keep a close eye on in the smarter search space. These products, along with a couple of new entrants, bear repeating, as they’re molding a new shape for the Internet.

Powerset raised the bar for UI with its Wikipedia search engine, and, perhaps more importantly, sent a tacit message to the tech world that a smarter search engine isn’t going to develop overnight. Regular readers of The Guidewire know I won’t shutup about Silobreaker, a current affairs search engine that really shines in visualization and content extraction. Its entity maps are among the best I’ve seen, providing instant insight into relevant connections between people, places and things. Evri is a newer entrant in the field, with a product that combines the visual maps of Silobreaker with the subject and verb extraction of Powerset to give users a deeper dive into news articles. Semantinet brings to mind Yoono‘s discovery feature, but on steroids, recognizing entities on the page you’re viewing and bringing in related info from other sites. It makes discovery more intriguing by adding a personal layer, i.e., while viewing a page about Italy, receiving notification that a Facebook friend just added photos from Italy. True Knowledge is Freebase meets Cyc with natural language search thrown in for good measure.

As I was writing this post, a couple of good articles popped up that are worth reading. ReadWriteWeb took a broad look at trends it believes will shake up search in the near future. And Epicenter’s Betsy Schiffman had an interesting take on Google’s dominance of the market. I have to respectfully disagree with Chris Sherman of Search Engine Land, who said in Schiffman’s piece:

“I certainly think the game’s over in terms of building a world-class search engine. It’s too expensive — Google and others are literally spending hundreds of millions of dollars, in terms of research, development and infrastructure. Start-ups don’t have access to those sorts of resources.”

I see his point about cost but think we’re far from declaring game over. Though may not appear any time soon, the smaller engines are working other avenues to build adoption and user base. The game has changed from pouring millions into a destination site to instead channeling resources into multiple iterations of a product to gain traction. Powerset, for example, is testing a new Facebook app that finally brings capable search into the social network, in addition to its just-announced iPhone app, which generated 46% of its total traffic yesterday.

Sherman is right in one aspect: with such a complex problem and so many angles from which to attack, going it alone as a destination engine makes for a tougher, longer and more expensive fight. Smart companies are now shifting focus, integrating their algorithms into existing communities that are sorely in need of sophisticated search. Look for this integration and coalescing to continue on multiple levels in the next year. It will no longer be about killing Google but about making search, discovery and recommenation a more seamless and intuitive part of our existing online lives.

Zemanta Pixie


  1. Awesome piece. Seriously. I’m passing this around!

  2. I’d like to see search engines operate more like e-commerce in terms of sorting and filtering results. Engines like Google assume the first few results will satisfy users. Most electronic commerce sites don’t make the same assumption. If you’re searching for shoes to buy, you’re able to sort and filter information about price, size, style, color, manufacturer, and assorted other parameters.

    I heard Alta Vista inventor Louise Monier, now with Cuill, complaining recently that none of the new search approaches – blended, social, semantic, or vertical – get to the heart what people need – access to everything on the Web (not just the most popular pages) and an easy way to cull the relevant from the irrelevant.

    Which new search engines (or those in development) do queries, analyze sources, organize by topics, and filter results by user-designated parameters much like a research assistant would?

  3. let me correct – that’s Louis Monier who invented Alta Vista not Louise.

  4. johnerik said

    I now have 8 tabs sitting open to the right of this one and i’m emailing this to the semantic web austin folks (the cyc folks in there will be excited).

    great piece.

  5. Peter said

    I completely agree. Semantic technologies are best suited, in the present consumer market, to being passive helpers, quietly working to feed better information to the user. To this point, I think that the acquisition of Powerset by Microsoft could be a large step in that direction.

  6. […] After years toiling in the uber-geek trenches, semantics just got uber-interesting. As I wrote late last week, the days of – an ultimate destination search site to rule them all – are behind […]

RSS feed for comments on this post · TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: