A Unified Theory of Search, Social Networking, Structured Blogging, RSS and the Active Web

The web has traditionally offered a very passive experience: If you want something, you have to go get it.  Sure, there are lots of interesting information and useful services out there, it’s just up to you to figure out where they are and how to use them.

RSS is exciting because it is the first widely accepted (and increasingly deployed) standard for transforming the web into more an active entity.  With RSS, you can now “listen” to the web and automatically receive updates without having to go looking for them.  But RSS is primarily a demand-side innovation.  It benefits consumers of information/services but not suppliers.

The real innovations yet to emerge are coming on the supply side of web.  And they are coming into being primarily as a result of the confluence of three important trends: social networking, search, structured blogging.

Much More Than Dating
I must admit, while I have written about social networking in the past, I really didn’t understand its potential significance until a few months ago.   I am not talking about the 1st generation “Do you know who I know?” sites such as Friendster and Orkut, but about the second generation “hang out” spots such as MySpace and Facebook.  Up until recently I had pretty much pigeon-holed such sites as just 2nd generation online dating sites.

While this is probably a valid short term characterization, these sites are likely to evolve over time into something more substantial.  Specifically, these sites (as well as sites such as MSN’s Spaces and Yahoo’s 360) are highly likely to evolve into an all encompassing digital identify for each user.   People will use these sites not just as a way to socialize, but as a hub to manage their broader digital identify on the web.  Part blog, part digital cubby hole, these sites will provide each human being with a digital homestead from which they can manage their entire digital identify on the web.

In the early 1990’s, if you had an e-mail on your business card you were regarded as a either a nerd or a curiosity, yet 10 years later almost everyone in the developed world had an e-mail and in many cases used it as their primary means of asynchronous communication.  Today people with their own websites are regarded as either curious or vain, yet I feel quite confident saying that 10 years from now everyone in the developed world will have their own website and the few that don’t will likely be regarded as hopeless Luddites much like those without e-mails are today.

Supply Side Economics
With every person in the world managing their own digital homestead, the supply side of the web starts to look very, very different especially when one adds in two key elements: structured blogging and search.  Structured blogging is a fairly recent movement which seeks to promote the use of standardized XML-based tags in blog posts so that search engines can readily recognize both the type of blog post and the individual elements within it.  For example, if a blogger is writing a product review, the structured blogging tags would enable a search engine to easily identify that particular post as a product review and even to identify the specific product.  Essentially structured blogging would allow search engines to take the guess work out of vertical search.  It would also allow search engines to easily incorporate blog posts into their database platforms (such as Google Base).   It is only a matter of time before structured blogging (or some variant of it) is embraced by not just every blogging platform, but by every social networking site.  What this will result in is a huge amount of index-ready supply side information.

This is where search comes in.  Using their spiders the search engines will now index and assimilate into their newly minted databases all of this structured supply side content.  The results will then be available to the entire world within hours.

Your Own Global Megaphone
People’s behavior will change radically once they realize that they can publish all forms of structured content on their own websites that will, within hours or minutes, be indexed and made available to the entire world by search engines and structured databases.   For example, why go the trouble of posting an apartment for rent on Craig’s List or Rent.com when you can just publish a “Apartment Rental” post on your personal website and know that this listing will shortly be available to everyone in the entire world, not just via search engines but via the specialized listings sites that the search engines will create to display such content.  It’s like having the biggest megaphone in the world at your personal disposal for any and all of your requests or desires.

Through the combination of social networking’s personal websites, structured blogging, and search, the supply side will essentially become a “write once, distribute everywhere” proposition.  No need to wander around the web trying to figure out how best to reach other people, just publish once on your site and let the “Active Web” take the information and distribute it as widely as possible.

Of course, there may be some information that you are not wild about sharing with the world, such as a dating profile or a job resume, but that is an easily solved problem because users can simply specify which search engines are allowed access to their content.  It is not hard to imagine a personal website 5 or 10 years from now that has a different tab for each major type of structured content.  On each of these tabs will not only be the structured content itself, but also check boxes that allow users to prevent different search engines from indexing their content. So if there’s a particular database you’d rather not have your resume end up in, you can just exclude that search engine from accessing your resume “tab”.

The Advent of the "Active Web"
RSS and its extensions on the demand side, combined with structured personal (and business) websites and search on the supply side will transform the web from a passive entity into the Active Web and will comprise the first major step towards a more autonomous web that adds real value to people’s daily lives without requiring constant effort and input.