The consensus is that it’s time for a new kind of search, one that finally brings the internet in-line with the sort of ‘breaking news’ we might see on TV, or that ubiquitous, useless ticker at the most of most 24-hour news channels.

Writing in Seattle’s The Stranger, Paul Constant recently explained why real-time is important:

The thing most people will probably, eventually use Twitter for is its clean and efficient search engine.

The search function on Twitter is an amazing thing: It’s a focused laser beam into what people are thinking about right now.

And it’s permanently set to “now”: It doesn’t have anything to do with the past or with archiving. That is an innovation in and of itself.

An inordinate amount of the internet is devoted to archiving and filing all the world that existed before the internet.

Twitter is reflexive, instinct driven, present. It doesn’t care about the past. It’s hard to find a post older than a week old. It’s work.

Twitter’s own search has been covered to death online, and the consensus is that it needs competition and improvement. As a result, an almost comical number of competitors have stepped up to sort and catalogue what’s happening in real-time on the internet right now.

Below, we look at 5 of the most interesting ones and let you know how they work, what they do for you, and whether you’ll want to use them every day.

Friendfeed

Friendfeed was the first search I tried. To me, it ended up looking more like a blog search—most of the stories were people posting major newspaper articles pulled by Google News, or from the New York Times directly, or from other major international sources like Spain’s El Mundo. I saw a few results pulled from delicious.com and also some posts by the New York Times’ own friendfeed stream as well. The real-time updating was nicely done, but for the subject I chose, there simply isn’t enough at-the-moment information coming out to make a search like this worthwhile.

After about 30 seconds of waiting for someone else to make a comment, tweet, or social-bookmark another article, I moved on. As it stands, I could get a nice idea of how a story is developing through conversation online, but a quick Google News search would get me more relevant information a whole lot faster, considering almost everything in the feed is just links to old-fashioned journalism anyway. Perhaps with a still-rapidly-developing story things might be different, but again, a huge amount of conversation (even about ultra-fresh breaking news) comes through as links to newspaper websites.

The Verdict:

A nice but very simple interface that pulls from a wide range of sources (tumblr, twitter, delicious, digg, facebook, youtube) to get its information. Whether that information is any good is another story. If you use it just to search your own friends’ streams (and have a lot of friends online) it might be great—otherwise it’s search away and hope for the best.

Scoopler

I liked Scoopler’s approach, as it shows real-time results updating in the center column, while displaying various ‘popular’ results (sorted into videos, links, and images) on the right. I tried following a few other ‘hot topics’ as suggested on the left, and most of the twitter results that showed up were near-unreadale, full of 6 or 7 #hashtags each and several links to spam sites.

The spam isn’t surprising—any new communication format online gets abused by spam, but usually we’re able to block it out, either by not following spammers on twitter, or relying on Google’s algorithms to block it for us. Scoopler, although I’m sure they’re trying valiantly to do so, hasn’t done it yet, although they are a brand new search engine.

The Verdict:

An OK mix of the real-time flow and a filtered, more popular resulsts list, but ultimately not very satisfying in helping me find anything. If you like studying the online zeitgeist, this is better than Google Trends, in most respects.

Topsy

Topsy is actually quite fantastic. Want to see how many people are talking about a particular article or several articles on a specific subject? Topsy does it beautifully, because it ranks the results using some specific algorithms. Certain tweeters get an ‘influential’ rating, based on a series of criteria that surely includes the number of followers, the amount of re-tweets they garner, and so on. So as something new and trending pops up online, you can get an immediate sense of ‘authoritative’ search that some of the other real-time engines lack. I’d like to know more about how Topsy evaluates its ‘influential’ criteria, but they’ve definitely hit on a good idea.

The other thing I’m enjoying about their site is the following: it gives me the satisfaction of google’s indexed, archived, older searches, only applied to Twitter. I can go through Topsy and immediately find my own profile, and then get a sense of how many other people tweeted the same links I did. Authors can also use this to find articles and blog posts of theirs that other people have been talking about—while Twitter’s main search lets you do this too, Topsy makes it more elegant and far better organized.

In the end, when it came to my breaking news search (Xinjiang), Topsy was much like the others, giving me general links to big newspaper stories with lots of authority, and little else. Perhaps I picked one that didn’t have much original content outside of news agencies—it was coming from China, after all, where there’s still plenty of censorship. Breaking stories elsewhere have a much different feel to them—faster links to amateur video on youtube, and so on.

The Verdict:

Out of all of these, Topsy is the one search engine I can see myself going back to, and using it to supplement a google search. I’ve already used it about five times today, which is the only thing that matters, in the end.

Collecta

Collecta is another startup (all of these are startups, of course) searching twitter, this time with an interface that lets you put into larger focus individual tweets, and lets you run several searches concurrently.

I liked both of those features, and the fact that it pulls comments from blogs, as well as blog posts and flickr photos. There was no smooth interface action where the newest tweets cause the old ones to elegantly slide down (something that FriendFeed does, which gives it a more ‘real time’ feeling, at least aesthetically), and there was still, as is inevitable, a ton of spam when it came to any ‘hot topic’.

Also, leaving the site gave me a message about disconnection, and going back to the site left me with a big wait before all my searches started again.

The Verdict:

Again, a nice interface, but nothing I’d use on a daily basis. I’m starting to notice a trend here—slight variations on the main twitter search engine—and I think a large part of this is because all of these search engines are just so damn new.

Almost.at

Here we’ve got a site that isn’t really a search engine, as much as a way to follow a series of ongoing events through real-time chatter online. You can choose major events from the left toolbar, and you get a series of twitter updates, flickr photos and youtube videos, and various links displayed.

What’s nice is that the timing is broken down minute-by-minute, so you get a nice clean view of the traffic for a specific event. You can also head back in the timeline to see what the chatter was at a previous time. Once I did this, though, I found it tough to get back to the ‘present moment’ and started forgetting today’s date.

The timestamping for the youtube and flick results also seemed to be non-existent.

The Verdict:

A cool idea for following live events, and the actual event choices (at least during the beta) are moderated, so the selection is small and relevant. The execution isn’t anywhere close to perfect, but the concept is one of the most interesting. Following protests using only #hashtags or twitter’s main search can become a spam-fest or a messy confusion, and almost.at’s concept of choosing specific ‘events’ and following just those is a nice way of organizing the information.

To Sum Up….

Pretty much all of these services are in beta, and they all show it. In the rush to capitalize on the new flood of twitter data that’s out there, it’s inevitable there are going to be a ton of real-time search engines popping up this year, and if the rumours are any guide, Google will surely launch their own effort soon enough.

Lots of chatter about real-time search tends to paint Google as an old-fashioned search engine relying on caches and ‘yesterday’s internet’. Be weary about such claims—determining relevance when it comes to search results takes time, and for now, the best algorithms are ones that can’t get the job done instantaneously.

And What About Google?

Another cliche in the chatter surrouding these sites is Google’s lack of a real-time offering as of yet, and the fact that they’ve admitted they’re somewhat ‘behind’ on the issue. While it is surprising that so many competitors have moved into the field while Google has released nothing so far (but goes ahead and announces stuff like the Chrome OS months and months in advance, without any previews or substantial details), it’s all just a matter of tracking and indexing data, something we all know Google is very good at.

And most of the data being analyzed is simply coming from Twitter streams, combined with reading the timestamp of various other postings. None of this is exceedingly difficult when it comes to search, so what’s really going to matter is getting relevant real-time results, blocking spam, and creating an interface that’s easy to use but robust underneath.

So far, a site like that doesn’t exist (although I’d vote for topsy as the best one so far). Might Google jump into the market with the best implementation of real-time search yet seen? Although the field seems insanely crowded already, even just a cursory glance like we’ve done here shows that there’s massive room for improvement. Real-time search is still in its ‘alpha’ phase, and needs a big player to pull it out. One is probably going to arrive any day now.

Obviously these questions matter for anyone trying to make money using different forms of content distribution, whether it be advertising in a local newspaper or taking out new ad space on YouTube. But what matters even more is how these new forms will actually affect the content itself. Let’s look at a few recent ideas and see what they can tell us.

Marshall McLuhan Is Still Pretty Damn Important.

Now, if you remember nothing else from this pioneering Canadian media theorist, just take his most famous aphorism: “the medium is the message“. It’s since been tweaked and refuted and played with so many times as to lose some of its original meaning, but in general, you can take it to mean form affects (or is) content.

Look at the Kindle, for example. The world’s biggest online bookseller has now entered the hardware game, creating a device that is designed to replace the very book form itself. And of course Amazon, being a huge seller of books, is hoping to control and monetize as much of the distribution method as possible. It’s very much along the lines of Apple’s iTunes store: create a device that a lot of people buy, then offer an elegant solution to sell content for that device.

Why Comparisons With the Music Industry Aren’t Always Perfect.

But comparisons with the music industry don’t work after a certain point: formats have changed quite frequently in the last several years, from vinyl->tape->CD->digital. Books have been the same for hundreds of years. So any time we get over-excited about the death of print, we need to step back and look at the longer view, like Dave Eggers suggests:

Eggers has a point. But there are some very convincing arguments to the contrary, and as Apple has proven, if a device good and useable enough comes out, enough people will buy one that it can change an entire industry in the course of a few years. But we do need to remember: completely ignoring print and assuming that a declining readership means a non-existent readership just isn’t a good idea.

A Microcosm For Us All: Intellectuals and Published Content.

A recent argument has also been floating around about Masters and PhD dissertations, and how the cost of publishing them is nearly always prohibitive. The argument usually continues that the ‘book’ form for academic argument is not always ideal; it has created the expectation that if a professor has something good to say on a subject, it must come in a minimum of 220 pages. While this might seem like a small corner of the publishing market, it’s actually a perfect microcosm of some bigger changes we might see in the future.

A long, read-by-no one dissertation is an antiquated notion. Don’t get me wrong—I’m sure it will continue for some time, but any progressive university is probably realizing that the whole ‘university publishing’ business will have to change radically over the next several years.

The Delivery Method Starts Changing the Stuff Inside.

Here is where the dozens of options available for content publishing (the form, or medium) will begin to change the content itself. Once the necessary existential debates on the role of the academy and the importance of intellectual discourse are hashed out, we are going to be looking at a very different landscape, one where serious academic thought is not walled up in quarterly journals but actively distributed across a whole range of channels.

Many of these channels will not be conducive to a 220-page dissertation. Some will support, say, a 15-page essay, others might be wonderful for a short book (say 100 pages) on a particular subject, others only a brief article. But all of these options, and the various ways in which content creators, editors, and distributors will try and monetize this process is definitely going to change the content itself.

And Just Why Does This Matter to Us?

Simple: if you get too focused on creating quality content along a single model, the speed of innovation might take you for a loop. Things don’t change so fast that all your hard work will disappear in a moment, but if you are trying to improve your website’s SEO, build a brand online, or market your product through any kind of content creation, you need to pay attention to how the mediums are changing.

Just because Twitter has become insanely popular in the last 3 years doesn’t mean you need to scrap your print advertising budget and start tweeting incessantly. That would be quite useless. But it does mean you need to be conscious when something like a Twitter pops up, and suddenly a new 140-character format for content exists. Remember not to get too comfortable and you’ll stay on the right track.

(photos by flickr users peagreengirl and cogdogblog. Used under a creative commons license.)

1. Content, Keywords and Links. As always ensure that your content, keywords and links are unique, of good quality and relevant to your business. Your back links should also be relevant so be sure to look out for and reject illogical links. Use absolute links which are less likely to have issues, but also allow you to get more backlink love if your content gets scraped. Remember that when it comes to keywords there is such thing as “too many” so don’t overdo it.

2. Make your links and keywords easily accessible to spiders. Drop down menus make the website user friendly, but they aren’t spider friendly. Somewhere on the page you should be placing good old-fashioned text links so spiders can find them. Also, be aware that SPIDERS CAN NOT CRAWL IMAGES OR FLASH. Be sure to label everything with text and include text descriptions so that spiders can find it more easily. Use minimal AJAX and Flash, and you shouldn’t be using frames at all.

3. Viral Videos and Pics. Enable the “Enhanced Image Search” option and list a video sitemap in your Google Webmaster Central account. Also, get your videos played on viral sites like YouTube, AOL, MSN, Yahoo, Metacafe and MySpace. All these places are crawled by Google. When posting captions and descriptions for pics and videos use words like “image” “pics” or “video” since many searches for pictures or videos include obvious words like that.

<img

4. Unique Title Tags. Make sure that each title tag on your website is different and has a keyword in it. Don’t put your company name in there unless you are so big and popular that people ask for you by name. In other words, unless you’re Wal-Mart, Gucci, or Vera Wang, you should put your name somewhere else on the page… like at the end.

5. Interior linkage. When linking pages within your website, don’t simply hyperlink the words “click here”. Use keywords instead. So if you’re selling diamonds then hyperlink the phrase “Buy Wholesale Diamonds” or “Wholesale Diamond Information”. Also, if you have “index” associated with your home page (ie: DelawareDiamonds.com/index.html) then be sure to arrange it so you’re not splitting your links. You don’t need index.html, default.php or any of those. Your URL should just be your URL with a plain-Jane dotcom (net, org, edu or whatever) ending. When linking internally though, spiders and search engines don’t pay attention to your URL file extension.

6. Location, Location, Location. If I’ve said it once, I’ve said it a thousand times, be aware of local search keywords. Don’t be afraid to put your location (i.e. Florida, New York, Presque Island,) into your keyword phrases. “Delaware Diamonds,” “New York Fashion” or even just “Presque Island Store” instead of “our store” helps people find you better. This is especially important if you count on your customers finding you online, and buying your products/services in-store.

7. Make sure to buy up any similar URLs to yours and have them redirect to your website. This is not the same thing as creating doorways, gates or mirror sites. What this does mean is that if your original URL is http://www.happybusiness.com then you could buy up http://happybusiness.com, happybusiness.net, happybusiness.org, and have them all automatically and instantly redirect to your original site. This typically works well for website URLs which are commonly misspelled by searchers. A good example is Barnes and Noble: you can enter www.barnesandnoble.com into your address bar and you’ll automatically be redirected to www.bn.com. Just a note though before you go buying up URLs; Don’t buy up links that haven’t been updated or used in over a month. The best way to tell is to do a Google search for “cache” plus whatever website your looking for. i.e.: “cache:www.whateverwebsite.com”

8. Sticky Forwarding. If you’re completely renovating your website and changing domain names then be sure to use “sticky forwarding.” This will allow users to get redirected from your old site to your new one and help them transition through the change better.

9. Check Yourself. Google up some server header checkers (you can actually do a search for “Check Server Header”) to get a tool that will check your redirected websites. You should be popping up with a report that says: ‘301 moved permanently” or “200 OK” if they are set up and used the right way. If not, either fix them or get someone to do it for you.

10. Guilt By Association. Make sure to run occasional blacklist checks if you are running on a shared server to make sure that you’re not sharing space with any banned or notoriously shady websites. This could actually affect your ratings with search engines. It also helps to make sure that your domain ownership information is visible by search engines.

11. Sponsorship. Philanthropy is actually rewarded by most search engines. Find non-profit organizations on the web, see if they are looking for sponsors and reap the benefits of those back links.

12. RSS Optimization. It is what it sounds like. You need to be optimizing your RSS feed the same way you would any of your web pages.

13. Beneficial Blogging. Your title tag and blog title are two different things so make sure you’re optimizing both separately. Use a “call to action” style of blogging where you provoke people to respond or react to what you’re blogging about. Also see if you can get someone of influence to post something special on you blog. Build up to it ahead of time: i.e. “Coffee Guru, Joe Schmoe will be posting with us on October 10th!” You can also ask the person of influence to post comments and opinions on blogs that you’ve written.

14. Social Networking is SEO. You can’t do SEO nowadays without incorporating social networking into the mix. Websites like Digg, Facebook, Twitter and Del.icio.us have become SEO essentials. Also, adding components like blogs, podcasts, social content, reviews, sharing apps, user ratings and comments help to boost your website’s “viral appeal.” It does take extra time and effort to do this right and not spam every blog you come across so make sure that you or whoever you hire knows what they’re doing so your website doesn’t get penalized.

15. Quality not Quantity. I know I talk a lot about page rank, but it’s actually not so important that you’re #1. There are plenty of websites that outrank the #1 spotters in hits, quality, sales and time spent on the site simply because the lesser-ranked website has better and more relevant content. Another important thing to do is keep adding to or refreshing the content that you have to legitimize your website to search engines. You should be adding new content or changing content about 3-5 times a week to keep spiders happy.