As argued in“Online news: bloggers v newspapers”, we are entering a new era where the bundle of stories represented by current magazines and newspapers is being torn apart and being replaced by smaller publications and individual journalists/ writers blogging out there on the web.

With the aid of RSS feeds & cross-platform e-book software like Calibre (the equivalent of iTunes podcast subscriptions applied to blogs), and the e-reader of your choice like the Kindle DX or iPad,  you can now read the best blogs as a ‘personalized newspaper’, sitting feet up in the comfort of your own couch. Yep, the age of the personalized newspaper is here now, you no longer have to wait around for it to be lobbed onto your doorstep by the paperboy.

And with 100m+ odd blogs out there,  by anyone from your favourite novelist to the zany mommy-blogger types, surely there are bound to be some superb ones?

Well here comes the difficult bit.  How do you find them?

Does looking for the best blogs make sense in any case?

Arguably it’s a stupid task to begin with, because isn’t the whole point of the web the fragmentation of audience and content, letting a thousand content flowers bloom? You no longer have to write for ‘Mr Average Interest’ because there will be someone in Guatemala who even finds the drivelry on Drivelry interesting (no offence to Guatemalans). 

Surely it’s a bit like asking ‘how do you find the best music to listen to’?

Well the unlikely you can do today -  the impossible takes a little longer. It is a reasonable question to ask where to find the best blogs. We all expect that art gallery curators can put together the best art in a particular period and it also seems reasonable that you should be able to find say, ‘the best economics blogs‘ or more narrowly even the best blogs on New York theater.

Perhaps there are even weirdos out there who would like to read a smattering of  the best economics blogs out there, coupled with best cartoons, and the best blogs on the implosion taking place in the publishing industry (a dangerously close description of my own interests).

General characteristics of the best blogs for your own personalized newspaper

Even though it may make sense to start your search (more later) in your particular subject of interest, there are a number of generic characteristics that you will need in your blogs:

1. The blog will need to have full RSS (Really Simple Syndication) feeds.  People who set up their RSS feeds to only provide the first para are not going to work for your personalized newspaper (even though it might work better for their advertising revenue). You would end up on the train with your Kindle newspaper consisting of first paras. Kinda annoying huh? When investigating a new blog go straight to the RSS feed. If it’s not a full feed, move on.
2. Blogs can’t be links to ’the best posts this week’ from other bloggers in that subject. Why bother? Again you’re after something that you can read on the run, in a plane, on the farm etc.  Sometimes it may well make sense to explore their idea of what blogs are best in a particular space - but you’ll probably end up adding that blog to your newspaper instead.
3. Blogs can’t be video blogs.  Apart from reservations that you may have anyway about  whether video suits you as an information consumer it’s simply not supported in many e-readers. Yes, it is in the iPad (albeit not Flash-based video) but that leads to a whole other discussion about whether you are using your e-reader to read (either skim reading or in-depth) or whether you are after a portable computing device. Video isn’t supported by the Kindle.
4. The blog needs to be regularly updated. One of the things you will find in your own wanderings of ‘best of blog sites’ (see below) is that half the blogs that are recommended to you were last posted to in 1996. Few bloggers heed the warning to write about what they like and you’ve got to really like your subject to keep going. For your own news service you won’t care if the blog is only published once a month (with RSS subscription software for your e-book like Calibre you can just set a retrieval interval of once a month) but once a year and you’re probably wasting your time even setting up a feed.

Best of blog directories and searches

A friend of mine who edits literary anthologies is hugely dismissive of search engines and considers that editors will always be essential and that you’ll always need an intermediary to curate information for you, “Google is useless“, she says, “it just returns so much crap!”

I have explained to her that Google partially makes decisions about what is important by looking at hypertext links created by humans to particular pieces of content (‘PageRank’) but in finding the best blogs there is an obvious flaw to Google in that ‘most popular’ – as expressed by the number of links to that content – is not necessarily ‘best’ (or we would be spending most of our time reading up on swine flu). 

Even where there is a more formal voting process for best blogs it suffers from the same issue of being driven by lowest common denominator numbers, and well known meta-blogging sites like Technorati also reflect their subject specificity (Technorati is great if you want to find out about technical blogs but not so good otherwise in my experience).

Sadly, the human-curated websites on great blogs like ‘Best of the Web’ (botw.org) or Eaton Web (eatonweb.com) seem to have a lot of out-of-date content and attempts to use these to browse interesting sites were not very successful (these days they look to me more like ‘pay to play’ linkfarms if you were going to be overly reductive).  Blogcatalog (blogcatalog.com) also seems not to live up to its home page promise “we’ve done the heavy lifting for you. Browse blogs by category to find the best in class”.

Software recommendation engines to find blogs you like

What seems logical is a recommendation engine that matches your personal subject area interests with that of others who have the same interest and which only returns recommendations to pages that are current. Two significant players in this space are Digg who introduced a recommendation engine in 2008 and an older company in this space, Stumbleupon.

A key point to keep in mind is that recommendation engines involve a hefty time investment to produce good results (you’ve got to spend time telling them what you like).  Some commentators discount Digg on the basis that their recommendation engine tends to reinforce suggestions to pages and sites that your friends like rather than finding you ‘hidden gems’  so I focused on building up a Stumbleupon profile (which you do by giving a thumbs up or thumbs down to the pages they suggest).

Results have been mixed from Stumbleupon - after what I’d estimate at 5 hours of solid recommendations (much of what you will see is collegehumour.com style graphic humour) I’ve only selected 44 favourites from the pages I’ve been shown and I estimate I’ve found about half a dozen blogs that I like (yeah it’s entirely possible I’m just hyper-critical). 

For the moment I’m keeping going with Stumbleupon, to some extent out of sympathy for what is a great concept but a terrible name – stumbling is not exactly the most useful sounding activity is it?

Conclusion: it’s just plain hard to find a good recommendation source for blogs

With the emergence of e-readers in large numbers it seems to me that there is an opportunity here for someone to produce a better human-curated selection of great blogs than some of the moribund websites mentioned above like BOTW and Eaton, and possibly there is such a site but I was not easily able to find it (let me know in the comments). 

Producing a good recommendation software engine by contrast requires a great algorithm and a lot of users (Stumbleupon after several years has 10 million+ users) but when you consider that many of them would be inactive and therefore generating little recommendation data, and that they’ve got several hundred subject categories, Stumbleupon’s source of preference data looks a lot poorer (perhaps my literary editor friend was right after all).

The best source of good blogs I found in the end was via Tweets from people I follow on Twitter, so if you can get access to Twitter’s data that looks like a great source. Whilst it appears that Facebook leads sharing of content, Twitter seems to be the only ‘open’ source of information (i.e. you don’t have to be accepted as someone’s friend to see what they recommend).

Otherwise there must be someone in an organisation out there, perhaps at Google or an ISP, who has a lot of blog content preference information.

C’mon venture capitalists! With the explosion in electronic books, creating a good blog recommendation engine is a non-trivial (i.e. barriers to entry) and potentially lucrative problem to solve.

Posted under Unanswerable questions, Wait for release 2.0

The next information wave

It’s easy to hate the most over-used phrases used to describe the web age – paradigm shift, knowledge revolution, Web 2.0 etc.  

But hey we’ve all faced the tyranny of an empty page – it’s tough out there. And faced with amazing changes to how we locate information represented by Google and the other search engines it’s easy to reach for the bottle of superlatives.  

Even the growth of email has meant a phenomenal change to the way we consume and create information in the last 15 years as documents and comments flow back and forth in minutes that used to take faxes, printers, and letters hours or days.  

Despite all these examples of card-carrying “gee whiz”  moments however it looks like we are just about to be swept up in another wave.  

We are just starting to consume knowledge in another way which may result in an equally dramatic lifestyle change when compared to the growth of search engines, or the early days of the web itself.  

The on-rushing wave is the e-book (or more properly the ‘E-Reader’), with an estimated 10 million being sold by the end of this year just in the US alone and dozens of different E-Readers in the works from different companies emerging in the next 12 months – with a combined marketing budget in the hundreds of millions that will make the recent Apple iPad launch look like a hiccup.  

On the surface of it is this just another example of hyperbolic silliness? Because isn’t the E-Reader just another way to consume books that we’re already reading in hardcopy anyway, and as many have pointed out there are a lot of advantages of traditional hardcopy over the electronic alternative?  

What is interesting about E-Readers is not the ‘E-book’ part at all. Instead E-Readers are going to very rapidly reshape how we consume information on the web. This is not about a ‘new device’ it’s about a new way to consume information, because as is somewhat obvious most of the information on the web is just not available in hardcopy form at all. And that really matters.  

Backlit screen reading is hard to do (sorry Apple I know you’re trying with the iPad). It’s hard on your eyes,  it’s not very portable (try reading a laptop with your feet up), and in this information-packed age unless you’re looking at a huge screen it is hard to skim read and quickly identify the key points in screen-based information.  

This has lead to an emphasis on other aspects of screen-based information that compensate for these drawbacks of the web on a computer screen by adding new functionality: dense hypertext links to make cross referencing easy, the ability to comment and interact with authors as represented by blogs, the ability to locate and reassemble information in many different ways as represented by search engines, and in most journalism courses a recognition that writing for the web requires a specific style (admonitions to not write more than two sentence paras etc).  

But the E-Reader is changing that (read “Customized newspapers on e-books: Calibre cross-platform e-book subscription management” to see how you can subscribe to websites out there on the web on your E-Reader, via their RSS feeds,  just like you can subscribe to a podcast on iTunes).  

What happens when everything from the web is as portable and easy to read as paper?  

It’s not JUST the E-Reader: the way this is going to turn our information-consuming lives upside down is driven by a groundswell of the growth of machine-readable info feeds on sites represented by Really Simple Syndication, and emerging platforms like Calibre which enable you ’subscribe’ to any website on any E-Reader – for FREE.  

The E-book just happens to be the final crest of the wave which is about to sweep away already undermined sections of the publishing industry.  

Still not convinced?  

The only thing worse than a ‘paradigm’ shift is an attempt to predict the future so here goes with a few near-term predictions:  

1. the world of journalism is about to get vastly more competitive – journalists in a newspaper will be competing with bloggers with more specialist knowledge who previously had limited circulation because you couldn’t purchase your average blog conveniently on a newstand or read it on the way to work on the train. The blogger is providing the information for free.
2. intermediaries that seek out big stories will get more important in their own verticals (wherever those stories are) whether they are using algorithms like Google or whether they are using human curated approaches like ‘The Week’, or social media mechanisms like Stumbleupon or Digg. Try and find for example, amongst the 100m+ or so out there which are the most entertaining blogs? To some extent this question was irrelevant  if those blogs didn’t get read – but not any more.
3. new ways of consuming stories will arise – for instance ‘collaborative reading’. On my Kindle or Sony Reader or iPad or whatever why should I not be able to view annotations on a text by my colleagues, or classmates, or friends?
4. we’re not talking individual wikis – we’re talking wiki world – why buy a ‘How To’ manual for  your motorcycle when there is a completely portable version?
5. traditional publishers who are congratulating themselves on being web savvy like the New York Times or Guardian are going to come under even more pressure because E-Readers in many cases will have the effect of destroying ad revenue and their ability to really track readership. With the aid of software on an E-Reader like Calibre this is the equivalent of Tivo for the publishing world: in your Kindle DX via Calibre no javascript is active (so Google Analytics aint going to know you’ve read the story) all it knows is that someone downloaded their RSS feed (not which particular stories were read).  And fancy ads are not going to come across because people writing recipes to extract your content with Calibre simply don’t want them.  Even if you instituted counter-measures by embedding an ad image joined with a story image, it’s almost the equivalent of going back to the flat ad serving done in newspapers. 
6. even the acknowledged guru of the web commerce world, Amazon, may be shooting themselves in the foot. They want you to buy e-books from them – but why? If you can read any old PDF or niche website out there on the web from the comfort of the couch? Or a million free book titles? I bought an Amazon Kindle DX and have yet to buy a book on it (but thanks Amazon anyway for paying the wireless subscription charges).
7. if it isn’t already obvious the E-reader is going to reduce further the cost of information globally even more: expect to add value with your startup company by better locating information, or better curation, but expect an uphill road if you think it’s about unique content (Mr Murdoch take note).
8. a bit like podcasting has broken the barriers to entry for local radio talk shows your decision is not ‘which paper do I buy in my home town’, instead it’s ‘which blogs do I want to read from the entire internet?’ Publications are becoming ‘unbundled’ and publishing deadlines paradoxically may actually become less important in an age of instant satisfaction. Why should I read some crap which some journalist stitched together at the last moment to make the Sun weekend edition when I can read a combination of bloggers who only produce pearls once a month? 

Like the sound of this new world or hate it?  

Is it not going to happen at all?  This is a blog – talk back.

Posted under Kewl

My household typically no longer buys newspapers.  Actually we rarely watch the television news either.

We get most ‘news’ via a couple of magazine subscriptions, podcasts (generally consumed in the car or whilst doing the washing up) and to a less extent online.

It seems we are not alone. The newspaper sector, for example in the UK and USA, is deeply unloved at the moment by investors, with ad revenue expected to fall in the USA by 16% or so over the next 2 years. In the UK the largest news publishing group, Trinity Mirror, has seen an 80% fall in its stock price over the last 2 years.

Companies like Trinity are therefore busily trumpeting the growth of their online properties to investors out there, the line being, “our readership and ad revenue is falling but hey look at the growth of this online revenue…”

Like the much awaited demise of cinema when television came along I’m not sure that newspapers will disappear (is everyone going to obtain their news from Adsense-funded individual bloggers out there? – I doubt it). However what would really help out these guys like Trinity is if they didn’t have to spend 40% of their revenue printing and distributing a big hunk of forest every day.

But let’s face it, the computer screen sucks for casual reading. An odd thing to say really when this is a blog (Windows users hit Alt+F4 now).

I have to admit that the blogosphere hasn’t been high on my reading list either.

The likely way I would consume blogs would be by RSS feed but I don’t even have a dedicated RSS reader (and haven’t explored the Windows IE RSS functionality) although I have been vaguely thinking this is where I would like to go:  getting ever closer to the ‘my newspaper’ ideas bruited about when I was working on websites in the heady days of the millenium. A large commercial website aimed at the business sector I’m involved with gets about 3.5% of page views via RSS feed: so RSS aint there yet (RSS is only double the number of article print requests we can see on the same site).  

So my suspicion is that people who currently read blogs generally write blogs, and they’re reading blogs primarily because blog writers they link to often link back, mutually feeding their blog’s respective Google PageRank scores.

What would change all of this however is that if a decent electronic book came along. And finally, it looks like within 3-5 years that is going to happen.

Amazon's Kindle reading device

Amazon’s Nov ‘07 launch of the Kindle has supposedly sold about 350,000 units, which is quite a lot for a device less than a year old, which has ongoing subscription charges to boot. Then there’s Sony’s Reader which only has USB connectivity (presumably that means forget subscribing to a blog RSS feed) and then there is the wifi-equipped Phillips iRex.  This aint like reading text on your PDA or computer - these ‘E Ink’ screens provide very clear typeface viewable in outside light conditions from multiple angles.

Sony's version of the electronic book

However despite the availability and initially encouraging sales it looks like they’ve not learnt from the music industry and digital rights management is rampant, so unlike a paperback you can’t give a copy of your Sony ebook to a friend. And on the Kindle you have to subscribe to Amazon’s mobile data service in order to subscribe to this blog…

What I’m waiting to buy (and possibly a few other people) is an electronic reader that works offline for normal books and with wifi for online RSS subscriptions, with no digital rights management. That’s the device I’m going to be sitting in a (wifi equipped) coffee shop reading.

And at that stage reading RSS-’my newspaper’ blogs may be an increasingly mainstream activity and a larger number of blogs will make sense from an ad revenue perspective.

Posted under Wait for release 2.0