Curation and Open Source

The hot new word for 2011 is curation. (This particular curator is a video game monster.)

Curation has been around for years, but it has mostly been done by people. Boingboing  is mainly about curation. In its heyday Slashdot was a great curation site. The Huffington Post is mainly about curation, and its big-bucks acquisition by AOL is what got everyone's attention on the space.

The term originates in the art world, where the selection and presentation of good stuff (as opposed to just old stuff) is a vital economic function. In the digital world  the true meaning isn't really given. We're talking, in the end, of becoming the source, the place people go to in order to find out what's going on.

Google News is considered a curation site, but it's mainly an illustration of how far curation has to go, in a software sense. Google News divides stories into headings, some created by the site, some by the user. (I use the heading open source regularly.) Google News also defines what gets treated seriously. The New York Times is considered serious. This site is not.

But that's an arbitrary decision, and when The New York Times paywall comes down  we have a problem. Links to it are already disappearing, just as they've become rarer to the Murdoch papers in England that created paywalls.

Google has to do that because customers who run into links demanding payment don't just get mad at the paywall, but at Google for leading them to a dead end. And this is one reason why the Times' paywall is doomed.

The other reason is curation. There is no “news” that can't be found elsewhere, through curation. And the whole point of a column (or a blog post) is to be read. Curation is vital to having a voice online. (Shown is Weston Naef, former curator of photographs at the Getty Museum.) Curating can be a fun career.)

When someone like Mark Cuban calls curation theft  he's engaging in craziness. To put it in terms he'd understand, he's trading away Dirk Nowitzki for Greg Oden. To put it another way, Mark, if newspapers and TV don't cover your ball club, you can't get butts in the seats.

The difference between effective curation and search lies in finding and knowing the good stuff, about discriminating. As the Google example shows this is very difficult to teach a computer. A computer can only take a first cut at it.

It was the lack of discrimination, an effort to become more of a curator, that forced Google to send out new algorithms against content farms. Despite the collateral damage it's a worthwhile endeavor, because sorting the wheat from the chaff is what we're all doing.

Any tool which makes that easier simplifies things. It speeds the creation of knowledge from mere data, and speeds the delivery of that knowledge to people who can create more knowledge from it. It creates a virtuous cycle of learning that raises global productivity.

Finding good stuff is just one piece of curation. Showing it off is also important. That requires more than a user interface. It also requires context. Nearly all curation tools I've ever seen lack this secret ingredient. Human intervention remains necessary.

But all this is a moving target. Which is why I want to close this piece with a call to action. The open source tools I've found that call themselves curators  are all about information collection. They're not about discrimination, nor presentation. We need some good open source projects aimed at those needs.