The Open Source trend for 2011 is cloud integration

Each year brings a new trend to open source development. It's gone mobile, we've had the rise of Apache, and we've all gone to the mattresses with our lawyers.

Developers generally ignore these legal fights. They have better things to do. The law is mainly concerned with portioning off the value society creates. Open source is mainly concerned with creating new value, with rapidly turning innovation into common currency.

This year, it's already clear, the new value will lie in integration with the cloud.

NOTE: I covered open source for ZDNet for over 5 years and would like to get back in the game. Meanwhile I'm producing some essays for this site covering important topics.

Before I get into the story, a quick primer on what clouds are and why you should care.

The cloud is a tech shorthand for hiding complexity.

Once software is virtualized, once it's an instance on VMWare or Microsoft's Hyper-V,  it can run on any hardware, and once it runs on any hardware these instances can live anywhere on a server farm. This lets you simplify operations and lower the cost of server computing for everyone. Since applications and their operating systems are no longer tied to a particular server, a server farm becomes a cloud — it's all there, you just don't have to know the where or how of it.

Amazon was the first company to create a cloud-for-hire, called EC2. Once it gained these benefits for its own online shopping system, it kept growing its infrastructure and selling access to it. A lot of the industry's focus the last two years has been on following Google's entry into this space, because everyone knows Google has the lowest infrastructure costs in the business.

But the cloud was an even-bigger threat to Web hosts. Soon after the Web was spun, Internet Service Providers (ISPs) began specializing, and those which specialized in hosting Web sites did very well. (Those that specialized in core infrastructure did less well, those that went into consumer access were murdered in Washington by the Bells and cable companies.) But the rise of cloud computing meant industry giants like Amazon (or Google, or Microsoft, or Salesforce.com) could now bring their low-cost infrastructure to bear on Web hosts, crushing them as the Bells crushed the consumer operations.

Open source offered a way forward. By getting together around an open source framework, Web hosts could share the cost of building their own clouds, not just with one another, but with other companies developing open source frameworks (even Google, nominally the competitor they feared most).

Thus we have OpenStack, the open stack cloud computing program led by Rackspace, a Web hosting company based in San Antonio, Texas. (Notice how the rise of open source enables competition to come from anywhere.)

This has become a big story because the future of competition in the space depends on the success of open source cloud integration. Without it, smaller Web hosts disappear, the big boys take over, and we have the same problems of choke points we have in the consumer space.

While it's true that free market competition naturally leads toward monopoly, it is much better to deal with this problem through the marketplace, through a mechanism like open source, than it is through the government. The history of the Internet is mainly a story of working-around bottlenecks of all sorts, monopoly being  just one, and this is one big reason why covering the space for almost three decades has been such an honor for me.

The success of any open source project depends on everyone getting along, and cloud integration is no exception. OpenStack has made a good start, but their success (or failure) in organizing the work, keeping everyone on task, and delivering something that works will determine just how much competition we have as Web hosting becomes nothing more than a cloudy sky.

Some folks are already implementing aspects of OpenStack, including in my home town of Atlanta. Cool.

It will be a fun story to watch.