Small Pieces Tightly Joined: Open Source in the Cloud

It’s not a shock to state that cloud computing will disrupt the business model of commercial software. But how it will affect the open source movement?

The rise of open source is clearly linked to the rise of the web. Buy a commodity piece of hardware, download source code of any of the thousands of open source projects and start to “scratch your own itch”. My Linux box will communicate with your Linux box as long as we stick to some minimal set of protocols. The web is loosely coupled and software can be developed independently in a bazaar style.

It’s not quite as straightforward in the cloud. Clouds are also composed of thousands of commodity PCs, but the cloud operator manages the overall architecture and deployment – power supply, cooling, hypervisors, security, networks and so on. We don’t rely on minimal set of protocols in the cloud. On the contrary the cloud is defined by fairly complex, high level APIs. Even though the actual cloud OS may come from the open source domain, the tightly coupled nature of the cloud prevents users from modifying the cloud software.

There’s a lot of talk today about setting up private clouds with an Open Source Cloud OS, but the idea of private clouds is simply a delusion. Since the owner of private cloud has to purchase all required HW upfront, private clouds don’t provide the main benefit of cloud computing: elasticity. Other people will claim that clouds are not compatible with the open source movement or call it outright ‘stupidity’.

I see two possible solutions to this problem:

Benevolent dictator: Leading cloud providers (Amazon, Google, MSFT) will open-source their complete stack. This means that they would let the community to inspect the code, fix bugs, suggest improvements and define a clear roadmap similar to the Linux roadmap. This will also require a role of benevolent dictator to manage the evolution of the cloud. Given the level of investment required to build and operate the cloud I don’t believe that this is likely scenario.
The new PC: The open source community accepts the cloud as the new HW/OS platform. Instead of building apps on top of x86 platforms (Wintel, Mac…), open source applications would be built on top of Amazon Web Services or Google AppEngine APIs. And these apps would handle the portability of data so that data doesn’t get locked in the cloud.

At the end of the day, cloud computing equals utility and utility creates stability. And a stable set of APIs, protocols and standards is a great place for open source to flourish. The best open source projects grew on top of stable standards: MySQL/SQL, Linux/x86, Firefox/http/HTML. I wonder what will be the most important OSS that will grow on top of the cloud…

Comments

  1. Roman,

    I think your “cloud is the new PC” analogy is the right one, but it’s not really a solution for open source. If the cloud is the OS, as I’ve been arguing since the beginning of my open source and web 2.0 advocacy, we have to ask what kind of OS it is. Some of the subsystems will be familiar: storage, compute, etc. But others will be data subsystems like location, the social graph, product ids, etc. And all of these look to be owned by someone.

    The real question is whether they are all owned by the SAME someone.

    If you look at Linux, the real secret may not be open source but the architecture that allows components from many sources to be integrated into what appears to be a single operating system, but is really a swarm of components from thousands of different providers.

    Yes, there is a Linux kernel, but the kernel alone doesn’t make or break a Linux system, or define all the characteristics of the OS.

    I hope that in the cloud OS, there are components from Amazon, Google, facebook, twitter, and many others, all cooperating using a similar set of rules.

    Defining what those rules are, as the architects of Unix and the Internet defined what the rules of cooperation were for their respective systems, is one of the great challenges for anyone who cares about openness and innovation in computing.

  2. I think there’s at least one case where the “private cloud” makes sense – if you’ve got *massive* amounts of data to process. (Think movies, 3D rendering, etc.)

    You do need the massive power of a compute cloud, but you have no way of transferring the data in/out quickly enough. Only solution – build your own data center.

    • Roman Stanek says:

      But is it really a cloud or just virtualized HW?

      • What is the difference? Does cloud computing require the computers to be remote? Is amazon then, by that definition, not using cloud computing? ;)

        If the difference is only scalability – that’s already there. A lot of places share cycles on workstations for expensive computations. And it would be nice if that sharing used the same API as sharing with outside resource providers. (Ultimately, you can use EC2 as a spillover for your local computing, then)

  3. Roman Stanek says:

    I believe that elasticity and not remote computing is key attribute of the cloud. In other words cloud is about opex and not capex.

    • As I said above – with the advent of multicore CPUs, you can have local elasticity, too. A lot of desktop compute work requires burstable capacity and leaves a lot on the table, most of the time.

    • If one replaces a bunch of departmental computing resources with an enterprise cloud, the departments then see the ‘elasticity’, and the enterprise likely sees lower capex due to better utilization of resources. And BTW, CIOs will probably love getting control over those departmental computing $.

      If you think that enterprises will be lining up to do HR or accounting or even engineering on SP clouds you are seriously underestimating folks’ security concerns, legal/licensing issues, etc.

  4. It’s too bad that a true ubiquitous computing seems to be going out of fashion. Who wouldn’t want to share resources securely with the guy next door just because he had the cloud os installed. It’d be interesting to see how disparate the nodes making up a cloud could get before the reliability went down and whether it could be done without a dictator, good or bad.

  5. defining the cloud as websphere (ibm) or just another implementation of xen (amazon) does not do the opportunity justice. why not think about standardized data (ox.io) instead? …!

    I think open source is all about standardization. it’s been wildly successfull in the operating system space (chrome os and google are based on linux, no?), why not be a little open minded and aim to replicate this success on top of the stack? where it counts.

    standardize data the open source way.

    to mashup servers. not browsers. to enable computers to work for us. instead of the other way around.

  6. jason gabris says:

    Roman,

    I don’t agree with your assertion that private clouds are “a delusion”. Despite the tremendous hype that we’re seeing around Google and Amazon, a large proportion of our Enterprise (and public sector) customers are looking at building what they term private clouds. (The definition of which, BTW, is all over the map…many of whom are really just using cloud as a euphemism for moving to a modern, heavily virtualized data center environment).

    We are seeing many enterprises *today* with funded projects for internal clouds. The elasticity mentioned is one of the benefits, but ranks behind others such as reduced OpEx, increased agility, and increased responsiveness to the business.

    In addition, many organizations have some serious misgivings about potentially moving work and data out to an external cloud — how do they ensure that these remote systems are in compliance with their regulations or operational policies? How do they measure and manage the SLAs that their business users expect? What about data security? How are these systems audited?

    I’m not saying that public cloud usage isn’t going to happen, but I believe that the preponderance of enterprise spending on “cloud” over the next couple years will be around private cloud creation and management – at least for production systems. Now, for development or non-production systems, I can easily see an uptake in the use of public clouds — there are far fewer concerns about the questions mentioned above, plus the workloads tend to be much more variable in scope and scale than for production systems.

  7. jason garbis says:

    Good lord, I didn’t even spell my own name correctly!
    I am chagrined…..

Trackbacks

  1. [...] Cloud Perspectives In his latest blog post, Roman Stanek contemplated about the disruptive effect of cloud computing. We already experienced its power in relation to the boom of social networking and as he notes it is undoubtedly going to disrupt the business model behind commercial software. Now, Roman asks how it will affect the open source movement? [...]

  2. [...] calling cloud computing, changing technological landscape as we knew it. As noted by Roman in his original post, the cloud environment comes with a different layer of interaction. The technological challenges [...]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 5,659 other followers