It’s not a shock to state that cloud computing will disrupt the business model of commercial software. But how it will affect the open source movement?
The rise of open source is clearly linked to the rise of the web. Buy a commodity piece of hardware, download source code of any of the thousands of open source projects and start to “scratch your own itch”. My Linux box will communicate with your Linux box as long as we stick to some minimal set of protocols. The web is loosely coupled and software can be developed independently in a bazaar style.
It’s not quite as straightforward in the cloud. Clouds are also composed of thousands of commodity PCs, but the cloud operator manages the overall architecture and deployment – power supply, cooling, hypervisors, security, networks and so on. We don’t rely on minimal set of protocols in the cloud. On the contrary the cloud is defined by fairly complex, high level APIs. Even though the actual cloud OS may come from the open source domain, the tightly coupled nature of the cloud prevents users from modifying the cloud software.
There’s a lot of talk today about setting up private clouds with an Open Source Cloud OS, but the idea of private clouds is simply a delusion. Since the owner of private cloud has to purchase all required HW upfront, private clouds don’t provide the main benefit of cloud computing: elasticity. Other people will claim that clouds are not compatible with the open source movement or call it outright ‘stupidity’.
I see two possible solutions to this problem:
Benevolent dictator: Leading cloud providers (Amazon, Google, MSFT) will open-source their complete stack. This means that they would let the community to inspect the code, fix bugs, suggest improvements and define a clear roadmap similar to the Linux roadmap. This will also require a role of benevolent dictator to manage the evolution of the cloud. Given the level of investment required to build and operate the cloud I don’t believe that this is likely scenario.
The new PC: The open source community accepts the cloud as the new HW/OS platform. Instead of building apps on top of x86 platforms (Wintel, Mac…), open source applications would be built on top of Amazon Web Services or Google AppEngine APIs. And these apps would handle the portability of data so that data doesn’t get locked in the cloud.
At the end of the day, cloud computing equals utility and utility creates stability. And a stable set of APIs, protocols and standards is a great place for open source to flourish. The best open source projects grew on top of stable standards: MySQL/SQL, Linux/x86, Firefox/http/HTML. I wonder what will be the most important OSS that will grow on top of the cloud…
Roman,
I think your “cloud is the new PC” analogy is the right one, but it’s not really a solution for open source. If the cloud is the OS, as I’ve been arguing since the beginning of my open source and web 2.0 advocacy, we have to ask what kind of OS it is. Some of the subsystems will be familiar: storage, compute, etc. But others will be data subsystems like location, the social graph, product ids, etc. And all of these look to be owned by someone.
The real question is whether they are all owned by the SAME someone.
If you look at Linux, the real secret may not be open source but the architecture that allows components from many sources to be integrated into what appears to be a single operating system, but is really a swarm of components from thousands of different providers.
Yes, there is a Linux kernel, but the kernel alone doesn’t make or break a Linux system, or define all the characteristics of the OS.
I hope that in the cloud OS, there are components from Amazon, Google, facebook, twitter, and many others, all cooperating using a similar set of rules.
Defining what those rules are, as the architects of Unix and the Internet defined what the rules of cooperation were for their respective systems, is one of the great challenges for anyone who cares about openness and innovation in computing.
I think there’s at least one case where the “private cloud” makes sense – if you’ve got *massive* amounts of data to process. (Think movies, 3D rendering, etc.)
You do need the massive power of a compute cloud, but you have no way of transferring the data in/out quickly enough. Only solution – build your own data center.
But is it really a cloud or just virtualized HW?
What is the difference? Does cloud computing require the computers to be remote? Is amazon then, by that definition, not using cloud computing?
If the difference is only scalability – that’s already there. A lot of places share cycles on workstations for expensive computations. And it would be nice if that sharing used the same API as sharing with outside resource providers. (Ultimately, you can use EC2 as a spillover for your local computing, then)
I believe that elasticity and not remote computing is key attribute of the cloud. In other words cloud is about opex and not capex.
As I said above – with the advent of multicore CPUs, you can have local elasticity, too. A lot of desktop compute work requires burstable capacity and leaves a lot on the table, most of the time.
If one replaces a bunch of departmental computing resources with an enterprise cloud, the departments then see the ‘elasticity’, and the enterprise likely sees lower capex due to better utilization of resources. And BTW, CIOs will probably love getting control over those departmental computing $.
If you think that enterprises will be lining up to do HR or accounting or even engineering on SP clouds you are seriously underestimating folks’ security concerns, legal/licensing issues, etc.
It’s too bad that a true ubiquitous computing seems to be going out of fashion. Who wouldn’t want to share resources securely with the guy next door just because he had the cloud os installed. It’d be interesting to see how disparate the nodes making up a cloud could get before the reliability went down and whether it could be done without a dictator, good or bad.
defining the cloud as websphere (ibm) or just another implementation of xen (amazon) does not do the opportunity justice. why not think about standardized data (ox.io) instead? …!
I think open source is all about standardization. it’s been wildly successfull in the operating system space (chrome os and google are based on linux, no?), why not be a little open minded and aim to replicate this success on top of the stack? where it counts.
standardize data the open source way.
to mashup servers. not browsers. to enable computers to work for us. instead of the other way around.
Roman,
I don’t agree with your assertion that private clouds are “a delusion”. Despite the tremendous hype that we’re seeing around Google and Amazon, a large proportion of our Enterprise (and public sector) customers are looking at building what they term private clouds. (The definition of which, BTW, is all over the map…many of whom are really just using cloud as a euphemism for moving to a modern, heavily virtualized data center environment).
We are seeing many enterprises *today* with funded projects for internal clouds. The elasticity mentioned is one of the benefits, but ranks behind others such as reduced OpEx, increased agility, and increased responsiveness to the business.
In addition, many organizations have some serious misgivings about potentially moving work and data out to an external cloud — how do they ensure that these remote systems are in compliance with their regulations or operational policies? How do they measure and manage the SLAs that their business users expect? What about data security? How are these systems audited?
I’m not saying that public cloud usage isn’t going to happen, but I believe that the preponderance of enterprise spending on “cloud” over the next couple years will be around private cloud creation and management – at least for production systems. Now, for development or non-production systems, I can easily see an uptake in the use of public clouds — there are far fewer concerns about the questions mentioned above, plus the workloads tend to be much more variable in scope and scale than for production systems.
I’ve seen this movie so many times before: Back in 1997 everybody told me that Java was by definition too slow and no developer would ever need Java based IDE (http://roman.stanek.org/innovation-case-study-netbeans/). In 2000 web services did not have the enterprise features of CORBA and EJB and look how many services we consume via REST today.
Tools for compliance, security, SLAs and policies are being built out as we speak (http://s3.amazonaws.com/aws_blog/AWS_Security_Whitepaper_2008_09.pdf). And employees of Amazon or Google are disinterested parties (unlike internal IT staff) and so less likely to breach the policies and procedures. The overall benefit of public clouds will be very obvious in a few years. …
BTW: Enterprises use cloud based HR for the last 50 years (http://www.investquest.com/iq/a/adp/main/archives/anniversary.htm): it is called ADP.
The Web Services analogy is an interesting and relevant one — demonstrating that while we can anticipate where technology will go, there is always a great deal of uncertainty about exactly where we’ll end up.
Let’s go back in time to the beginning of Web Services, when the industry was on track to re-create all the CORBA services in XML, leading to a smorgasbord of in-progress WS-* standards. So where are today? The majority of us use REST – a far simpler and far more successful system.
So yes, public cloud providers can and will build out the kinds of management and compliance infrastructures that enterprises need to be able to confidently make use of them. But to be clear, private clouds are no delusion…we have dozens of enterprise customers building them today. We’re seeing several RFI’s for internal cloud projects each month!
Good lord, I didn’t even spell my own name correctly!
I am chagrined…..
Asking the open source industry to think of the cloud as the new HW/OS platform makes sense. But there’s a key reason why most open source runs on a LAMP stack and not a Windows stack, and that is sovereignty.
While as Tim notes, LAMP is an aggregation of components from lots of different sources, none of those components is exclusively licensed or locks you in any other way to a specific vendor. That means we can have a competitive commercial ecosystem of LAMP hosting providers, which drives all sorts of beneficial behaviors, competing on price and quality of service and SLA.
In contrast, if I build to Google App Engine, I have no other hosting choice if I decide that their hosting of my app is insufficient. If there is an independent implementation of the App Engine runtime that can be hosted by other cloud providers, then clearly that’s less of a concern. Amazon EC2 has less inherent lock-in since the deployment image is mostly the same as what you’d run in any other virtualization system; but the more you use company-specific APIs like S3, or AE, or Salesforce.com’s language, the more lock-in you inherently have.
I believe that the usual market pressures will push the cloud providers towards common interfaces and possibly eventually a common platform, the way that they drove the Unix vendors to Linux. Cloud app builders can help us avoid another twenty-year mistake like that, by encouraging cloud providers to move to common platforms (or by implementing alternative runtimes – OpenS3, or OpenAppEngine, anyone?)
Brian
p.s. – my gut tells me there *should* be an “OpenAppEngine” runtime project somewhere already, but I couldn’t find it, please prove me wrong! Especially now that we have a decent BigTable equivalent with Hadoop, etc.
Roman, I have the opposite view when it comes to private clouds. In particular, every organization has tons of underused capacity. That’s your source of elasticity; there’s no need to go to Amazon to get that. Instead what you need is simple tools that make that available to all ala EC2++ style.
I blogged about this and its relationship to Google Chrome OS and SOA a few days ago. See: http://sanjiva.weerawarana.org/2009/07/googles-new-os-project-soa-and.html
Hi Roman,
Personally, I think that there is a lot to be learned from the Twitter ecosystem. Here you have at the core, a message, and that message can have a payload like a URL, but with bit.ly you start to have simple wrapper mechanism for the payload itself.
This is an elegantly simple and powerful concept, inasmuch as because of the openness and transportability of this messaging model, and thanks to an open API structure, you have third party service builders that are creating all sorts of interesting handling logic mechanisms for aggregating, filtering, transforming and presenting this data in new and interesting ways.
Some, like TweetDeck, build client-side extensions. Others, like StockTwits, build server-side extensions. Yet the same message at the core remains unaffected.
Granted, a 140 character message is a heck of a lot simpler than a complex data store, but it seems that when you solve the right problem at the core, all sorts of unexpected upside surprises drop into your lap.
Hence, to me the fundamental “IT” becomes figuring out what it means to be native and cloud-ified; how you deal with asynchronous and real-time states within this domain; and what layer of the stack is elemental, and as such, needs to be open and transportable, and what layer can be proprietary and closed.
On some level, it hearkens back to the seven layer OSI networking model, which facilitated both interoperability and proprietary-ness at the same time.
Cheers,
Mark
–
READ: Right Here Now Services: Weaving a Real Time Web Around Status
http://bit.ly/i40h
The blackberry Tablet is one of the very best for yourself by comparing the deals offered inside the industry with this astounding device.
Samsung has thrown into the mix more on those in
a second. Still, we’d say it’s getting quite hard
to tell these tablets apart from one another, on the other
hand, you can customize this application however you desire.