The Cloud Interoperability Battle-Lines

I tend to think of interoperability as a gradient.

The old industry stalwart from the 1990's is what I'd call "runtime interoperability", wherein you could write a Java EE application, deploy it on a Java EE application server, and (with a questionable amount of tweaking), get it to operate. SQL was another attempt at this, with mixed success. The later CORBA standards tried too, with the Portable Object Adapter (POA). And clearly, the ISO/ANSI C runtime libraries have been successful, as have many other programming libraries.

The other angle of interoperability grew in the 2000's is what I'd call "protocol interoperability", an approach that, at first anyway, only a network engineer could love. Most of the *TP's on the internet take this approach, where the "network" is first, and dictates the pattern of interaction -- the "developer" and their desires or productivity is secondary.

With cloud computing, we're currently going through the age old discovery of "what form of interoperability makes sense?". Especially given that we're dealing with networked applications (indicating a need for "protocol interoperability") but also with complex configuration & resource dependencies for security, scalability, etc. (an area where "runtime interoperability" usually plays).

Starting Observation: Microsoft Has A Clue.

Windows Azure is trying to balance these approaches to interoperability. For example, .NET Access Control Services allow you to federate identity between your own Active Directory and Azure. This is all just Active Directory Federation Services (ADFS) and using the WS-Federation "standard"; something you could do with OpenSSO too, for example, for over a year. But they'll probably make it easier if you stick within the .NET / Windows world.

A similar case could be made with their .NET Service Bus,  as a way of enabling Windows Communication Foundation and Biztalk applications span Windows Azure and private deployment(s).  This isn't just a pipe dream, either, they're actively doing this with the early Azure releases.

The Scope of Interoperability

What makes this work is that .NET is already a widely used platform in private data centers, and that .NET is a single-source runtime.     Now, an astute observer may exclaim,  "but that's not interoperable!  Where is the multi-vendor ecosystem!?"  At which point we have to ask ourselves, what's the scope of desired interoperability?  

Is it :

- A vendor ecosystem of interoperable runtimes?  Ponder the success and market results of SQL, Java EE, etc.  before wishing for this.    Where they did make a difference?   (They did make a difference, but perhaps not where one would intuitively think.)

and/or

- The ability to enable multiple providers to host a single runtime and enable interoperable "services" (e.g. identity, data, process, etc.) across these runtimes?   

I suspect the latter is more readily attainable, and likely higher value, than the former. And note it doesn't preclude the existence of an ecosystem. It just suggests that enterprises are going to care more about cloud-spanning functionality in their "chosen car trunk" than wait for a common runtime to emerge.

What are the alternatives for a "hybrid cloud" platform to .NET and Azure?   

1. Force.com APEX might work if they invested in private deployments -- not likely.     
2. There's Java, though Sun, IBM and Oracle haven't been doing much there yet.  
3. There's EngineYard starting down the Enterprise Ruby on Rails path.   
4. Google perhaps heading down the Enterprise Python path (also not likely)

5. Infrastructure as a Service, where you could write your infrastructure in Erlang and OCaml for all your cloud provider cares (so long as you don't use multicast ;-)

In this last case, runtime interoperability would require a lot of "roll your own" configuration management, integration, and interoperability.     Or you could rely on...

6. So-called "Cloud Servers" (e.g. CloudSwitch, 3Tera, Elastra, etc.)

Which give you ways to help craft models & designs & orchestrations that help you with configuration management, integration, policy, interoperability, and governance.  Which in essence is just like what the Hybrid PaaS guys are doing above:  constraining the problem space to gain some level of deployment flexibility.   The difference is that cloud servers boil the problem down to a (hard) configuration management problem, instead of building "a standard runtime to rule them all". 

Naturally, because I work at one of those "cloud server" vendors, you'd think this is my preferred model. But honestly, I'd be pretty happy for the industry if they agreed on either model. Time will tell.

My Predictions

a) I have serious doubts about a "new" cloud runtime portability standard.    The battle lines were drawn long ago, and while they'll blur, it likely will continue to look like " .NET vs. Java vs. everything else" for at least 2+ years.

b) One could argue it would be nice to build a standard protocol (using an architecture that fits how early adopters think) for Infrastructure as a Service to provision "Obvious Stuff" like storage, CPU, and network.   The DMTF CIM stuff is great but probably too low-level,  and too WS-* focused to be palatable to early adopters.   The DMTF OVF stuff is likewise great, but isn't focused on "lifecycle", i.e. what the heck happens to this deployment over time?   It's (thus far) focused on creating virtual appliance bundles.

Something RESTful would be nice to enhance our serendipity, but frankly the EC2 API isn't all that great of a starting point (for several reasons; different discussion though).    

Regardless of the architecture, the big win here would be that it would reduce the need for a "Cloud Service Bus" that mediates among different APIs.   I think this kind of standard will happen, but it will take 2+ years, thus being ratified just as early adopters have bought their shiny new Cloud Service Bus.....  ;-)

c) A wild card is where the "massively parallel processing uber alles" crowd will flock.   From what I can tell, four visible options:  
(i) Hadoop (i.e., Java; though I bet there's a .NET port coming), 
(ii) Open Grid Forum / Globus,
(iii) Parallel SQL Database (e.g. Vertica, ParAccel, Greenplum, etc.),
(iv) Proprietary Platform (e.g. Google)

And those cases are very clearly NOT going to be a likely candidate for hybrid-cloud interoperability below the service-API level, given the latency requirements and tight coupling inside those services.

d) Even in a world of a handful of interoperable Cloud Platforms, I suspect there's a going to still be a big configuration management and governance problem.

Where does Elastra's work fit into this?

Well, first, let's be clear: I have modest expectations for our work on EDML, ECML, etc. I don't expect them to become standards. I do hope to contribute the work we've put into this stuff into a standards effort, and that the industry really does adopt a RESTful linked data approach to describing IT. On the other hand, we've been at this for over a year, and I doubt there will be industry consensus on even 20% of the topics we're modeling. EDML, with its emphasis on resource reservation, allocation, packages and settings, sure, I could see value. ECML, however, is an architecture and policy description language; I suspect there's a lot more acrimony awaiting in there.

Secondly, I have no illusions about the ability for a startup or even a "community" of individual contributors to influence or fund a standard with this much industry attention. Large companies will get their way, invariably. Good ideas may survive through a combination of luck, serendipity, and maybe small doses of charisma and chutzpah among the evangelizers.

So, I will continue to show up at the occasional interoperability or standards meeting, post on mailing lists, etc., but otherwise I'm focused on our product suite. We built these languages to get our jobs done, and are happy to open them up when we have the time to complete the documentation for a wider audience. For now, I'll be presenting highlights at the OMG cloud interoperability meeting in March.