Stu says stuff

The FSF wants you to avoid the iPhone 3G in favour of OpenMoko Neo FreeRunner.

Some of the FSF's points have merit, but some are outright deceptive - for example that the iPhone exposes your whereabouts without your knowledge. It doesn't; in fact it always pops up with a confirmation before accessing the "location services", whether GPS or cell-tower triangulation. And you can turn it off through a global setting.

The biggest joke is the alternative. Freedom to modify and tinker and extend means little when the basis is barely useable. People will often tradeoff one freedom for another freedom. A more usable device arguably is just another form of freedom -- from hassle, annoyance, and wasted time.

Open source makes economic sense in many contexts, but I think they're at least 10 years away from even denting the mainstream in this market. Which is why, BTW, I think the FSF is targeting the iPhone. The iPhone represents a refutation to the idea that open, free, collaboration will necessarily lead to better products, and that proprietary software makers cannot compete.

The OpenMoko counter-argument is "give it time, in the long run, it will win". And look, in a way, I hope so. Using the iPhone is a great case of following Keynes' adage, in the long run, we are all dead., where we optimize for short term gratification at the expense of our future. By using the iPhone, we're supporting and contributing something that doesn't build something open for our collective future, but instead leases our future over to Apple. On the other hand, the iPhone does represent something that is important to our future -- the triumph of entrepreneurship over bureaucrats and technocrats. More on that in a moment.

I'll note that the economic angle isn't usually the FSF's preferred line of argument. It's more the OSI's tactic, though the FSF has certainly referenced it. On the other hand, the FSF argues for free software from an ethical stance. Without getting into the muck, my opinion basically is that the sort of freedom the FSF advocates is not, IMO, political freedom, and thus I don't consider it sacred. Whether one chooses to be imprisoned by license is an economic tradeoff, not capitulation to evil. I believe the author should have the option to retain certain limited rights, for a limited time, over its users. I believe in entrepreneurship and the Schumpeterian model of the economy. Profits should go to the innovators for a limited time, as profit is the source of tomorrow's jobs and developments.

The iPhone represents a triumph of entrepreneurship -- the Cathedral over the Bazaar. The user experience loosens the telecom bureaucrat's insistence on device control, or the technocrat's desire for infinite options. The rules of bazaar development are flipped: Scratch the user's itch, not the developer's. Release "when it's ready", not early. Users are not co-developers, and developers aren't even co-developers, they're cordoned off into their controlled area for the sake of the user experience.

But wait, several complications to this picture:

- Open source is not incompatible with entrepreneurship. The Mozilla Foundation demonstrates this regularly with Firefox. OTOH, the market dynamics of browsers imply that there's not a lot of money to be made through direct distribution. Moz is funded largely by complementary product placement: redirecting searches to Google.

- The iPhone OS X already uses plenty of open source: its web browser is open source, as is much of its operating system layer (including both BSD and GPLv2 code). They're both complex, mature areas of computing, where it's (again) hard to declare some kind of "secret sauce" that needs protection. So it is a great place for open source collaboration.

- Apple is exceptional in its ability to successfully deliver great software and build a thriving community with a cathedral model. Most aren't. Though, looking through Freshmeat, I'm not sure the ability to build solid software with a thriving community is intrinsic to either model. It's just hard to do.

One perspective is that the architecture of a project likely has more to do with its success, quoting Roy Fielding:

In spite of the hype and hysteria surrounding open source software development, there is very little that can be said of open source in general. Open source projects range in scope from the miniscule, such as the thousands of non-maintained code dumps left behind at the end of class projects, dissertations, and failed commercial ventures, to the truly international, with thousands of developers collaborating, directly or indirectly, on a common platform. One characteristic that is shared by the largest and most successful open source projects, however, is a software architecture designed to promote anarchic collaboration through extensions while at the same time preserving centralized control over the interfaces.

Which may bode well for Apple's approach to applications.

- Apple also has found a "secret sauce" that free software rarely measures up to: usability and aesthetics. "Taste" is a hard thing to replicate, especially when it's delivered as a tight coupling between software and hardware.

A followup on the Convenience vs. Correctness debate. I made a longish comment on SJ's blog about what the nut of our disagreement seems to be: how to improve IT! Please forgive typos and grammatical errors, I wrote it on the iPhone 3G...

One approach assumes that technology will always be a crapfest, thus we should focus on the business problems, create a roadmap, and map backwards on to whatever technology turd pile that stinks least for our business / skillset / culture.

The other approach assumes that business can and will change due to technology progress, thus technology can be made to smell better. We should focus on improving our current tech turds and crapfests to better fit big picture challenges (like those of businesses) so that they stink less, and thus businesses can leverage them to change their competitiveness.

I don't think these approaches target the same audiences, one is clearly the mindset of an IT consultant, the other seems to come from activists at IT vendors.

I also don't think either approach is necessarily wrong, except that both approaches carry biases make communication frustrating. ;-)

In our last episode, a minor bun fight ensued about the way to think about convenience vs. correctness in distributed interactions.

I made a number of bold claims, such as "the vast majority of RPC systems, from RMI, to CORBA, have --all-- been focused on mapping the design of a system into programming language constructs that completely gloss over most of the issues inherent in distribution".

In the comments section, Dan Creswell asked, "Can you provide some example issues that RMI and CORBA gloss over please?" I guess I'm asking are we talking latency, failure, something else?

Similarly, Steve Jones asked, "Seriously I just don't know the people who worked like that. I was working on distributed systems throughout the 90s and in NONE of them did we consider local = remote."

To which I say, "yes latency, yes failure, but mostly semantics.".

To use latency as an example, I have worked on at least two or three financial trading systems that involved 50 to 500 remote procedure calls to perform a transaction, wreaking havoc on both performance and recoverability. I know that anyone who works in the wireless telecom industry is likely familiar with a certain telecom billing systems' Java wrapper API, which was designed completely around performing approximately 2-3 network round trips for every field you needed to set in a transaction (which was a stateful session bean that talked to entity beans), leading to between 30 and 150 RPC's for every transaction. People wonder why so many business transactions failed, and why so much infrastructure was required to support a small volume of business transactions.

The problem is that, as I believe Steve Jones has pointed out before, there are too many projects without any real experts available on them, so the developers building the systems (which involve billions of dollars of transactions) aren't aware of any innate issues of distribution, so build the network interactions with the approach they know -- local object-oriented programming. This is also why vendors continue to build tooling that doesn't deviate from local object-oriented programming, as it's risky to try to shift the mindset of developers to something different.

The semantics area is one that's been discussed many times with regards to REST's uniform interface, so I won't repeat it here, though we could dig into it more if some would like to.

Then, Steve Jones noted the following:

This is my issue. At the moment people are chasing a perceived technical perfection over a practical set of improvements. RPC works sometimes, REST works sometimes, Messaging works sometimes, flying monkeys only work on certain edge cases. The real focus should be how to think about the overall business problem, the continual belief that the next technology will make IT more successful is a big part of the problem.

I have run into this opinion before from Steve several times in the past. It almost is, to me, a form of technology fatalism. Many IT executives that I know and respect share this opinion. So, I respectfully disagree, and will try to explain why.

I agree that technology alone will not make IT successful.

I do not deny that there are variable levels of skill and productivity among IT workers.

I reject the notion that ensuring that all IT workers understand the business better, though highly desirable, is a realistic solution to the problem of IT failure.

I also reject the notion that we cannot raise the bar of productivity across the board through better technology, for a variety of reasons that I will explain next.

Firstly, there will always be significant numbers of IT workers that do not, and cannot (!) understand business problems to a greater degree than technological problems. This is for a variety of factors (educational, for one; engineers aren't usually schooled in those liberal arts that provides insights into humanity and society). This problem goes both ways: there will always be significant numbers of business people that are clueless with regards to technology, for better or worse (as one's has only so much time in the day and thus must choose to be ignorant of some topics). It may be possible to fix this problem, but it is a problem for entire generations of workers, not something fixable with anything less than a 30 year time window.

So yes, if you don't understand the business, you're dead. But, there's a lingering problem: if your business problem relies on technology, and you're either not aware of the appropriate technology to solve your problem, or you screw the technology up, you're likely still dead.

In short, contrary to the popular saying, it's not just about the business. But it's also not just about the technology. It's about both. They reflexively impact and depend upon one another.

History seems to show this relationship. The development of the steam engine, the telephone, the railroad, the automobile, the television, the computer, the web, etc. have all shaped society in ways it did not expect. Few of these were driven by 'understanding the business better'. This is not to say that all IT projects are on the level of these innovations. It is, however, an recognition that major innovations come with major changes, some of which are hard to interpret for those stuck in the prior model. (e.g. Television impacted U.S. presidential politics as early as 1960, with the Nixon vs. JFK debate).

The moral of this story is that the technology-influenced mindset and architectures we take to our business initiatives will have a direct impact on how the business performs. There's an old saying in software product development: "TEAM = PRODUCT", also known as Conway's Law. Technology is often a necessary, even if not sufficient, enabler for those other, more important business changes. And much of that enablement is emergent behaviour, i.e. not directly traceable to the functional implementation of a business requirement. Rigid, brittle technology, mixed with a dose of fear, can often provide enough inertia to kill the best laid hopes of broader change.

So, tying back to our discussion of network interactions and RPC's. The problems I have with suggestions such as "an expert can make RPC (or EJB, or WS-*, or.. *) work well in practice" are:
a) they're generally correct in their outlook, for a particular audience;
b) they sometimes use the Internet's architecture and principles as an example of the right way to do things;
c) but they expect every IT department to build their own version of the Internet's architecture with the toolboxes that vendors give them.

Yet the vendors themselves are stuck in the approaches and techniques of yesteryear - old wine in new bottles, while they have nearly always worked to actively dissuade adoption of alternatives to their approach, or even destroy those who support new alternatives. Technology debates and standards bodies become proxy wars for market competition, and progress in the technology debates and standards is often more about fixing the political situation, not the actual technology itself. How is an IT department supposed to know any better if industry leaders aren't raising the bar for them?

The problem with approaches like "yet another RPC stack" are that they are not catalyzing changes in technology architecture much beyond where they were in 1984. Similarly, the WS-* world has not been catalyzing changes in technology architecture much beyond where it was in 1998. It's still, basically, component software, just now focused on protocols of runtimes. Much of what we have is improved tooling for Visual COBOL. And there are even signs that the vendors are moving back to emphasizing runtimes.

In summary, I think improving technology is necessary along with (not instead of) an increased understanding of business problems, because I think they help each other out. IT's business value basically comes down to providing tools and catalytic ideas to change the transaction cost structure of a firm. You can't do it without the business, but you often won't have the germ of the thought without innovative technology and associated architectures.

A response to Steve Jones' post criticizing what I thought was a great article by Steve Vinoski. In this response I will refer to Steve Jones as "Steve" and Steve Vinoski as "Vinoski".

Steve, I normally enjoy your business services architecture discussions, but I think you are flat wrong in this post. You seem to be revising a whole lot of historical context in this entry, and I think you've seriously misread many aspects of the article. Furthermore, I think you've contradicted yourself several times in this rant.

"No decent architect built an RPC system which assumed local = remote"

But, that was *the whole point* about RPC systems to start with, and that tradition was carried on WELL into the late 1990's.

" What a load of crap. Seriously this is an unmitigated pile of tripe in what it means to write distributed systems. It makes two basic errors.... That the architecture and design of a system is focused on a programming language"

And that was entirely his point! The point of the article was that one should never view this. YET -- vast majority of RPC systems, from RMI, to CORBA, have --all-- been focused on mapping the design of a system into programming language constructs that completely gloss over most of the issues inherent in distribution, and forces the semantics of the network interchange to conform to what's comfortable in the programming language!

Look back at Don Box's "XML Manifesto" from 1998, which was basically claiming that the reason for XML was to wrest control of distributed systems interoperability from the programming geeks and use a document geek's standard, because that's the only way we can stop this "my language is better than your language" crap.

At worst you could accuse Vinoski of repeating an insight that's 10 to 15 years old. But, in my opinion, he is going down this road because

1. we still see RPC systems being built and released (Etch) for some reason
   
2. most SOAP toolkits are still RPC-oriented (hello JAX-WS) even though they could be messaging-oriented. I recall when JAXM was rolled back -- no one wanted it!
   
3. Most WSDL interfaces are still very RPC-oriented
   
4. a common mistake in RESTful design is trying to overlay RPC on it because it's easier for a programmer to work with
   

The answer to interacting with the network should not be "bend the network to make my life easier", it should be "make my life as easy as the reality of the network will let me".

"Architectures are normally programming language independent and anyone who starts a distributed system by writing C++ is a muppet who should be fired."

Yikes. My Top Gear cockometer has just been broken (you sure you don't drive an M3? ;-)

Unless I'm misreading, "Programming languages don't matter, but if you're using C++, you're a muppet." I understand in an enterprise IT environment using C++ is somewhat anathema (for some good reasons), but not everyone is in an enterprise IT environment.

"Errr apart from muppets who thinks that a distributed RPC call can be treated the same as a remote call? Every single RPC approach has a litany of exceptions that you have to catch because you are doing a remote call, that is baked into the frameworks."

Those muppets you cite seem to be every vendor that's ever built a toolkit around RMI, CORBA, or SOAP.

And no, not every single RPC approach has a litany of exceptions that you have to catch. That was Waldo's "innovation" in RMI -- forcing RemoteException to be checked. Because, as even a cursory examination of history will show you, most distributed developers never checked it. It was brittle because failure was rarely taken into account.

This is getting better, but I've seen plenty lot of cases in the EJB, CORBA, and SOAP world where failure was rarely taken into account other than basic retry logic. But I'd even see retries in non-idempotent, non-transactional invocations (oops, sorry we billed you twice!). "Oh, but this never would happen in my system", you may scoff. Great, I'm glad for you. But that's not a scalable solution to the problem.

Now, here is one large steaming contradiction:

"The writer's position is that everyone should know about the network, this is a false position as it requires everyone to be of the same sophistication as the, very talented, writer."

vs.

"The reality is that all decent architects have always known that the two are different and have worked accordingly... This really is the worst kind of argument. Setting up a completely specious argument as the basis for this area really does undermine the whole credibility of the writer. If the writer is saying that when they built RPC systems in the past, which they apparently have, that they treated remote and local as the same then the writer is a muppet."

Which is it? Any decent architect "knows this stuff"-- they need to know the network. Yet, they can't know the network, because they're not that sophisticated?

Why is it that all the toolkits provide dumb interfaces to the network *without really handling any of these issues*?

What about the end-to-end argument, wherein you can't really provide for true reliability and security without the application-layer understanding the implications (i.e. it's not something you can "abstract" away and still have a complete solution)? This argument belies much of the Internet's architecture and has served us well. Shall we dispose it because it doesn't fit into our worldview of "ubermench and muppets"?

The point is that a skilled architect can't just provide an "uber abstraction" to make these problems go away in a transparent fashion. I know in your world of muppets and puppetmasters this is desirable. But the whole point of Vinoski's paper, and Waldo's, and Brewer's conjecture, and even Vogel's writings on the topic, is "Abstraction > Transparency". You can, at best, abstract the requisite variety of a network down to a protocol or API, but you still have to understand the domain that the abstraction is describing, a domain that is not necessarily aligned your programming paradigm, eg. objects, arrays, triples, relations, etc.

You of all people should understand this argument as it's the whole reason why so much technology in SOA is problematic and the source of "business SOA vs. technical SOA". Many of the toolkits developed are written by programmers, for programmers, and not with the reality of the network in mind.

"The suggestion in the text around is that its impossible to build a large scale system using RPC as it lacks these abilities. .... this must come as a big surprise the to the companies doing ESBs, Web Service monitoring or Web Service gateways which in fact allow all of these elements in just such the RPC environment that the writer claims isn't possible. Nothing in RPC could ever indicate whether a result is cacheable for instance."

The point is not that this is impossible. It's that it is very difficult to achieve at scale due to the lack of standardized semantics! Sure, anyone can solve those problems in their little pocket of the world, but how are you going to get multiple organizations in multiple geographies under different governance structures to agree on these things?

As an example, caching at an XML infoset level is hard to do, slow, and brittle. The ESB would need to have coupled knowledge of all of the IDLs or WSDLs that pass through it, and each operation's safeness or idempotency, to be able to understand what is and is not cacheable. We would need a standard representation for a cache key -- would that be a WS-Addressing reference parameter? An XPath into my SOAP body? What? Then, if the body were cacheable, we'd need to deal with XML canonicalization issues. None of this is standardized. So the solution usually is to give up with intermediaries and just do caching at the implementation layer.

The point is that introducing an intermediary in an RPC network is an exercise in neurosurgery. Introducing an interemediary on the Web happens all the time. Does it magically whisk away enterprise-specific needs, like data transformation? Not necessarily. But it has been shown to work well for global-scale general purpose features, like routing (see CDNs), caching, streaming, and redirection.

REST of course address all of these concerns (and more) by having... a field that says you can cache the result something that is trivial to add to an RPC environment but is lauded as hugely different. REST is great because it doesn't fit with normal programming language abstractions.

The point is not that adding a caching header is somehow new or different. The point is that, regardless of the nature of the data passing through the interface, *all of it* is accessible in a uniform semantics, and thus *all of it* is inherently cacheable. Not a bunch of local optima solutions sprouting up that need to be integrated after the fact.

So REST can be made to fit in a "normal" programming language (unless someone can show me an example that couldn't fit) so again this isn't a real argument its just prejudice and an example of the lack of sophistication in current REST frameworks and tools.

I think you've missed the point. REST is an architectural style. Can you make "Client/Server" fit in a normal programming language? Sure, but that doesn't tell me much. I usually need a lot more facilities than a programming language to make it work (file I/O, network I/O, etc.)

Contrast:

a) The purpose of most RPC toolkits is to make the network look, as much as possible, as a regular programming construct (Java object, C call, etc.) This holds for RMI, CORBA, JAX-RPC, etc. It was the whole point of object/relational mappers (which wraps SQL RPCs), and EJB entity and session beans. ON the bright side, RMI did make some things explicit that weren't in the past (e.g Serializable objects to distinguish value vs. reference semantics).

b) The purpose of a messaging toolkit (e.g. JMS or an AMQP kit like RabbitMQ) is to make the network look like an abstraction that covers much of the requisite variety in the network though still enables you build your own custom semantics of interaction. Notice that the point isn't entirely to make everything "convenient for a programmer", as anyone using JMS will attest. It has a different goal in mind than appeasing a programmer who just wants to make a remote call and be done with it.

c) The purpose of a RESTful toolkit (e.g. JAX-RS or HTTPClient or Restlet) is to make the network look like a distributed hypermedia system that covers much of the requisite variety in the network and specifies uniform semantics for all interactions as part of a hypermedia application. It too has a different goal than a programmer that just wants to send a message and be done with it.

REST is trying to raise the bar of abstraction above message passing into one of a hypermedia application, to assist interoperability and scalability at the expense of efficiency in some cases. Is it always appropriate? Heck, no. But it is a legitimate attempt of "raising the bar of abstraction" beyond just message interactions.

The problem with current approaches to RESTful client access is that the facilities are somewhat new and we don't have a lot of experience of what they should look like. There's innovation going on. One example is a client-side representation cache, like what a Web browser has. Good feature, it's an identified component in the architecture, but I don't see it in any of the REST toolkits yet.

I would also highlight that if you think all of this uniformity is somehow useless, then why are the vendors submitting several standards to the W3C that basically define standard semantics for web service interactions? And adopt some of REST's constraints? It's not because these are RESTafarians. It's because in the IT management domain, they realized they couldn't make things work if every possible IT resource had their own WSDL interface!

What frustrates me about interchanges like these is the utter disrespect and hubris thrown around. See, the way I read it, besides the REST stuff (which I think you inherently can't or won't wrap your head around), your position isn't actually that far from Vinoski's position on RPC.

Vinoski is one of the world's most experienced distributed systems architects and developers, but based on your response, you'd think the man was a recent college graduate being upbraded for inexperience. Anyone that's used CORBA in anger has likely used IONA's products in the 1990's and will know what it was capable of. Yet you immediately leap to showing how much better and more experienced you are instead of, oh, I don't know, questioning your bias? Asking Vinoski more precisely what he meant? Non sequiturs and ad hominem arguments work well in enterprise IT power politics, but I don't give them much merit when we're actually trying to solve real problems instead of spouting off how smart we are compared to the 'muppets'.

Update: Steve Vinoski adds his own response to this.

Two notes

1. I've recently been promoted - my new title at Elastra is "Chief Software Architect", and I'm now reporting to the CEO, Kirill Sheynkman. I am responsible for Elastra's technology direction across its product lines.

2. Elastra is hiring! If you want to change the world of enterprise service management and provisioning, can contribute to a world-class team, and are located in (or want to relocate to) the San Francisco Bay Area -- send me an email.

General musings about RESTful design that I'd like to write down. Feel free to comment.

For the love of ponies, stop trying to change HTTP to suit your world view.

The uniform request semantics of HTTP are generic for a reason -- it's hard to be uniform if you're not generic. If you want to describe or interpret something more specific, you'll need a media type to help you out.

The biggest problem with applying the Web Architecture for enterprise systems is that we have no broad community (to my knowledge) that's experimenting in media types to help solve problems that are common inside enterprises. That, I think, is a cultural and psychological limitation, of both the REST community and enterprise developers -- and not due to the Web Architecture being inappropriate for enterprise systems-of-systems interaction.

The main problem when you jump into the enterprise with the Web Architecture is the semantic interoperability problem. This is the same problem with other architectures, but with REST's benefits of easier interoperability, it becomes more important. We might be able to specify media types for infrastructure-level topics, like identity, authentication, or syndication, or content publishing, but we have much less success with, say, "Job Applications" or "Order Management", because those domains change faster than a media type should, and the semantics themselves are subject to disagreement across trust boundaries.

The takeway is not "and thus REST is useless in the enterprise". The point is that we have a lot of work to build media types and tools to bridge the gap between the communication actions that are uniform and the business actions that are non-uniform. See below for more on this.

REST is not CRUD.

PUT means "replace" but doesn't imply a byte-for-byte guarantee that what you send will be what you GET later. It means "here is my desired state for this URI, please replace it". It cannot be used to replace POST which means "process this please, impacting whatever resources you think make sense".

Furthermore, DELETE does not guarantee you will get a 404 Not Found or 410 Gone at the URI afterwards. There may still be a representation at that URI later!

HTTP is an abstract interface about describing agent intent. That's it. All of the intended specific side effects and preconditions need to be described in hypermedia. You can't infer too much from the methods.

State transition are a useful way to describe the lifecycle of both resources and representiations; but note that they're not necessarily the same thing

A little known fact about UML 2 statecharts: there are two types. The "protocol statechart", describes the lifecycle from the perspective of interaction, but says nothing about internals. An "object statechart", on the other hand. describes the internal states of an object.

A hypermedia representation type describes the former -- it's how an agent knows what hyperlinks are supplements to the current representation, which links are a sensing action to a separate resource, and which links are an state-transitioning action. Perhaps these categories are wrong-headed; maybe there are other sorts of links. But those three came to mind based on what I know of HTML inline links vs. anchors vs. forms. If we're to build more RESTful agents, we need to work on coming up with a general sense of how different sorts of links impact the knowledge of the agent.

BTW, only the origin server knows the internal object lifecycle, and how it corresponds to the protocol state transitions. They may be very dissimilar in practice. That's the whole point of abstraction.

Requesting domain-specific action on a resource requires the use of POST.

State transitions in the Web are modeled with PUT if you modify the representation of the state directly, or POST if you're specifying a more domain-specific (aka "business") action. Anyone who says POST is anathema to the Web is making shit up. POST is essential to the Web Architecture. If you want to remove it in your world, fine, but it's not the Web, and won't help the Web.

The problem with POST is when we abuse it by having it perform things that are more expressive in one of the other methods. GET being the obvious one that needs no hypermedia description. For the other methods, a good design guideline is that you MUST not break the general contract of the HTTP method you choose -- but you SHOULD describe the specific intent of that method in hypermedia.

For example, POST in HTML means "process this form". POST in AtomPub means "create this entry". They're very different intents. The meaning of these actions aren't inherent in POST, they are inherent in the media types (specifically the hyperlinks in an HTML form tag or an AtomPub collection tag). This is no different if you wanted to design an "order" action for an Order Processing media type.

I note that the RESTful Web Services book doesn't like POST, but they did invent their own REST-influenced architecture, ROA, which you can feel free to agree or disagree with. Much of their advice is well intentioned and right, but this one case is, in my opinion, very wrong.

Encoding a business domain in a media type

If you're using a media type to describe domain-specific data, then you have two choices:

a) use a domain-specific media type, and deal with the versioning issues,
b) use a more generic media type that describes actions and their consequences.

(a) is not aligned with the goals of REST because media types aren't meant to change very often. Contracts that change are supposed to be described (and discovered) through hypermedia. Unfortunately, we don't have a media type that does this. (On the other hand, most other approaches don't describe contracts all that well either).

But wait -- one may say, "what about RDF?" Indeed, RDF (and RDFa) does cover a good chunk of this problem. But besides issues of RDF usability, we don't really have a well-understood way of using RDF to describe *contracts* of action
successor-state axioms (i.e. state transition post conditions). It probably could do this, we just haven't really used it that way, to my knowledge. If Linked Data is to be a success, we need RDF (or some other description framework) to help describe what a POST or a PUT means for a particular hyperlink. I have my own ideas on how to do it, but I doubt I'm the only one who has thought about the problem; I just haven't seen too many solutions yet.

On using colloquial XML to describe business domains

One could use colloquial XML (i.e. XML elements that correspond to domain entities and relationships) as your media type and build forward compatibility into the squishy areas; this isn't much different to how people do things in the enterprise today with XML messaging. Just make sure you get your change management straight. Plan for forward compatibility in the XML schema, backward compability in the origin server, and versioning either in the URIspace or the schema, depending on how incompatible the version is. Contrary to rumor, RESTful versioning really isn't all that different than versioning with plain XML messaging, and it's arguably much smoother for forward compatibility than alternatives.

When using colloquial XML in this way, you're not really getting all of the benefits of REST, since, by definition, your media type is way more specific than it should be. The main benefit of building such interactions on the Web architecture is the pervasive use of URIs instead of just embedding primary keys into your documents, a practice that's very common and a big problem with data management in SOA. URIs give the benefit of at least some well-understood semantics (e.g. GET and response codes) without requiring a whole lot of media type description.

That's enough for now. I have more to come...

A belated response of sorts to JJ's post back in May...

Stu, I work for a BEA shop and my management had asked me to elaborate on all the SOA domains that you guys have defined. I can assure you that you guys were millions of miles away from providing guidance that would deliver successful SOAs. You simply did not get it.

Compared to your vision, perhaps. Though I'll note that not very many people understand your vision. This isn't to say you're wrong -- I happen to agree with some of what you say, particularly around interaction, though I don't certainly agree with everything. But you continue to denigrate pretty much everyone in this industry as clueless, whereas you seem to have "the answer", and not only is that hard to believe, it's hard to figure out what your problem with them is other than that they're "wrong". I suggest a more effective approach is to cite your sources and those others that agree with you; perhaps it will lead to better communication. Because, as much as you claim the REST community is providing no solutions, you're not really providing them either, if people are unable to communicate with you.

Regarding BEA: compared to other competitors, and even most system integrators, I think the thought leaders within BEA really did get it, at least for a while. When getting into deep discussions or consulting engagements, however, BEA's performance often was a regional affair; the U.S. West field region had a different style and approach than U.S. East, or my area, Canada.

Today, your analysis is that returning to the principles of the Web is somehow going to give us this magical programming model that everyone is looking for. What's the evidence? I am desperately seeking that evidence. Even though some people don't like my style, I am genuinely open to anything that works. I have no technology or product interest, I work for an IT organization. I do expect that people would be just as open as I am, but that's not always true.

Dude, look around. Look at how we're communicating. There's lots of evidence that the Web's architecture is something to embrace, not something to throw away or minimize. If you want more evidence of that, all I can say is "stay tuned".

Could there be a possibility that one more time you would be going in the wrong direction? Have you ever considered that your experience and training conditioned you to pretty much create the same programming model over and over? from EJB to Web Services to REST. Behind the bytes, isn't it the same thing?

Of course there's a possibility that this is the wrong direction. Point me to either a sound theory (with references) or working alternative software that should be a better direction. Though note, I'm pretty sure that the WS-* stack is not it.

Behind the bytes, from EJB to Web Services, they absolutely are similar things, at least in the session bean sense of EJB. With REST, it's pretty different. I do think it has some similarities to object-orientation (identity, reference vs. value, message-orientation, shared protocols as a basis of interchange) . I do think it can be abused to defeat the properties that it fosters. I don't think it's the be-all end-all architectural style, but basically, I like that it forces us to revisit the role of data vs. behaviour in a scalable system.

I would argue that data concerns (identity, reference, base semantics of access, provenance, the role of time & concurrency) should be the foundation of a scalable networked system-of-systems; only after we have that foundation can we address issues of behaviour, interaction and choreography. That's the main reason why I'm a REST proponent; if you want to convince me otherwise, you'll probably have to start with that assumption (e.g. should behavioural interactions be the basis? how would we ever agree on the semantics?)

Now, isn't it clear that the Web, even though it is the largest information system by far, has a very limited programming model unsuitable for building enterprise class information systems?

It's clear to me is that the Web isn't an architecture for building an localized information system. You need several styles needed for any of such things.

The Web's benefit is as a "system of systems" architecture; it's a way of dealing with issues of governance when you span systems and agencies. I don't think that it's the final approach -- there will be successors. But I think aspects of it, such as the URI, probably will endure for decades. And I think enterprises would benefit greatly if they adopted, at the very least, URIs more prevalently.

So please note, I don't think 'the answer to IT woes' is to rebuild your enterprise systems as "web applications" with RESTful resources, full stop. I think that's hopelessly naive. The point, again, is that there are many architectural styles, and the web's exhibits properties that I think would benefit most enterprises for what they've been usually been using WS-* for, in my experience. But it's far from the only architectural style that the enterprise uses or will use in the future.

Stu, the way I view it, when you say the "Web is enough" you are simply rejecting the all the vendors (small or large, from Redmond to Waldorf) as well as the standards and technologies they produce, rather than convincingly provide a solution to the problem.

I never said the Web is enough. I said that it's probably the way forward - a foundation for a next generation of sharing information at scale. I think any successors in the near future will have to start with the Web and drop or change the parts that don't work (for their domain) rather than inventing something from scratch. For example, turning the Web from client/server to event-based is an obvious one. Though we'd have to be careful not to lose the RESTful benefits; asynchronous messaging tends to lead to more complicated and less scalable systems when designed in an amateurish way.

As for rejecting all the vendors' standards and technologies -- considering the level of RESTful activity going on in Redmond and Armonk these days, I'd say they're (on some levels) doing a good enough job on their own of at least hedging their own efforts. Most standards these days are not driven by good technology, they're driven by power politics. There's only so many times you can compromise for political reasons before you just want to duck the whole system and try something new.

So, how am I providing a solution to the problem of making the Web more suitable for enterprise systems? Firstly, this isn't something one can solve in a blog. It requires working software and specifications. I don't think a Grand Unified Theory of Everything approach, the approach I worry we sometimes were moving towards in our "chats" about BPM and interactions and choreography in early 2008, is productive -- it just leads to frustration and communication breakdowns. All I can do is hope to catalyze the growth of knowledge in this area. To that end:

(a) I would like to dedicate time to frame discussions on some key problems; there's a long-awaited article I've been sitting on to discuss an architecture for business process & interactions on top of REST, which will be useful food for thought (it's a big topic, so I've had to revisit it several times, to the chagrin of my very patient editor, who's likely given up by now [grin]);

(b) There's a long dormant article or blog post on semantics of action that I'd like to get around to (about why REST or ARRESTED is a suitable foundation for enterprise systems-of-systems interactions) ;

(c) What takes most of my time up is that I'm now the technology leader at a startup in a growing industry; I'm going to do what I can to drive forward approaches that I think are productive & open. You can bet that the Web's architecture will be a part (but not anywhere near the only part) of that effort.

John Willis interviewed me yesterday for this week's Cloud Cafe podcast. We mostly talk about Elastra and cloud computing in general. I do insert some REST stuff into it ;-) I hope you enjoy it.

I've long felt that if there's ever a band that will break progressive death metal to the masses, it will be Opeth. They're the closest band I've found to stirring me the way Metallica did back in the 1980's, and after nine (!) albums, they're getting even better.

Some food for thought:

Metallica - Master of Puppets
1986 - Billboard - Position: #29
1986 - UK Albums Chart - Position #41

Opeth - Watershed (praised by the NY Times, Pitchfork, Blender, AMG, etc, reaching 88 on Metacritic)
2008 - Billboard - Position: #23
2008 - UK Album Chart - Position: #34
2008 - Finland - #1
2008 - Sweden, Norway & Australia: #7

Not that I'm predicting anything. I'm just sayin' that Watershed is, I hope, a sign of things to come.

p.s. I'm really looking forward to Death Magnetic too, by what I've heard so far. It's been over 12 years since I've been able to say that about a Metallica record.

For those attending, I will be presenting at Jazoon in Zurich on Wednesday June 25th. The talk is influenced by my work the past year on data in SOA, how REST offers some useful guidance to the problem, and how some WS-* specs are attempting to address this area.

Feel free to email me or come up and say hello!