On effective architectures for enterprise IT

A belated response of sorts to JJ's post back in May...

Stu, I work for a BEA shop and my management had asked me to elaborate on all the SOA domains that you guys have defined. I can assure you that you guys were millions of miles away from providing guidance that would deliver successful SOAs. You simply did not get it.

Compared to your vision, perhaps. Though I'll note that not very many people understand your vision. This isn't to say you're wrong -- I happen to agree with some of what you say, particularly around interaction, though I don't certainly agree with everything. But you continue to denigrate pretty much everyone in this industry as clueless, whereas you seem to have "the answer", and not only is that hard to believe, it's hard to figure out what your problem with them is other than that they're "wrong". I suggest a more effective approach is to cite your sources and those others that agree with you; perhaps it will lead to better communication. Because, as much as you claim the REST community is providing no solutions, you're not really providing them either, if people are unable to communicate with you.

Regarding BEA: compared to other competitors, and even most system integrators, I think the thought leaders within BEA really did get it, at least for a while. When getting into deep discussions or consulting engagements, however, BEA's performance often was a regional affair; the U.S. West field region had a different style and approach than U.S. East, or my area, Canada.

Today, your analysis is that returning to the principles of the Web is somehow going to give us this magical programming model that everyone is looking for. What's the evidence? I am desperately seeking that evidence. Even though some people don't like my style, I am genuinely open to anything that works. I have no technology or product interest, I work for an IT organization. I do expect that people would be just as open as I am, but that's not always true.

Dude, look around. Look at how we're communicating. There's lots of evidence that the Web's architecture is something to embrace, not something to throw away or minimize. If you want more evidence of that, all I can say is "stay tuned".

Could there be a possibility that one more time you would be going in the wrong direction? Have you ever considered that your experience and training conditioned you to pretty much create the same programming model over and over? from EJB to Web Services to REST. Behind the bytes, isn't it the same thing?

Of course there's a possibility that this is the wrong direction. Point me to either a sound theory (with references) or working alternative software that should be a better direction. Though note, I'm pretty sure that the WS-* stack is not it.

Behind the bytes, from EJB to Web Services, they absolutely are similar things, at least in the session bean sense of EJB. With REST, it's pretty different. I do think it has some similarities to object-orientation (identity, reference vs. value, message-orientation, shared protocols as a basis of interchange) . I do think it can be abused to defeat the properties that it fosters. I don't think it's the be-all end-all architectural style, but basically, I like that it forces us to revisit the role of data vs. behaviour in a scalable system.

I would argue that data concerns (identity, reference, base semantics of access, provenance, the role of time & concurrency) should be the foundation of a scalable networked system-of-systems; only after we have that foundation can we address issues of behaviour, interaction and choreography. That's the main reason why I'm a REST proponent; if you want to convince me otherwise, you'll probably have to start with that assumption (e.g. should behavioural interactions be the basis? how would we ever agree on the semantics?)

Now, isn't it clear that the Web, even though it is the largest information system by far, has a very limited programming model unsuitable for building enterprise class information systems?

It's clear to me is that the Web isn't an architecture for building an localized information system. You need several styles needed for any of such things.

The Web's benefit is as a "system of systems" architecture; it's a way of dealing with issues of governance when you span systems and agencies. I don't think that it's the final approach -- there will be successors. But I think aspects of it, such as the URI, probably will endure for decades. And I think enterprises would benefit greatly if they adopted, at the very least, URIs more prevalently.

So please note, I don't think 'the answer to IT woes' is to rebuild your enterprise systems as "web applications" with RESTful resources, full stop. I think that's hopelessly naive. The point, again, is that there are many architectural styles, and the web's exhibits properties that I think would benefit most enterprises for what they've been usually been using WS-* for, in my experience. But it's far from the only architectural style that the enterprise uses or will use in the future.

Stu, the way I view it, when you say the "Web is enough" you are simply rejecting the all the vendors (small or large, from Redmond to Waldorf) as well as the standards and technologies they produce, rather than convincingly provide a solution to the problem.

I never said the Web is enough. I said that it's probably the way forward - a foundation for a next generation of sharing information at scale. I think any successors in the near future will have to start with the Web and drop or change the parts that don't work (for their domain) rather than inventing something from scratch. For example, turning the Web from client/server to event-based is an obvious one. Though we'd have to be careful not to lose the RESTful benefits; asynchronous messaging tends to lead to more complicated and less scalable systems when designed in an amateurish way.

As for rejecting all the vendors' standards and technologies -- considering the level of RESTful activity going on in Redmond and Armonk these days, I'd say they're (on some levels) doing a good enough job on their own of at least hedging their own efforts. Most standards these days are not driven by good technology, they're driven by power politics. There's only so many times you can compromise for political reasons before you just want to duck the whole system and try something new.

So, how am I providing a solution to the problem of making the Web more suitable for enterprise systems? Firstly, this isn't something one can solve in a blog. It requires working software and specifications. I don't think a Grand Unified Theory of Everything approach, the approach I worry we sometimes were moving towards in our "chats" about BPM and interactions and choreography in early 2008, is productive -- it just leads to frustration and communication breakdowns. All I can do is hope to catalyze the growth of knowledge in this area. To that end:

(a) I would like to dedicate time to frame discussions on some key problems; there's a long-awaited article I've been sitting on to discuss an architecture for business process & interactions on top of REST, which will be useful food for thought (it's a big topic, so I've had to revisit it several times, to the chagrin of my very patient editor, who's likely given up by now [grin]);

(b) There's a long dormant article or blog post on semantics of action that I'd like to get around to (about why REST or ARRESTED is a suitable foundation for enterprise systems-of-systems interactions) ;

(c) What takes most of my time up is that I'm now the technology leader at a startup in a growing industry; I'm going to do what I can to drive forward approaches that I think are productive & open. You can bet that the Web's architecture will be a part (but not anywhere near the only part) of that effort.