October 26, 2007

But sometimes things do change

End of an Era...

"So after the show, the SOA advocates shuffled off to meet with Michael and I for a spot of coffee and confrontation.... During a relaxed and wide ranging conversation exploring resource oriented versus message based architectures, I suddenly realised, there was no argument anymore. Getting all those silly vendors to agree on “something, anything” was the battle, but going forward, it’s obvious the Web has won. All we have to do now is to help those pour souls still trapped in Middleware hell to walk into the light and pass the bovril and blankets. If you know someone still slipping around on the SOAP, don’t hate them, just warn them the longer they continue the sillier they look. They deserve your sympathy, not hate. Just give them lots of hugs!"

Thanks, Paul, this made my day.

Hugs, not hate, is the approach I've been taking since mid-2006 within BEA on this topic, with some success, at least in consulting. Though a whole division of BEA figured this out on their own a long while ago.

Posted by stu at 06:09 PM

The more things change...

Technology moves quickly? bah!

From: Stuart Charlton <stuartcharlton@hotmail.com>
Subject: Re: [dist-obj] Was Souls, Now S2S, P2P.. Web Services
Date: February 28, 2001 5:26:53 PM EST (CA)
To: Mark Baker <distobj@acm.org>
Cc: dist-obj@distributedcoalition.org

> Stu!

Mark!

> It looks to me like you're only considering the RPC use of SOAP. SOAP
> is so much more than RPC, but also so much less than a lot of people
> think. For instance, SOAP defines no application semantics. It relies
> on an application protocol to do that, such as HTTP. Almost all of the
> benefit that is attributed to SOAP in the press, is made possible by
> HTTP. In fact, you don't even need SOAP. All it adds is;

Yes. I did an "XML messaging without SOAP" project back in September when I
was running the "new hires" training program for a Wall Street bank.

We really didn't want to use a CORBA/COM bridge to talk have VB talk to our
Enterprise JavaBeans. Most of the "ease of development" came out of HTTP.
We created a generic-data DTD (simplified XML-Data), and a simple invocation
DTD and would basically call / query our beans using a very thin servlet
that did reflective calls on the beans. We put an IE component inside our
VB application to render our data using XSL.

Didn't need SOAP then, don't (really) need it now. :) But it seems to be
the direction everyone's moving in...

[snip]

Cheers
Stu

Posted by stu at 04:55 PM

October 22, 2007

The Web: Distributed Objects Realized!

Here are the slides from me and Mark Baker's half-day OOPSLA tutorial.

It's a slightly different twist on the "why and when is REST useful?" question.

Here's a motivating paper we also wrote -- it's on GooDocs but I've provided a local link for now.

Posted by stu at 09:37 AM

October 18, 2007

On Chaos in IT

Steve Jones:


...in part Stefan has a good point, namely that IT systems currently suck. But where I'd disagree is that the goal of IT should be to create such a chaotic system with so little governance and control. This is one challenge I have when people talk about applying Web principles to the enterprise, it misses out on a fundamental difference between businesses and the internet. Namely that of compulsion.

I think the point is that the Web is an architecture of participation, wherein we set up constraints to enable value by converging on a small number of strong rules, even if you diverge in many other aspects. And even in businesses, people disagree on issues, but still need to work together.

People have often referred enterprise architecture as "city planning", primarily because the business does not speak with one voice -- it is very typically decentralized. Weill & Ross' excellent book IT Governance discusses the variety of governance styles, and very few are "Monarchy" or "Duopoly", wherein the compulsory standards will be likely adhered to. "Feudal" seems to be the dysfunctional norm, where each profit center doing what it wants, and "Federal" as an acceptable, if politicized, alternative.

The other note, similar to what John Hagel & John Seeley Brown have been saying, channeling Drucker, is that the borders of the enterprise are dissolving, and interaction is occurring outside of its walls at an increasing rate. Why adopt an architecture that is inwardly focused, when all results, most opportunities, and threats are on the outside of the legal fiction of the organization?

I guess the core question is whether the large organization *fundamentally* tends towards convergence or divergence in nature. If it's divergent, you're rarely going to get broad compulsory agreement on many domains of value, and even when you do, you need to invest heavily to maintain that agreement.

The alternative is to adopt a collaborative agreement, wherein the participants have incentives to join, and the benefits are emergent. With the web, the assumed incentive is exchanging and transforming an ever increasing amount of wildly diverse information.

Of course this is not the end of history, but I think it's a step towards better IT.

Posted by stu at 10:30 AM

October 16, 2007

Rechtin

A paraphrase from the late, great, Eberhardt Rechtin:

"Most architectures are the products of deliberate and centrally controlled development efforts. There was an identifiable client or customer (singular or plural), clearly identifiable builders, and users. The role of the architect existed, even if it was hard to trace to a particular individual or organization. The system was the result of a deliberate value judgment by the client and existed under the control of the client.

However , many systems are not under central control, either in their conception, their development, or their operation. The Internet is the canonical example, but many others exist, including electrical power systems, multinational defense systems, joint military operations, and intelligent transportation systems. These systems are all collaborative in the sense that they are assembled and operate through the voluntary choices of the participants, not through the dictates of an individual client. These systems are built and operated only through a collaborative process."

"In a collaborative system, the greatest leverage in system architecting is at the interfaces. The greatest dangers are also at the interfaces. When the components of a system are highly independent, operationally and managerially, the architecture of the system IS the interfaces. The architect is trying to create emergent capability. The emergent capability is the whole point of the system; but, the architect may only be able to influence the interfaces among the nearly independent parts. The components are outside the scope of and control of the architect of the whole."

"Virtual collaborative systems lack both a central management authority and centrally agreed-upon purposes. Large-scale behavior emerges, and may be desirable, but the overall system must rely upon relatively invisable mechanisms to maintain it."

"The Web is even more [of a distributed collaborative system] than the Internet in that no agency ever exerted direct central control, except at the earliest stages. Control has been exerted only through the publication of standards for resource naming, navigation, and document structure. Although essentially just by social agreement, major decisions about Web architecture filter through very few people. Web sites choose to obey the standards or not, at their own discretion. The system is controlled by the forces that make cooperation and compliance to the core standards desirable. The standards do not evolve in a controlled way, rather they emerge from the market success of various innovators. Moreover the purposes the system fulfills are dynamic and change at the whim of the users."

"A collaboration is a network good; the more of it there is the better. Minimize entrance costs and provide clear conformance criteria."

Posted by stu at 01:39 AM

October 11, 2007

Planned vs. Seredipitous Reuse

One problem with SOA is that it is very "heavy", with a partial focus, like CBD before it, on planned reuse.

In some industries, planned "product line" reuse has been shown to work, such as with car platforms. It's also appropriate for very general purpose programming libraries, etc., and could also be appropriate in software (there's a fair amount of "software product lines" literature out there).

From this viewpoint, "build it and maybe people will use it later" is a bad thing. SOA proponents really dislike this approach, where one exposes thousands of services in hopes of serendipity -- because it never actually happens.

Yet, on the Web, we do this all the time. The Web architecture is all about serendipity, and letting a thousand information flowers bloom, regardless of whether it serves some greater, over arching, aligned need. We expose resources based on use, but the constraints on the architecture enables reuse without planning. Serendipity seems to result from good linking habits, stable URIs, a clear indication of the meaning of a particular resource, and good search algorithms to harvest & rank this meaning.

This difference is one major hurdle to overcome if we are to unify these two important schools of thought, and build better information systems out of it.

Posted by stu at 02:08 PM

October 04, 2007

Zimbra

I missed this a few weeks back... Zimbra was acquired by Yahoo!. Zimbra's CTO, Scott Dietzen, was BEA's former CTO.

This is great news for a great company. I can only hope this will make their technology more mainstream. We need competitors to Exchange & Outlook, dammit.

Posted by stu at 11:29 PM

Dynamo

A detailed technical paper on Amazon's advanced key-value storage system. A great practical example of the CAP theorem in action, wherein we sacrifice some consistency for greater availability and partition tolerance.

Posted by stu at 10:51 PM

on ESBs and disposable software

Another Dev2Dev blog post, which I should use more often when the word "BEA" appears here...

Posted by stu at 10:38 AM

On effective architecture

Sometimes we keep talking past each other in these debates about architecture.... SJ claims that REST isn't an architectural style after all, but rather a design pattern. And in the comments, client-server isn't a style either.

Well I've been known to use "architectural pattern" as a synonym for style, in that it is a set of interactions and/or constraints that provide particular benefits. But it's not about implementation mechanics.

IF we want to play the definition game, I would not trust Wikipedia. Here's Clements, Bass, Kazman, and Northrop -- pretty reputable people in the software field -- describing architectures & styles, in p25 of their book:


For example, client-server is a common architectural pattern. Client and server are two element types, and their coordination is described in terms of the protocol that the server uses to communicate with each of its clients. Use of the term client-server implies only that multiple clients exist; the clients themselves are not identified, and there is no discussion of what functionality, other than implementation of the protocols, has been assigned to any of the clients or to the server. Countless archtiectures are of the client-server pattern... but they are different from each other.

An architectural pattern is not an architecture, then, but it still conveys a useful image of the system -- it imposes useful constraints on the architecture, and in turn, on the system....

...Choosing an architectural pattern is often the architect's first major design choice. The term architectural style has also been widely used to describe the same concept.

This sort of thing applies to other fields. In organizational design, we also have a number of patterns with a variety of benefits: functional, geographic, matrix, customer segmented, etc.

This got me thinking about a talk I gave at BEA's Worldwide SOA Practice Meeting in Boston last week. It was about "alignment vs. effectiveness" in architecture, and dealt directly with this topic. The MIT Sloan article Avoiding the Alignment Trap in IT was the inspiration, along with elements of Roy's recent presentation. The reaction was very positive, but a few didn't get it (though admittedly I plowed through the preso in 1 hour) or didn't agree (though they didn't say why).

Anyway, here's the story:

SOA is a way of describing architecture. I am not talking about Business Services Architecture here, which strikes me as an attempt of rejigging organizational design theory with technology concepts -- something that seems valuable but still lacks clarity.

I'm talking about describing arrangement of software. With SOA, instead of describing an architecture in terms of components, connectors, data elements, etc., I describe it in terms of interfaces, implementations, and contracts, which includes descriptions all of the data elements.

And here is where I believe the disconnect lies: SOA principles have everything to do with alignment of IT assets with the organization. And for good reason: we've often ignored business needs in favour of technical justifications, and SOA is more about a framework for thinking about "what to deploy where" -- alignment -- then helping you to arrange the interactions in an effective way. The problem is, that we seem to have forgotten about effectiveness!

For example, REST doesn't tell you how to build a web site. It doesn't tell you what should link to what, why, and when. That's what a lot of the SOA work has been about: of your candidate services, which should be deployed, and where, and how do their contracts interrelate? On the other hand, if your business requirements need a certain level of scale, interoperability, etc., then a RESTful style would be a class of SOA that would be very beneficial to your problem.


See, effectiveness, which is how well an architecture will perform in practice, given the constraints & properties you apply to it, is where the many years of a systems architect's experience comes in. This is an understanding of how certain interactions have certain tradeoffs associated with them.


Another view of the problem: SOA folks often suffer from an ailment I call "producer-itis", meaning that they focus from the service producer's vantage point. The consumer's vantage point -- those that will actually use the services, whether humans or other services -- is often secondary. Now, think tanks such as ZapThink and CBDI have long advocated "twin track analysis", where producer and consumer considerations are both taken into account, and indeed, this might be the biggest drive for SOA in the first place! But many SOA analysts have embedded the "WSDL metamodel" into their brain, which is of the "if you build it they will come" variety of architecture -- I deploy an interface, I register it, you use it. Ignoring that the classes of consumers are likely to be way more heterogeneous and large scale than the producers, if your SOA initiative is successful. ;-)

The business requirements, for example, may require a particular interaction pattern (or "message exchange pattern", ugh) between services today, but that says nothing about the properties I gain or lose from such an exchange. Or what happens when the business changes. With SOA, we seem to have devolved in to describing architecture as a passive observer writing down observational behaviour for a contract, instead of influencing interactions based on how effective they will be in practice.

Without appropriate application of architectural styles, we risk becoming fully aligned, but unable to get anything done -- the alignment trap.

This isn't implementation to me. This seems to be about two schools of thought on form - one that contorts itself to remain aligned to the business, and one that understands, at a more abstract level, the nature and tradeoffs of interactions between elements. I think they're both relevant, but clearly we seem to talk past each other because they represent different value systems.

Posted by stu at 09:51 AM

The new music biz model rears its head again

I recall since the Napster days that the effective way to monetize music production (and maybe software production) in the future will be a capital market, where the dividends are "more software or music from this artist". This eliminates the need to monetize every single copy, something nearly impossible given how prevalent digital copying is (and should be).

Now, Radiohead is saying the price of their new album, digital or disc box, "is up to you"


Of course, this is only for the next few months, before traditional distribution kicks in. But it is telling.

What will be interesting here is how the whole free rider issue plays out. There's the traditional free riders, those who copy off of Torrents, Newsgroups, or a P2P network. The question is whether a new class of free riders will emerge to pay very little for a legitimate copy.

What's missing here, I think, are market indicators -- how much are other people paying? What's the bid spread? Prices will tend to converge, then.

Posted by stu at 09:05 AM