16 October 2008 - 19:31REST and WS - part 5

One of the best presentations that I have seen lately is Mark Little talking about REST and WS at QCon London 2008, or rather about the differences between the uniform interfaces and the specific interface. Mark Little gets right at the bottom of the differences between REST and WS, and these differences are not all differences between architectural styles, but rather differences in organizations.

One of the best interpretations of the differences in REST vs WS that I have seen is the one that Mark makes in this presentation and that I will try to reproduce below is the division of tasks of mapping a message sent to an entity (could be a service, could be an URL) and the component/processor within that entity that will process it: when you send a message to a specific interface you will be specifying the method and the component that will process that message, so the message-to-processor mapping is done by the client. Whereas when you send a message to a specific interface you need to attach some meta-data to that message which will be used by the entity implementing the specific interface for mapping that message to its appropriate processor, therefore the message-to-processor mapping is done by the entity.

The message-to-processor mapping is externalized in the case of specific interfaces while the uniform interface internalizes it. These different mappings come with different transaction costs: in the case of the external mapping the costs are synchronizing the client with changes in the specific interface while in the case of the internal mapping the costs are adapting the uniform interface’s mapper to changes in the meta-data (a new type of messages should be mapped to a new processor).
Now, if we apply economics to the task of message-to-processor mapping we would start to get some interesting results. According to Coase’s theorem you can externalize an activity if the transaction costs for that carrying that activity are low enough and if the transaction costs are high then that activity is better internalized. It would follow that when the transaction costs of this mapping are low you can externalize it via a specific interface, otherwise you would be better off to internalize it via an uniform interface. 
The transaction costs of the specific interface are the synchronization of distributed clients and this could be resolved either thru versioning, or by not modifying the specific interface or by standards. If you can keep these costs down it follows that you can delegate the message-to-processor mapping efficiently to the client else you are better off with a uniform interface. Keeping these costs down also implies that the number of methods that get exposed is kept down in order to avoid the number of changes (more methods, more opportunities for changing them). Generally speaking interaction via specific interfaces is best limited to a few coarse-grained components, between a few participants and with components which do not change very often.
The above synchronization costs run counter to scale in number of clients: as the number of clients starts to grow you are starting to have problems synchronizing all of them and eventually the transaction costs of using the specific interface will become so high, it will not make sense to use it. One corollary of this is that the uniform interface is far better at the scale of the internet and this is probably the main reason why it is used at this scale.
The above will provide a pretty good guide to an architect trying to figure out whether to use a specific interface or an uniform one because their use is not driven by technology, but rather by the organization(s) within which they are employed.

Another interesting point in his presentation was the effort for creating a REST-ful standard for carrying out distributed transactions and the need for agreement between multiple parties. As I was saying in a previous post about REST and WS the differences between these 2 camps are more of organizational nature than anything else. Mark touched upon the fact that even if the need for carrying out distributed transactions via REST exists there is still no standard for doing so, that the REST community seems pretty much happy with implementing low-level infrastructure and not with higher-level, and higher-value, components. I find this a pity and I have the impression that this need will be fulfilled only it raises to a level at which major corporations will find profitable for getting involved. The REST community appears simply way too fragmented to be able to carry out this effort on its own…

Anyway, go ahead and watch the presentation, it is the most thoughtful presentation on distributed computing that I have seen in a while.

No Comments | Tags: Econo-computing, Favorites, Management

28 August 2008 - 13:13Technical debt

You are probably familiar with Martin Fowler’s Technical Debt, a metaphor around the idea that doing things the quick-and-dirty way creates bad code, bad code which can be viewed as a debt which needs to be paid later in installments (the principal is the re-factoring of the bad code and the installments are the extra effort that this bad code forces on development). The concept was originally set-up by Ward Cunninghan.

Technical Debt is pretty much different from ordinary, everyday debt primarily because there is no creditor and no maturity date. Also, unlike regular debt, it is very hard to quantify. Technical Debt seems a bastardized version of regular debt in the sense that some development costs are identified as the principal (the re-factoring of the bad code) and some as the interest (the hacks used for dealing with this bad code) and it is built by taking some concepts out of the ordinary debt concept as the author sees appropriate in order to underline some development costs.
At the same time Tech Debt is defined exclusively from the point of view of the borrower who has to worry about paying the principal and the interest leaving out other actors (even though once you take the creditor out of the picture you have to wonder why you have to pay this debt at all ;-)). But, like Martin Fowler said at the beginning of his article, this is a metaphor, which leaves a lot of room for stretching various concepts. Fair enough.

However, Technical Debt seems to have been taken up and spawned quite a few siblings as you can see in this infoq article: liquid assets, moral hazard, fertile assets (???) etc…, which are either unfortunate enough to bear little or no resemblance to the original economic concepts they try to refer to or unfortunate enough to not refer to anything (fertile assets stands out as a prime example).
Let’s pick for example “Liquid Asset” which is defined this way: Perhaps the term “technical debt” focuses us on the wrong things; maybe focusing on the converse, on the investment side of things might be more effective.
First of all, a real-world definition: a liquid asset is an asset for which transaction costs are lower and for which there exists a market in which this asset can be sold and bought for a reasonable fee. Try to look into the above definition (or in this link which is provided next to it in the infoq example) and if you can spot the asset that is referred to, as well as the market in which this asset is sold and bought and the fees for these transactions please let me know.
Second, switching terms in order to motivate people displays how shallow these concepts are to begin with: if Technical Debt would be a concept that relates to somethings tangible in the real world, if it would manage to encapsulate a real problem or some real costs then Tech Debt should should not be swept under the rug for fear of under-mining the developers’ morale, but rather dealt with up-front because in doing so you would solve a real problem. The fact that Tech Debt is used the way this way points to the fact that it is so vaguely defined that few people can extract some somthing valuable out of it and that it can be stretched in any direction you want to.

What I see when I am reading the infoq article mentioned above are the first steps in the attempt to “marry” 2 fields which are pretty different one from another: economics and writing code and the excesses which appear when interest start to pick up in some vague concept (excesses similar to the introduction of the SOA concept). There is a desire to bring IT development under control and one way to look at it is to define costs and benefits associated with various actions and then apply economics to these costs and benefits in order to write code more efficiently. I don’t know where this will lead because we are at the beginning of exploring this.

As far as I am concerned I have developed an interest in economics a while ago and I read on it as much as I can. I also tried to “marry” the domains of writing code with and of economics, I even have an Econo-computing category, but I find it pretty hard. So far the only economic concept I found that can be applied pretty well to software development are transaction costs because they actually encapsulate pretty well some very real costs which arise when various entities are interacting one with another.

I will be following this Econo-computing field with a lot of interest and, who knows, maybe the concepts in this field will actually start to relate to something tangigle in the real world and will improve developer productivity.

No Comments | Tags: Development, Econo-computing

21 January 2008 - 21:36Communication problems in dynamic languages

I was reading this post on manageability.org which is hi-liting some of the problems with large projects in dynamic languages. I am not familiar with these problems, never having to code in a dynamic language on a large-scale project, but I am currently working on a large scale project in Java with many distributed teams and in such an environment it is crucial that the code created by different teams is put together fast and reliably. Putting the code from various sources together fast is necessary for efficient collaboration between teams.

Probably the number one problem of distributed teams working on the same project is to make sure that the code doesn’t contain references to methods which don’t exist or which have changed. Statically-typed languages solve this problem by their own definition at compile time. Dynamically-typed languages probably have some solutions to this problem (I heard some people documenting the Ruby code that they write), but the biggest drawback of them is that this type of solutions are human-powered and not automated, an automated solution would be something similar to a compiler inspecting your code for invalid method calls. I would say that the solving code-conflicts that arise in distributed teams are more expensive in dynamic languages than in static languages, therefore when choosing a dynamic language you should probably stay away from coding in a distributed development environment (*).

One problem that I see with Ruby and all those elegant languages is that they are not very expressive to a large audience. They encourage brevity, but I find that they encourage it at the expense of readability and this narrows their audience (this is why I find it unlikely that Ruby will break out of a small elite). A developer working on an application needs to understand pretty fast what a particular method does rather than marvel at the “beauty” of that particular piece of code. I find that the information stored in Ruby code is harder to extract than the same information stored in Java code. It is “beautiful” (**), but also inefficient and it is more expensive to work with it.

I would say that the biggest challenge using dynamic languages revolves around collaboration and communication. Statically-typed languages put a smaller cost on communicating thru code, dynamic languages can achieve the same thru some external mechanism. While dynamic languages have more features, this freedom comes at the expense of collaboration. Evidence would seem to make this supposition true, most development in dynamic languages happens in small teams concentrated locally where the needs for communication are small.

* All the above being said I remember Martin Fowler talking about agile projects coded by distributed teams which did very well. He didn’t mention what language these teams are using, but given the fact that ThoughtWorks seems to favor Ruby I would assume that these distributed teams were using Ruby as well. It would be interesting to see the steps taken by them in order to be able to handle these communication problems.

** I personally find it brain-dead to talk about the “beauty” of code because this is not the purpose for which code is written. A statue should be beautiful because this is its purpose, to be enjoyed by who watches it, a piece of code’s purpose is to work as it is supposed to.

Later Edit: The communication and collaboration costs of dynamic languages will define the population of developers that use them and their usage within that population. I would not be too surprised is most of Ruby and Groovy usage will be restricted to small scripts which do not require a lot of collaboration while a small niche of very productive teams will be able to use them on a large scale.

2 Comments | Tags: Econo-computing, Management

28 December 2007 - 17:14Random notes on the economics of knowledge

The biggest cost associated with knowledge is learning: in order for one person to absorb the necessary knowledge so that it operates efficiently in a particular environment it is necessary for it to learn that knowledge. It is important to acknowledge this cost because the cost of operating in that environment without the necessary knowledge is typically greater: a person/team operating with insufficient knowledge has large management costs. Taken to extreme insufficient knowledge can lead to micro-management.

Disseminating knowledge is a process usually tied to the domain to which that knowledge is pertinent, to the organization dispersing the knowledge and the people that need to receive that knowledge. In this process you have to rely both on people as well as on systems in order to make it more efficient. Ideally, you would not swamp the persons with requests about knowledge, but rather put them in a position where they can guide people needing the knowledge to the appropriate knowledge resources.
Documentation is the primary vector of disseminating knowledge about a project*, it is the first point of contact between a person and that project and a point of contact to which someone will refer continously during the interaction with that project. It is important that we acknowledge this double role of documentation (initial point of contact and then continous point of contact) and that we structure it to accommodate both roles: a high-level view for initial contact and a lower-level view for displaying the internals of a project.

The knowledge about a domain “encapsulated” in one person depreciates slowly for a small period of time and then rapidly as more time passes with that person outside of the domain. For example, a person that is absent from a domain for a small period of time, during which the domain has changed, will be able to absorb the domain changes quickly, if it has been absent for a big period of time it may have problems getting back in shape. However, previous domain knowledge will help.

Storage and retrieval of knowledge is usually expensive. The more “fuzzy” the domain of that knowledge the more expensive it gets to retrieve good knowledge about it. In “fuzzy” domains you usually rely on people to get to good information. In “fuzzy” domains you will also need people in order to resolve conflicts in the interpretation of knowledge. It is important to identify these persons and set-up a communication channel with them.
The need for people in processing knowledge is probably one reason for the informal networks that get established in large organizations: sometimes it pays to know who knows something, sometimes it is the most efficient way to deal with knowledge. Getting to a person that knows something that you need to know is a process of discovery which will ideally be optimized.

Blogs can be used for delegating knowledge management to a person that is motivated to provide that knowledge continously. Bloggers are pretty attached to the subjects they blog about and are likely to follow these subjects and report on them efficiently (at least good bloggers should report efficiently). These reports constitute knowledge about the blogger’s subjects which are pretty precious. Another nice thing about blogs is that you can initiate conversations with their authors and get pointers from them, blogs act both as systems for disseminating knowledge and as people disseminating knowledge at the same time.
In an organization blogs could be used for finding persons that have knowledge about something and for contacting this person about that knowledge. We should be able to differentiate between blogs and to assign blogs to various knowledge resources.
I think there is a lot more to knowledge management in blogs than I can express right now, I will leave this for later.

The book Economics of Knowledge by Dominique Foray is a pretty good book on this subject.

* I will use the term project as the object to which knowledge applies.

No Comments | Tags: Econo-computing, Management

3 December 2007 - 18:29Transaction* costs in software development

Probably the most interesting thing that I read in quite a while is the paper “The Nature of the Firm” ** by Robert Coase which can be viewed here. Ronald Coase answers a pretty interesting question: why in some cases it makes sense to gather resources together to carry out an economic activity while in other cases it makes sense to delegate that altogether to an outside party.
The answer lays in the extra costs that lay in carrying out a transaction or, as they are known, in transaction costs. Let’s say that you want to have your own house. You have 2 choices: one is to sub-contract parts of building the house and orchestrate the activities of the companies that you have sub-contracted: the one that pours the foundation, the one that puts up the walls, the people that paint your house, etc… or you can delegate the operation to the external entity, the home builder. The costs of each choice are (at first sight) the following:
1) carrying out the orchestration on your own: paying for the company that pours the foundation, the one putting up the walls, the one putting up the roof, etc…
2) delegating this to the home builder and paying for all of the above (the home builder will have to pay the sub-contractors just like you do) + the profit of the home builder who will be doing the orchestration for you.
At first sight it appears that you should be doing all the stuff yourself and come off cheaper than by delegating this to the home builder. But apart from the costs of the subcontractors you also incur the costs of interacting with them: you need to inspect the work of each one because you don`t know how good that sub-contractor is, you need to look around for good subcontractors, you waste a lot of time negotiating with them, etc… No suprise that most people prefer to go with a builder***.

What basically The nature of the Firm says is that if the costs of carrying out the activity on your own are less than the costs of delegating this to a dedicated party you should carry out this activity on your own rather than delegate it. You can see Coase’s theorem at work in the current outsourcing/offshoring trend: the transaction costs for outsourcing some operations are so low to the fact that a corporation can delegate efficiently various tasks to outside partners.

So how would this apply to software development? I would say that transaction costs demarcate pretty well what needs to be grouped together and carried out by one entity (which assumes a specialization of some sort) and what needs to be delegated to a different party. Well, this is what is needed for architecting a project: you need to determine how to group various classes, packages, systems together so that they minimize the interactions between them. The interactions between systems (or classes or packages) can be thought of as transactions: one class calls the other to do some work just like a home-builder calls the carpenter to install the hard-wood floor. By looking at the transaction costs between various components of an application you could determine what transactions (method calls) should be consolidated into component that handles them and what transactions should be allowed to exist on their own.

I would say that one big cost associated with a method call (or call to a package/outside system) is extensive orchestration between various classes (a method that calls 15 other methods from 7 different classes). If you have a pattern of method calls repeating over your code-base you should probably try to take a step back and consolidate these calls into a component that will serve the purpose of that particular orchestration. This way you will create a specialized component to which you delegate this orchestration, rather than replicating it multiple times. Efficient specialization of the components will incur lower transaction costs because it will push down on the number of interactions between components. For this you need domain knowledge. While designing a system you should look for specialization and for ways to achieve this specialization and once this specialization achieved you should look for ways to interact with these specialized components efficiently. If this is done properly you will end up with a system in which the cost of interaction between various components is low.

Another big cost for interacting with a different system is knowledge about the other system and class. The more knowledge you need to have about an external entity the more costly it is for you to interact with that system: you will need to be kept up-to-date with changes in this knowledge, there is a ramp-up cost, you can handle only this much knowledge, etc… It is a good thing to minimize the knowledge required for interacting with an external entity, this way a component can interact more efficiently with other components.
One corollary would be that you need to externalize as little as possible from any component and try to keep most of its sub-components private. One side effect of having too much of a component available is that you run the danger of a component coupling to it unnecessarily.

One cost for interacting with an external system is API coupling: the fact that you need classes exported by that external system**** in order to connect to it. If your system shares classes with an external system you will need a way to synchronize the shared classes with the external system. This is done usually either by having the external system having different versions for its points of interaction or by having a mechanism that publishes classes used for interaction. Either alternative is pretty costly, so you may try to keep away from sharing classes with an external system (though sometimes it may be impossible).
Transaction costs can be applied in project management as well: you should look out complex interactions between teams and try to assign them to a resource best dedicated to handle them (this resource could be a team-member, a well-defined process, etc…). However, in this case you should avoid assigning this interaction to a real person because you will run into scarcity constraints (i.e. you will run out of people) very fast if you keep delegating to real people. This is why these interactions are probably best addressed to a process.

I think that developing a software project while keeping an eye on the transaction costs is a good approach.

Later Edit. Transaction costs are very useful in determining the interaction between large components (systems, services, applications) and they are probably best used for managing this interaction. When you design the interaction of various systems the interaction costs (which are transaction costs) should be kept to a minimum for that interaction to occur without many problems.

* In this post transaction will be used for its economic meaning: i.e. it will stand for an exchange between 2 parties.

** If I am not mistaken Ronald Coase received the Nobel prize in economics for his work on transaction costs and this paper.

*** Please note that there are people that actually build their own houses on their own. The vast majority of them are in the construction business or have connections in the construction business so their transaction costs are a lot lower that for the ordinary person.
On a different note, the orchestration needed for building a home (calling the people that put up the jeep-rock after the 2/4`s have been erected, calling the painter for painting the walls after the walls have been put up, etc…) is a very simple business process without any intrisic value and prone to copying. No wonder that the builder itself makes a pretty small margin while the lion`s share of the profits go to the entity which owns the land…

**** You are generally sharing classes with an external system if you interact with it via RMI or EJB. Interacting with an external system via WS-* or REST is a lighter coupling.

No Comments | Tags: Econo-computing, Favorites

15 October 2007 - 15:37Annotations used for deployment

I was reading this post from Bill Burke’s blog and I found myself shocked by the sheer number of annotations required for making a business object WS-BA compatible (if I understand well the post).
One thing that I don’t like about using annotations is that they tend to stay with the code. I think that these documentation provided by annotations should be outside of the code because in this case you are coupling your business component to an external concern: you are coupling your component to the WS specification.
What you get in Bill’s post is the first stage of annotation creep: you need to externalize your component in order to plug it into an operational environment and you do it thru an annotation. Next operational environment comes with a brand-new set of annotations. And so on…
I keep thinking that the best way to deal with all these annotations would be to sub-class the original interface and apply annotations to it. You will have quite a lot of problems: the classes which are mapped to the original interface will need to be sub-classed as well and their sub-classes mapped to the annotated interface. You will have problems when updating the original interface, you will find that you need to update all its annotated sub-interfaces. And a whole other set of problems.
You feel like you almost need to define inheritance in annotations: an interface would inherit the methods from an interface and the annotations from a different place and you would work with this new interface properly annotated for the operational environment. My opinion is that annotations, like any other type of static, hard-to-change documentation, have serious limitations when it comes to exposing a class to many, varied environments and when you are using it for documenting operational concerns. No wonder so many people prefer XML…

BTW, this post uses Bill Burke’s post only for displaying an example of annotation-creep.

Later edit: If you think about the way annotations are currently used (putting them all over your code base in order to insert a component into various frameworks) they can be thought of as some sort of cross-cutting concerns (they are cross-cutting to a certain extent). I wonder if using something like an aspect would not do a better job of expressing the various concerns that annotations are currently used for. Aspects are orthogonal to the code-base at the same time, a pretty neat thing.
Wouldn’t it make sense to WS-BA-enable your whole application by specifying a point-cut and then applying annotations against this point-cut? It would be a lot easier and it would de-couple your code-base from external concerns. I am not sure if it is doable (I could think of a solution, but it is not pretty).
I think that the way annotations are currently used is not 100% OK and that it poses some maintainability problems in the future. Putting information into annotations is partly a response to XML-creep (as Bill Burke pointed out in the comments) but it is a solution far from perfect. What we are dealing with here is the need to add some information to a component in order to plug it into a framework. Annotations, as well as XML, are not doing a good job because they address this problem (plugging the component into the framework) at the individual level, at the component level. What we really need is a mechanism that would let you plug a whole set of components into a framework easily (I suggested using point-cuts).

Later Edit 2.0: I think that adding querying capabilities at the code-base level (similar to AspectJ`s point-cut language) along with a mechanism to interpret these queries into Java would be the killer feature that Java lovers wait for. It would create a whole new way of developing: for example AOP would be implemented by joining point-cuts to an interception mechanism. A lot of the current mappings (in XML or in annotations) could be solved by defining a point-cut and then a mechanism to interpret this point cut. In the above case you would define a point-cut of your components that you want to WS-BA-enable and a mechanism for working with the components returned by this point-cut. Embedding a framework into an application would consist of creating a point-cut and then passing it to the framework in order to process it.
This would be application-level introspection and reflection and it would do away with both annotation-creep and XML-creep. I can`t dwell too much on it cause I gotta go to sleep.

Later Edit 3: In this post Bill Burke argues about using meta-annotation in order to de-couple your code-base from framework specific annotations. Adding a level of indirection between framework-specific annotations will free your application from framework-specific dependencies, this is for sure, but you will still tie your code to a representation (the representation specified by your annotations that serve as end-points for the original framework specific annotations). I am not sure that this is desired. I still think that your code-base should not be polluted by any external concerns.
It is interesting to see what will happen:
1) Will the world continue with annotation-creep and XML-creep for plugging their applications into various frameworks?
2) Will the world settle for a mid-way solution, like the meta-annotation solution suggested by Bill for the same problem?
3) Will the world try to solve this thru a different design?
4) Will the world decide that frame-work plugging is so costly that we might as well migrate to a different language/platform in order to reduce these costs?
It is all a question of supply and demand basic economics: plugging an application into a framework is currently a pretty costly operation. If the demand for this operation rises significantly (the number of times an application is plugged into a framework increases) it would make sense to make an investment (new methodology, new framework, etc…) in order to slash the costs. Either you make the investment or your language/platform becomes so costly so that it will prove cost-effective to move to a different language/platform.

10 Comments | Tags: Development, Econo-computing, Favorites

28 June 2007 - 2:44When to use business-logic aspects?

I was reading this post on infoq about applying AOP for implementing business logic transparently. It was a pretty good example (actually the author displayed a very interesting way of consolidating a series of different business concerns into a general advice using a Map of Command objects), but what caught my attention was a comment at the end of the post:
It always surprises me to see how much code people actually place within the AOP advice creating an architectural dependency on AOP that makes it impossible to reuse in another context such as an explicit programmatic interface. I personally prefer to use AOP (or interceptors) as bridges or adaptors from one domain/layer/tier to another domain/layer/tier. I normally start with creating the underlying system divorced of AOP concerns and then add AOP into the mix to alleviate the coding effort and simplifying the programming model for typical extension use cases. I always ensure that it is possible to use an particular feature without the presence of a particular framework or instrumentation runtime.

The reader had a pretty good point: you can use decorators* for implementing this business logic. So when would you use AOP? I would use for implementing business logic concerns if they are cross-cutting concerns, if they apply widely across the code-base. In the above example the advice was a notification module that was sending an email when the user was registering. If you would have needed to send a notification when the user was purchasing an item, when an item was shipped to the user, etc… you would have a cross-cutting concern (a business logic concern) that needs to be applied transparently to the application. Another example when I would use AOP is for transient concerns, for concerns that should not modify the code base but that need to be implemented.
Business logic advices come with management costs (you have a piece of business logic detached from your code-base, you have to manage that). Implementing interceptors thru decorators come with coding costs (you usually need to create a new class). One should look at these costs and determine which approach is more cost-effective. As always, use AOP only for binding components together, the business logic advice should contain no business logic at all, but simply logic for binding a business logic bean (the notification module in the above example) to a different bean. I am not sure if the example followed this approach, though.

* AOP can be thought of as decoratoring unleashed, an advice applied to a class is a decorator.

No Comments | Tags: AOP, Econo-computing

20 June 2007 - 14:11Econo-computing

It looks like there are other people trying to use economic concepts for describing software development. Pretty cool…

No Comments | Tags: Econo-computing

15 May 2007 - 20:44What used to drive IT change

I was watching this rather boring infoq presentation on SOA when Gregor Hohpe, the guy giving the presentation, asked this question:
“We went from client-server development to thin client and now to services. The CIO, the guy paying for your IT implementations should ask you: What is next?”
Gregor Hohpe went on, but it left behind a pretty interesting question: why did we move from client-server to thin client and next to services? To be honest with you I don’t think the proper answer could possibly come from IT person, because I think that IT has been changed (at least till now) by the the economics of hardware rather than anything else.

I get the impression that client-server started to become main stream once the price for PCs started to drop and it became worth-while to start using these PCs rather than dumb-terminals. The alternative to rich-clients back then was ASCII-only dumb-terminals, it was obvious that the work-force would become a lot more productive if they switched to these rich-clients. I wonder why the rich-client was chosen rather than the thin client in the post-dumb-terminal IT environment. My impression is that this was done in order to minimize the traffic between the client and the server, at those speeds chatty applications were not possible.
Once the PCs prices started to drop the next barrier was the network. This took a while to go down, but once reasonable network speeds could be achieved this opened the door to a new concept: that of the application stored in a central place and of thin clients whose functionality it making requests to this application. One advantage to this approach is that by consolidating the application in a central place it obviates the versioning hell lived thru during the previous period. This was first massively used for pushing new applications (such as Amazon.com, eBay.com, yahoo.com, etc…) thru the web to consumers. Another advantage to the thin client was that it lowered the barrier for accessing an application by not requiring the customer to do anything (no download, no clumsy set-up process, no clumsy removal process, etc…). One potential problem with the rich client is the PC’s real-estate: you can fit and efficiently manage only that many applications on your PC, this puts a barrier of entry on the applications that can be run on a PC (how many would have installed an Amazon-type book browsing application on their PCs in the 90’s?). With the thin client this barrier goes away. I would argue that the drop in telecommunications costs drove the thin-client model and the economics behind it.
As telecommunications costs continued to drop the applications started becoming more and more chatty, leading to the point where the interactions behind them had to be managed efficiently. SOA comes to the rescue. End of story.

It appears that events in the CONSUMER’s market actually drove the ENTERPRISE IT environment. I guess the effects the consumer’s market has on the price of various computing devices (by effectively turning them into commodities) creates cost-savings which are recognized and used by enterprise IT. It would be interesting to muse over what the future factors of change for IT in an environment that has already seen massive price decreases. With rock bottom prices both in hardware and in telecommunications the next thing to decrease in price is probably software itself (and the open-source community is busy working on that).

There are a lot of complains that all these migrations (which are true paradigms shifts) are costing the companies lots of money. I agree with this, but you should consider that they also save a lot because otherwise nobody would do it. My .02$.

P.S. Another thing that I see happening right now is the re-emergence of the rich-client model. I see this as being correlated to the emergence of the mobile devices (another event in the CONSUMER electronics space) and their differences (reduced bandwidth and, more importantly, a very different interaction with the user) rather than recognizing that the rich-client model has any value.

P.P.S. I wrote this post in a hurry. It probably shows…

Later edit: If my assumption that hardware price changes due to demand in the consumer electronics space drove enterprise IT is correct it would be interesting to see what would be the next drivers. Changes in the price of software is a possible driver. It would be possible that changes in IT would be done on “merit”: i.e. a certain paradigm would be implemented because it is a more efficient way to write software rather than because it is related to a drop in the price of a certain hardware platform.

No Comments | Tags: Econo-computing, Miscellaneous

7 May 2007 - 14:53The demise/change of the template design pattern?

You are probably familiar with the template design pattern. It lets you define a life-cycle and declare functionality unknown at design time in abstract methods, effectively delegating this functionality to the sub-classes. While initially described in C++ terms in the famous Go4 book it is widely used in Java applications (Spring uses it heavily for managing connections of all sorts: JDBC, JMS, etc…)

In Java it is a pretty limiting pattern: due to the single inheritance rule you cannot have the sub-class of a template (let’s call it “templated” class) extend multiple super-classes. Sometimes this is pretty limiting, I find it. One work-around to this problem is to change the template class so that it is not abstract anymore and declare all the abstract methods in an interface (the “life-cycle” interface) which the template class is using as a member variable. Then the interface would be implemented by a “templated” class which doesn’t need to sub-class the template class, which gives the “templated” class more freedom. The template class will be populated at run-time with the appropriate interface implementation thru an IoC mechanism. I find that the life-cycle defined in the template class can be extended as well by defining a sub-class of the “life-cycle” interface which could extend its contract. This way you can define more flexible life-cycles. (You can do the same thing with the traditional template design pattern as well but I find it to be harder).
I find that this arrangement pretty much does away with the template design pattern and makes your application more flexible by eliminating the need for sub-classing.

One interesting question would be why the above approach (the interface-flavored template) has not been used before and the template design patten has been used extensively? I have 2 answers to this:
1) The template design pattern pretty much casts in stone your life-cycle. Once defined the life-cycle can be changed only at great costs, and some times the stability given by this inflexibility is desired.
2) The drop in the cost of interface-to-implementation mapping with the introduction of IoC. Before IoC interface-to-implementation mappings were usually constructed with tons of factories (another rather inflexible design pattern) and with tons of overhead. With IoC the interface-to-implementation costs plummeted weakening the case for the template design pattern.

It would be interesting to see how the template design pattern will evolve. Ironically enough it is a design principle championed by the architects of the most popular IoC (Spring) so there is some backing for it. It gives some stability to the architecture by making its extension harder, this could be desired in some applications. The flexibility generated by the interface-flavored template comes with some management costs: you need to manage the relationships that will be spawned because of the use of this method.
Time will tell…

P.S.: One key difference between the “old-style” template and the interface-flavored template is the inheritance issue: in the old-style template you export the life-cycle thru inheriting from the template class (and you have only one chance at inheriting a class in Java) while in the interface-flavored one you export the life-cycle by defining it in an interface (which can be inherit from multiple sources). The “templated” class will not have a chance to inherit from another class in the old-style template, while it will be able to extend another one in the interface template. I think that because of this big difference the old-style template doesn’t scale well: its mechanism for exporting the life-cycle (sub-classing) cannot be re-used, I have not spent too much time thinking about it.
Another difference between the 2 approaches is that the ‘original’ template pattern puts together the lifecycle definition (the abstract methods) and the lifecycle execution (the code calling the abstract methods) in one place: the template class. The interface method splits this in 2 parts: the life-cycle definition is in the interface and the lifecycle execution is in a class using the lifecycle interface. Sometimes you may need the de-coupling given by the interface template pattern.

2 Comments | Tags: Development, Econo-computing, Favorites