I’m not the first person to write about the transformation of IT. It is pretty obvious when you look out there that you have today is not what you had 5 years ago, Microsoft, Oracle, IBM, HP, Apple, IT’s blue chips have changed dramatically. They are no longer pure tech-players: IBM moved into services, Microsoft and Apple moved into entertainment, Oracle is moving into HR and retail, etc… Sun is one of the last pure-tech players and, sadly, has a lot of problems.
Commodization of software infrastructure is the common theme of this transformation. Databases, application servers, operating systems, various desktop programs, even computer chips design or message-oriented-middleware infrastructure are starting to cost absolutely nothing. Right now you can get a pretty good computer (hardware + OS + applications) for close to 500$. The big IT behemoths have sensed this for a while and are starting to try different other business models, their former businesses turning into money-losing propositions.
Today it looks like that you cannot expect anymore to make money handling information (storing, querying, sharing, securing, etc…) except in very small niches (such as semantic queries, NLP, etc…). Is what you do with this information that will be valuable. It is the knowledge to actually work with this information rather than handling it that will be valuable.
My opinion is that this shift is part of the transition to a knowledge based economy, an economy in which the knowledge that you own and that you work with is valuable and not the way you push bits from a hard-disk to the network and viceversa. The applications that Oracle, MS and IBM have are IT infrastructure and in a knowledge economy IT infrastructure doesn’t add much value because its functionality is simple (or has become/will become simple) and therefore unprofitable. IT infrastructure is becoming a commodity.

In the words of late Peter Drucker: People tend to concentrate on the word "Technology" when faced with the term "Information Technology". It is time to concentrate on "Information".
It is time to do something with this information. Handling it has become a piece of cake.

P.S. A lot of open source usually stands for developing infrastructure software; think about Sendmail, various web-servers, application servers, etc… So when you see that whole applications are getting open-sourced (think CMS, shopping carts, catalogs, CRM, etc…) it is a good bet that these applications are actually infrastructure, that they are the building blocks for something bigger and more valuable. It means that while previously you were supposed to work at a very low-level with database tables, triggers and Java code, from now on you may have to work with whole application stacks. Of course, not all development will be like this, someone will have to create these applications. But more and more of the work will consists of integrating these applications in order to create something of value.

At one point I wrote a code-generator that was looking up the table description in an Oracle database, take a look at some configuration files and create a DAO for accessing that table. The DB was accessed thru straight JDBC and not thru an ORM framework. We basically had to generate 2 classes that were supposed to be used by their clients: a POJO that was mapping to the table and could have some business logic methods and an access bean that would implement all the CRUD operations and could also have some business logic.
The solution was to generate 4 files: 2 files for the POJO and 2 files for the access bean. The POJO would have a super-class that was generated all the time by the tool and contained the getters and setters for the POJO and a sub-class that would be generated only the first time and that would extend the super-class. The sub-class would contain business-logic methods. The same things were generated for the access bean: a super-class that implemented the CRUD operations and get generated all the time and a sub-class that would contain business-logic methods and would be generated only the first time. Again, the sub-class would extend the super-class.
I was thinking about it and I came to the conclusion that what I basically did was to separate infrastructure (getters and setters, CRUD operations) from business logic. The infrastructure was implemented in the super-class and the sub-class was concerned only with business logic. The infrastructure could be generated and re-generated from various descriptors while the business logic could not be generated because it was too fine grained for it to be expressed in descriptors.
I think this is a good separation of concerns and one that would be beneficial to development if it is thought out carefully. I am not too crazy about code generators and in the above case we used it because we didn’t have a ORM framework at the time (this was pre-Hibernate and all that), but this separation is important and could be used beyond code-generation. You could use it to determine what components of an application are generic enough so that they can be either generated by a code generator or handled by a framework targeted exclusively at managing that component of the application. The rest of your application would be behavior so fine grained that it cannot be expressed in config files and is better expressed in business logic beans.
The difference between infrastructure and business logic is basically a difference in the degree of expression of various components in the application. Infrastructure has a very low degree of expression: it can be described in config files. Business logic, on the other side has a very high degree of expression, it needs to be coded.

To give an example for the above ideas: let’s say that you have a client that connects to a web-service. One way to implement this would be to use a code-generator (such as the one supplied with Apache Axis) and generate the stubs that connect to this web-service. Well, it is possible that the web-service would change in the future. In order to adapt to this change easily you could generate the web-service connection classes in a super-class (that would implement the infrastructure) and create a sub-class that would be called from your application (that would implement the business-logic). Changes in the web-service would result in changes in the super-class (which is not used by your application) and would not require big refactoring on your application.
Once again, I am not a big fan of code-generation. I was using the above as an example of the benefits of separating infrastructure from business-logic. You could use this separation for implementing generic behavior in a Template for example, but it is important to notice this separation and enforce it.

I am looking at AOP as an example of applying the theory of Division of Labor to development. With AOP it is possible to have different types of professionals working on the same project at the same time.
In theory a security expert would secure the application or even better a security expert in one domain would secure one part of the application, while a security expert in another domain would secure another part of the application, a transaction expert would trace transaction boundaries within the application, a concurrency engineer would serialize calls to a state repository in order to implement concurrent access, etc… I say this is possible in theory because managing this interaction takes some good managerial skills and processes, processes with which I am not familiar for the time being.
The division of labor is a process that scales very well in diversity and ultimately in complexity. Various tasks are assigned to people trained specifically for them and are executed in an efficient manner. At the end of it the process has managed to put together a product which is quite complex and has received input for people of various backgrounds. This could be very well the solution to managing complex interactions when developing software: dividing them into tasks which can be performed individually by experts and assembling them at the end. Just like when you have a coffee you don’t have to worry about growing a coffee tree, harvesting the beans and roasting them because someone else did that for you when you are writing an application you should not concern yourself with securing it or implemeting failover thru exception handling because someone else would do this as well.
The division of labor pulled the human race out of the Middle Ages and into the industrial age. Hopefully AOP will be able to pull large, complex projects out of the quagmire of pointless meetings, weekly merges, versions, etc… and into functional, deployed applications.

P.S. I wrote this post on the most horrible coffee I had in a long time. If it had an effect on the post itself I am sorry.
P.P.S. Separation of concerns resembles the division of labor as well.

A new kind of APIs is emerging: the APIs written by businesses rather than by IT corporations. Some examples would be Operations Support Systems for Java and Transaction Workflow Innovation Standards. Various corporations are collaborating and setting up standards regarding the processes that they have to manage, and I think this is the right way to go about it, after all they know what they are talking about, they have decades of experience in their business, they have some ideas about where it is going. They know what interactions are performed between different IT systems in their organization. They are the ones in the best position to determine how their processes should be abstracted.
Sun, IBM, BEA, Oracle, etc… should concern themselves with the plumbing of the IT infrastructure rather than with defining how a business process should occur and interact with other processes. They should avoid business-oriented APIs and delegate this to the businesses that have been running this process. As far as JCP is involved they should be inviting parties outside the IT landscape for supervising business-oriented JSRs and defining the main business rules. JSR 144 is a fine example of this approach: take a look at its expert group and count the organizations which are not involved in IT.
As far as the developer is concerned: today your average developer is expected to know a few infrastructure APIs: the Servlet spec, Struts, Spring, maybe EJBs… I wouldn’t wonder if in the future a developer working for a telco would be required to know telco-specific, business-driven APIs in order for it to be more productive. In other words the developer would be expected to know the business rules which this API is formalizing. Another step in bridging the gap between a business analyst and a developer.

P.S. The involvement of businesses in the creation and management of business-oriented JSRs benefits the IT corporations backing JEE a lot. The effort made by a business thru-out this process will translate in some form of a commitment to the JEE platform which would benefit the IT corporations involved and ultimately JEE. The JCP process could (and should) act as a communication channel between big-business and JEE.