29 May 2007 - 21:01Event driven architecture

One interesting development is the appearance (re-appearance?) of EDAs and the traction they are gaining in the developers mind-share. I see the interest in EDA to be an extension of the interest in asynchronous processing, EDA is asynchronous processing taken to a higher level. It is interesting because it is a pretty novel technology which (as far as I know) has not been tested in large scale systems (large scale in terms of number of nodes which receive and process events). So far the examples which are given out on sites such as infoq are toys and I find this very telling for the level of adoption on this architectural style.
Nevertheless, the concept is bubbling all over the place and the seeds for the next architectural atrocities and miserable failures are being sown right now: posts such as this are already advocating “event patterns”. Event patterns is a pretty pompous name for an architectural construct which appears to be in process of development right now. You need a bit of experience before starting to lay out some guidelines and a few EDA applications don’t give you this. Let’s wait a few more years and see something emerge. The famous GoF design patterns were written quite a few years after C++ became widely used, I’d say we should take the same approach for events.

So far I see events being used either as gateways into larger applications or the infrastructure for implementing simple, one-way, non-circular work-flows. In the infoq example the main hard problem resolved around circular flows: a parent sends a message to a child which sends it back to the parent which sends it again to the child. The solution advocated was pretty simplistic, if I remember the presenter suggested keeping track of the nodes a message has visited and do something in case it is visiting the same node twice, probably inserting the messages in an envelope which gets sent from node to node, hopefully in a transparent manner that would allow for different algorithm for detecting and resolving circular flows depending on the nature of the application using EDA.

Applications using event-driven-architectures are essentially message-passing applications and fall pretty easily within the distributed computing domain. Distributed computing problems are very hard problems and the interactions between distributed components are pretty hard to follow. I don’t think that we are currently in a position to use EDAs on a large scale because I don’t see the modeling tools and frameworks that could solve the hard problems invariably generated by a large number of nodes passing messages to one another (mind you, this need could be turned into a business case). However, most of the world would beg to differ: take a look at the example given here. It is pretty interesting to see a problem being solved by slapping the tag EDA on it. The article brings together a complex web of interactions, states EDA and assumes that this solves the problem.

Another problem that I see with EDA is the fact that you couple systems indirectly. With EDA you need the whole picture of the systems, you cannot really say that you will send a message to a system thru a web-service and you finished with it, it is possible that this system will get back to you in such a way as to create a circular flow. Given the fact that this flow spans different systems it could be very hard to implement algorithms that would detect the fact that a message has been sent multiple times because of a circular flow (such as in the infoq example. In the infoq example you could implement such a detection mechanism because you had control over the whole system). This scenario should be considered when implementing inter-system flows.

So I would conclude this post by saying that for the time being EDA is a technology which is at its beginnings and, as a result of it, which has a lot more fluff than stuff. It may evolve into something bigger in time, but it poses very hard problems once the number of nodes involved in a particular event chain increases. Also, as far as I am aware, there are not good modeling tools that can be used for applying EDA on a large scale.

Later Edit: From this article it sounds like TIBCO (a well-established distributed computing vendor) offers a solution called BusinessEvents for event processing. Judging from a podcast with some guy from TIBCO I would not be surprised if it turns out a pretty good product.

Later Edit: With Complex Event Processing and Event Stream Processing it looks like the industry is moving towards processing event streams at a single point in a network rather than trying to manage the events at the level of the whole network. Maybe this is what is needed by the market. It is also part of the push towards asynchronous processing that we are witnessing. Maybe I was too harsh on it…