The writer Neil Gaiman introduced the Soft Places to the world with issue 39 of the comic-book series “The Sandman” in 1992. As explained there, the Soft Places are blurry spots where our Reality and the Dreaming meet. During their slumbering, a few, fortunate dreamers might find themselves wandering through that blended lands. And, while staying there, they would experience a weird convergence between what is allowed to happen in our space-time and what can happen in the Dreaming.
Like stumbling upon Marco Polo, as is shown in the picture above.
Time in the Soft Places is fluid, which turns staying wandering there for a very long time pretty dangerous. And yet, despite the risk of getting lost, I believe we should not fear exploring whatever Soft Places we may have the opportunity to visit. For they are also the places where unexpected ideas can blossom.
It was during one of these absolutely free explorations that I end up thinking about Eventful architectures. More precisely, I started wondering what kind of application would we end up with if all we could use were Events.
To make sense of this question, we will need to review some other topics first.
What if Events were the beginning and the end
Technically speaking, software applications that rely on Messages () to achieve decoupling among components are called Reactive. I’ll say more about the distinction between Messages and Events in a second. For those interested, the best short description of Reactive Architectures is the one provided by their original promoters in the Reactive Manifesto.
It is quite common to call reactive applications Event-based applications too, due to the key role that Events play in them. This alternative name is unfortunate, though, because Messages and Events are not synonymous.
As I explained further in the 2-post series Fiction in Software, Events are representations of Facts. Or, in other words, Events are a tool that software architectures invented to transport meaningful collections of data from the Domain (i.e., the real world) to the Codomain (i.e., where software applications run).
On the other hand, Messages are a means to support transporting data among autonomous software components. Hence, Messages belong to the Infrastructure layer and could be replaced by any other alternative way of transporting data with no change in the application whatsoever.
For example, you might use an existing infrastructure put in place to transmit Events to also distribute meaningless messages like a heartbeat. These heartbeat messages are simply a tool that listening services would consume to acknowledge that everything is OK with the resources in that infrastructure. In that sense, all Messages are useful, but only some of them (for example, Events) carry meaning.
In practice, Events are transported over Messages, which makes Events a kind of Message. Thus the confusion.
Another kind of Message that is also meaningful are Commands (i.e., demands of execution). Commands do not represent something that already happened, for the order they carry may or may not be fulfilled. We may see Commands as promises of a future fact, a promise that, if succeeded, should trigger the publication of one true Event into the application as a consequence.
Due to the fact that Event-Driven architectures are heavily used, software architects eventually had to invent less confusing names for meaningful Messages. They start calling them Domain Messages.
The common rule that all Domain Messages must fulfil is to be meaningful: they must express their meaning with the terms in the Ubiquitous Language, and they may be understandable independently of the software components that publish and consume them. In other words, by humans.
Now, let’s imagine that all the facts that happened to a given Client, or to a given Item, were represented by Events published when those facts happened and that we saved a permanent, read-only copy of those Events. By gathering all those Events we would have become able of retelling the complete History of that Client or Item.
It is very rare that organizations keep this kind of record. They only exist in very specific industries, and the software is bespoke, limited in scope, and expensive. In a scenario wherein the software industry is going to be regulated, and potentially dangerous tasks are going to be fully automated, being able to keep a record of what happened and produce a retrospective view everyone can trust is becoming, in my opinion, more relevant than ever before.
Another beneficial side effect of this kind of History recording would be that it avoids precluding future uses of the data we keep. Who knows what kind of data-driven decisions we might take in the future? Thanks to the possibility that retrospective views provide, by the way, a view that current applications cannot provide, those decisions of tomorrow are not limited by what we decide to save in our databases today.
So, in summary, we are longing for an architecture that:
- keeps record of all Domain Messages,
- and permits retrospective navigation within that record.
Here is where the idea of Eventful architecture comes into play.
What if all that we have in our persistence layer were Events?
Another way of defining Eventful Architectures would be as those that extend the ability to produce a complete representation of any given Entity to any moment in time. Hence, Eventful applications will be capable of providing a complete representation of an Entity as it was today, yesterday, or two years ago.
Actually, they would potentially become capable of even more. It would be possible to reconstruct the full chain of Facts that drove that Entity from any state in its past to its present state, which opens the door to some kind of causality tracking.
Of course, we must not forget that there are practical reasons why software applications tend to focus on the present. First of all, the majority of users work with present, or very recent, data. Also, development teams performance is commonly measured by the value they deliver, and delivery is based on upfront requirements and expectations. Developing something for if in the future we would want to look at it would be considered off the trail.
So, Eventful applications would need to include a mechanism to keep track of the current properties of all their Entities, so that experience of the majority of users, those who work with the present, is at least as accessible, fast and affordable as common applications are right now.
What type of mechanism would that be?
Here comes the Ledger
I think this mechanism could be a Ledger. Long story short, a Ledger-based application takes Events as its data source and produces coarse-grained representations, or States, out of them. The Ledger in its name refers to the element, either physical or logical, that would group, and store, all those States for further use.
It is key to distinguish between snapshots and States, for they only have in common that both are timestamped.
A snapshot is exactly what its name means: a static picture of all the properties of an Entity as they are right now. As such, they are usually complete, although they might include some selected properties only.
States, on the other hand, are first-state citizens in the Eventful architectures because they are meaningful. States can aggregate plenty of Events together, or they can be the outcome of complex calculations. With States, the sky is the limit.
For example, we would have an Item Ledger, a Client Ledger. Or an Inventory Ledger. Or whatever else collection of States we might find useful producing.
Let’s think, for instance, of an imaginary Ledger-based application handling goods in a warehouse. The overall monetary value, in euros, US dollars, or whatever other currency you’d rather choose, of all the goods in that warehouse at a particular moment in time is an Inventory State that may be computed and then persisted in the Inventory Ledger.
As shown in the images below, multiple representations can provide different views of the same data and therefore multiple States, each produced with different purposes in mind, may then be saved in the Ledger.
The simplest, most obvious use of Ledgers in an Eventful application would be to keep the last State of every Entity to provide a reliable, excellent experience to everyday users. These simple States would be the replacement of what we usually obtain from standard software applications.
That would be the baseline. What we might build on top of that only true daydreamers would say.
Copyright: the artwork at the top is taken from the DC Comics series The Sandman issue 38 (1992) without permission.
how possible 