Entities of a different kind

FeaturedHexagonal Architecture

Note: In what it follows, the term “Entities” refers to both Entities and Aggregates indistinctly.

The Hexagonal Architecture sets a clear distinction between Domain and Application layers, which drives us to locate all entities in the Domain layer for it is there where the meaning (what the software is about) has its home, whereas we must put whatever run factual changes in those entities, as use cases require, in the Application layer.

For example, Customers and Items, as entities, belong to the Domain layer, whereas Sale an Item, Bill an Order, or Ship some Items, belong to the Application layer because they represent in the application (translated to the language the software can understand) what in real life we would consider facts that happened concerning particular Customers and Items.

Just as a brief reminder, in this same Hexagonal Architecture, the UI layer would be where what triggers those changes live, whereas the Infrastructure layer would be where we put every piece of software needed to have the application run (frameworks, ports connecting to databases, file system, message transporters, etc.).

Not all entities are created equal

A simple inspection through the list of entities in any given application would show us that some entities play a role, and have a life span, fundamentally different than others:

  • There are Master Entities, which represent in the Domain model that every particular application implements what in real life we would call “entities” too: Customers, Items, Suppliers, Service Providers, Employees, etc. One of the elements of external reality (external to our software) they exhibit is an id of their own: either you decide relying on those ids to identifiy instances of these entities in your application or not, that is a property (ISBN, Tax ID, Social Security Number, etc.) you are likely to handle somehow.
    Master entities change in a very slow pace: once created, and approved, you would change them seldom, for instance, to add a new Customer Address, or update the description of an Item, or mark an Employee as not working in the organization anymore. Conversely, master entities show a long lifespan, ordinarily of years.
  • There are Transaction Entities, which play in the Domain model what in real life are Orders of any kind (sales, purchases, work orders, etc.), Bills, Shipments, etc.
    Transaction entities may suffer plenty of changes, for each change represents a step in the fulfillment of the business process in progress. However, once completed, these entities never change again, and so their lifespan is very short, commonly not more than days.
    In opposition to Master Entities, your application will surely need to provide unique ids for every instance of these Transaction Entities.

That distinction between master entities and transaction entities has indeed a long history behind if you accept database tables as true ancestors of entities. In fact, I took both names from the field of Business Intelligence, in which, quite approximately, transaction entities constitute the data origin read to feed the so-called fact tables, whereas master entities play the role of dimension tables, due to their role as axis in the charts where the calculations provided by the BI engine are pictured.

Incidentally, there is another feature that is also different in magnitude between master and transaction entities: their number. With time, the number of transactions grows far quicker than the number of instances of any master entity. Although this is not true in all cases, for it depends on every particular organization, it is often true that the number of Customers is far less than the number of Sales Orders.

So, we might be tempted to state that master entities are fundamental and transaction entities are accidental, for master entities are supposed to be there, perfectly defined and steady, even though while nothing happens to them.

Jose Ortega y Gasset
Ortega y Gasset

However, as the Spanish philosopher José Ortega y Gasset said, “I am me and my circumstance“; hence, Mr. Ortega would be a master entity, and his circumstances (what impacted, or happened to him), a plethora of transaction entities.
In other words, no matter how permanent we humans might look, our self is defined by a never-ending interaction with our surroundings, in a way that makes us a work in progress which never gets accomplished. It’s that dialogue what really makes us.

Everything in its right place

Time to come back to software. In my opinion, the different roles that both master entities and transaction entities play in the application constitute a fundamental trait which must be made apparent in how we organize the elements of the code.

One easy way to achieve that purpose is to keep them both in the same Domain layer, but segregating one set of entities of the other, for instance, in separated folders. This option is quite conservative, so it might avoid unexpected surprises to eyes dropping a casual gaze on a repo organized like this. But it looks wrong because it ignores the key fact that master entities exist before any use case might run, whereas transaction entities are tightly intertwined to the execution of use cases: with no execution of a particular use case, there cannot be any instance of the related transaction entity whatsoever.

We might say that master entities keep the major part of the meaning of the Domain, whereas the transaction entities only carry an operational meaning. Though appealing, this is wrong, because ultimately all entities are defined out of operational foundations. For instance, Customers are defined because of their action of buying something to the organization; Items are defined because they are what the organization sells, buys, or utilizes somehow.

All Entities, either master or transaction, are defined out of operational traits

Therefore, what looks more natural to me is to relocate the transaction entities in the Application layer. For they belong there as the use cases do too, and, as I said above, with no use cases, there are no transaction entities neither. IMHO, it is not enough that transaction entities are persisted as well to keep their place in the Domain layer, compared to the key facts we have seen:

  1. Master entities have a lifespan far longer than transaction entities.
  2. Master entities pre-exist transaction entities, for the former may exist without the latter, but the opposite is not true.
  3. Master entities have got their own unique ids, whereas transaction entities need to be provided with one by our software.

Does this feel right to you?

Copyright notice: featured image taken from the Java Viet Nam blog, though it is not original of the blogger.

Global Software Architecture Summit 2019 – episode V

Sketch by Simon Brown on Software Architecture for Developers

This is the fifth, and last, episode in a series of posts reviewing the first Global Software Architecture Summit that took place in Barcelona the 10th of October. To see the whole list of episodes, please visit this.

Architectural Thinking

Mark Richards
Mark Richards

This talk, given by Mark Richards (@markrichardssa), was the best first class in Software Architecture that I have ever seen, and the best definition of what Software Architecture is that anyone might find out there. He also emphasized the natural existence of tradeoffs in every architecture alternative there is, and the key role of software architects in making explicit the prons and cons of them, in order to pick the one that fits in the scenario in hand.

So, should have you the opportunity to attend one of Mark’s courses, or attend to any of his talks, just go!

He also gave us the advice to spend the first 20 minutes of our days learning something, and in particular something we were not even aware that existed: the unknown unknowns. How could you learn about anything that you didn’t know that you did not know? Staying tune for what is said out there, in related media: podcats, blogs, aggregators, meetups, etc.

Effective Architecture Practices Debate

This third debate was moderated by Álvaro García (@alvarobiz), and the invitees on stage were Sandro Mancuso (@sandromancuso), George Fairbanks (@GHFairbanks), Juan Manuel Serrano (@juanshac), Carlos Blé (@carlosble), and Eoin Woods (@eoinwoodz).

Their interventions were eventually centered on how implement changes in architecture style in practice, with particularly wise advices from Sando Mancuso. For instance, it is always key to have a roadmap as an expression of a direction from now to where the organization envision to go, even though being concious that the straight line drawn in the roadmap is going to become a curvy road in real life.

Incidentally, someone mentioned the risk of overengineering that might arise when teams try out what influencers in technology are claiming. IMHO, this is a debate born out of a prejudice. In my more than 20 years in the profession, I only found one example of overengineering, whereas I had to struggle with plenty of examples of bad code, bad practices, or bad architecture. I am not saying overengineering is a fake problem, just that I feel like we should put our efforts on the 95% of the issues before facing the (being generous) 5% of them.

Choosing the Right Architecture Style Debate

The last debate, and the last session, of the event was moderated by Christian Cotes (@ccotesg) and had Alex Soto (@alexsotob), Viktor Farcic (@vfarcic), and Ana-Maria Mihalceanu (@ammbra1508) as guests.

Again, the debate circled around the adoption of microservices as architecture style, and more in general about how hard it seems to distinguish between what is hyped and what is reasonable. It was certainly a very lively conversation, mostly because of the funny style of Viktor Farcic, the author of the book “DevOps Paradox“.

Unfortunately, there was nothing new to add to what we had already listened during previous sessions: the microservices debate, and the fear of change some professionals exhibit, though they likely consider themselves wise and not fearful. Maybe the organizers should consider how to avoid this reiteration of topics in further editions of the event.

Data-Driven Scalability and Cost Analysis for Evolutionary Architecture In the Cloud

Between the two debates above, Ian Gorton presented the results of a study that the research team he leads in the University of Northampton in Seattle conducted to analyse how the cloud costs vary depending on the programming language, and also how those costs depend on which are the values we set in the scale configuration panels of cloud services.

In my opinion, was not a surprise that Go was the most performant programming language in Google Cloud, not because both are initiatives born at Google headquarters, but because Go community shows to have one, and only one, focus: performance.

No surprising neither that the default values suggested by Google Cloud were not eventually the cheapest. That is why companies which whole business model is help organizations reduce their cloud bills exist.

Copyright notice: Featured image taken from PlanetGeek.ch blog. Visit the blog for a full set of other enjoyable pictures.

Global Software Architecture Summit 2019 – episode IV

This is the fourth episode in a series of posts reviewing the first Global Software Architecture Summit that took place in Barcelona the 10th of October. To see the whole list of episodes, please visit this.

Reactive Architecture Patterns Debate

This second debate was moderated by Jakub Marchwicki (@kubem), and speakers were David Farley (@davefarley77), Mark Richards (@markrichardssa), Len Bass (@LenBass4), and Christian Ciceri. It had the Reactive architecture style, and incidentally the Reactive Manifesto and its advocacy of asynchronous procedures, as its key topic, deriving into complexity often too.

Actually, I signed the Manifesto 5 years ago (under the name of my company at a time, though), and have been working with asynchronous messaging since, whenever no dialogue with humans is required, because it improves performance at the UI level, and this is something that, as it is immediately obvious, makes sense.

Reactive architecture style has become extremely popular: Node.js was asynchronous from scratch, all programming languages got their implementations of asynchronicity in recent years, and several flavours of Event-centric architecture styles arose as well. Even a Reactive Foundation showed up, now as part of the Linux Foundation. Clearly, async came to stay.

However, if decoupled transactions, cut in pieces to be handled by autonomous services, were already hard to govern, making them asynchronous turns out to be even harder, as Len Bass repetedly said during the debate. Are our teams ready to handle this complexity? And more importantly, is this complexity necessary in all the contexts when it is being applied now? Unfortunately, this debate on complexity kept the speakers to explore other implications, or technologies which implement reactiveness which might not be known by the audience.

Cover picture of the end of work book
The End of Work book

Another fascinating idea dropped by one of the speakers, but not followed up, was this seek of humanless that technology seems to be in seek of recently. I am afraid people with no technical background is increasingly feeling upset, even scared, by this pursuit, and I do not see that people with technical background (like the speakers, for instance) spend enough time stating clearly that the work is not ending. What we seek is automation of tasks that, by their own nature, do not require understanding to be completed.

For Artificial Intelligence, no matter its name says, is not turning machines into smart entities: machines can be trained to recognise cats in pictures, but not to know what a cat is. Some people believe that machines might achieve that ability (let’s say true intelligence) in the future, but I do not see this happening soon because just a few animal species reached that point, and evolution needed thousands of milions of years for them to appear. To behold tiny algorithms transcending their machine nature and become beings is not a spectacle I am expecting to see in my life.

Global Software Architecture Summit 2019 – episode III

ci-cd pipeline picture

This is the third episode in a series of posts reviewing the first Global Software Architecture Summit that took place in Barcelona the 10th of October. To see the whole list of episodes, please visit this.

Software architecture evolution, what to expect? Panel Discussion

This panel was moderated by Alvaro García (@alvarobiz) and the participants were Eoin Woods (@eoinwoodz), Michael Feathers (@mfeathers), and Ian Gorton. They had an animated conversation, during which the trouble with Microservices came up again.

No one doubts that Microservices is a very demanding architectural style. Everything is easier if done in a monolith than with microservices. Unfortunately, as I pointed out here, monoliths aversion to change prevent organizations to evolve by making this evolution tougher day by day. And there are plenty of reasons for organizations to embrace change: for it makes them more competitive, quicker to response to unexpected shifts in economy, or laws and regulations; in a word: keep them safe from obsolescence.

So, on one hand, monoliths are easier and cheaper, but age us. On the other hand, microservices (as an example of decoupled architectures, or, in a wider sense, of those more complex architecture styles we heard about in the Episode II of this series), keep us afresh, though they are tough, and expensive. What to do, then?

As I see them, microservices and monoliths are two opposite poles in a whole spectrum of factually possible adoptions of architecture styles. I mean, it looks reasonable (and I saw it frequently in the past three years) to have 5, 10, or 15 microservices running in the very same ecosystem that a heavier set of coupled functionalities that we may still call a monolith. This is not a mess, this is an implementation in progress towards a more and more efficient ecosystem step by step.

Monoliths and microservices are two poles in a full spectrum of possible architectures

Maybe microservices detractors feel afraid of gigantic projects which purpose is to decouple completely a fully functional monolith into an ecosystem of microservices, all in all within a timeframe of, let’s say, two or three years. Well, I must agree with them that this kind of transformation is not only too risky, but, in most cases, unnecessary. Mixed architectures, laying somewhere in that spectrum I mentioned above, may be the best possible answer.

The panel also emphasized the key importance of experimentation and observation. Again, they advocated the adoption of the Scientific Method (come up with a theory, run experiments, gather information, and obtain useful knowledge), in a neverending process a colleegue of mine named “trial and error“. But this name does not feel right to me, because I am afraid it is missing the relevant part of the point, and makes it something trivial, when it is not.

I’d rather call this Continuous Change, as a practice and as a third companion for the Continouous Integration/Continuous Delivery, duo. To me, the Continuous Change would be a more general approach to the creation of software, whereas CI/CD would be a particular application of it. For CI/CD, as in the famous diagram modeled out of the Möbius strip, tighly integrates the creation of the code with its deployment, in a way, for instance, that makes testing in production an obvious consequence.

But they are not. The Möbius strip is one possible topology to picture the production of software, among others. In this topology velocity prevails, in a way that the bigger the number of iterations in a given period, the better. Actually, the CI/CD cycle prevents experimentation unless it is reducible enough to be feature flagged, in a mechanism which purpose is to make changes as atomic as possible. Following this approach further is how even chunk commits look self-evident.

Unfortunately, I was not able to find a better idea to integrate experiments in the common CI/CD cycle in a more satisfactory way. A topic for another day, I am afraid.

Copyright notice: Featured image taken from the Linux Juggernaut web site without permission.

Global Software Architecture Summit 2019 – review series

GSAS 2019 logo

Thanks to my employer, I had the opportunity to attend to the first Global Software Architecture Summit that took place in Barcelona the 10th of October. The event, organized by Apiumhub, consisted in a one day track of speaches, debates, and one panel, as well as the usual breaks to mingle with the community, all around the topic of Software Architecture. 8 different sessions, overall provided by a bunch of illuminating, and funny, speakers who brought a huge level of expertise to the stage, and that ended up being an extremely inspiring, though exhausting, day for me.

So, I decided to share the ideas I came up with during the event here, hoping that some of you might eventually feel that inspiration too, at least as much as I can recreate it. Problem is, there was so much to say, and so many paths to follow starting at any of the topics dropped by the speakers during the event, that it came eventually obvious that I would rather cut the whole review in smaller, digestable pieces.

This is how this series came into light. The episodes are the following:


Some of the episodes are longer than others, whereas some topics show up repeatedly in some, or all, the sessions. Should I list the main topics, I’d clearly separate the incidental from the fundamental: micro-services, kubernetes, and complexity, as incidental; and the high value driven by experimentation, as fundamental.

Certainly, overall the event was highly inspiring, and I left the place with plenty of ideas that, I am sure, worth spending some time exploring.

As a final note, my advice to everyone involved in producing software: attend to events, mingle with the community, learn, and enjoy!

Copyright notice: Featured image taken from the GSAS Twitter account without permission.

Global Software Architecture Summit 2019 – episode II

A depiction of observability by Haproxy

This is the second episode in a series of posts reviewing the first Global Software Architecture Summit that took place in Barcelona the 10th of October. To see the whole list of episodes, please visit this.

Applying architectural principles, processes, and tools

Picture of Len Bass
Len Bass

The second speaker was Len Bass (@LenBass4), who started by giving us what seemed to me a first-day lesson in college on the 5 key characteristics a software architect must be proficient in, but when the time for questions arrived, dropped plenty of valuable thoughts.

One of the main vectors of change in how software is created has been the recent incursion of Operations in it. Traditionally, developers cared very few of how the code they produced was actually run. Since the DevOps, the role of developers has evolved quite quickly, and right now a developer who does willingly ignore the operational part of the software gets (or should get) easily unemployed.

Which, incidentally, drives in another of the main leit-motives of the whole event: which level of architectural complexity should we accept because of that invasion of Operations in software. Illustrated with the neverending monolith versus microservices debate. Some of the speakers in the debates were contrary to the adoption of microservices and others were in favour of them, if and only if microservices fit the problem and context. This latter seems a pretty cautious position to me, but definitely it was not conservative enough for those who complain every time things evolve faster than they can follow.

We will have time to come to this again. Before that, I would like to note three other thoughts from Len Bass: one was the idea that it is very rare that software crafters participate in the creation of any application which is not similar to other applications already created. An example of a true novelty is the Amazon’s invention of a bunch of new technologies, tools, and procedures in its seek of Infrastructure Scaling. In general, though, we should keep ourselves aware of what other people is doing all the time, as well as being open and communicative so that others may also benefit of our work.

A second thought is that log files are quite more important than we think: log files mainly exist to help first responders in case of an emergency. But they also help us gather deeply valuable information of our software’s behaviour. Which reminded me of Observability, a field that fascinates me for its promise of providing us with the ability to query the system any time, and which depends on log files to deliver its magic.

For those also interested in Observability, I highly recommend this introductory page in the Honeycomb website (I have no liaison with this particular company, except by the fact that I got completely mesmerized by a speech of their former CEO, and current CTO, Charity Majors, back in 2018 in London). There is a podcast that I follow too: o11ycast. And a conference, O11ycon, which happened in San Francisco last year. Slides and videos are free to access online.

A third idea that Len Bass said that shocked me was that Microservices potentially drive technology inconsistency into our application ecosystem. This is a common criticism, though I doubt that it is fair. It seems reasonable to assume that there is a limit in how much diversity organizations can handle. Where this limit lies on, and why is the blame put on microservices when they just expose this limit, are the two factors that look unfair to me.

Bearing in mind that Microservices allow organizations to isolate disparate technologies as long as they communicate via standard protocols, like HTTP or asynchronous messaging (by the way, another schoking moment was when someone in the audience asked a question in a way thay made me think this person believes microservices are asynchronous per se, which is wrong), the potential incosistency in adopting those disparate technologies cannot come from them as an architecture style. For it is completely possible, even reasonable in some contexts, to adopt microservices using just one technology.

I believe that microservices allow us to pick the technology that best fits in every case, but this is my believe, not something that microservices force organizations to do. And even I accept that this is an ideal: in practice, the resources are scarce, and organizations must focus on less than 5 technologies. I worked in a company where even more than 5 thecnologies were in use, and everyone was aware of the reasons why, but I do neither recommend that, nor see it happening a lot. On the contrary, my advice would be to focus on the operational, and the infrastructure, issues, get skilled on that, before bringing in additional diversity.

In my opinion, inconsistency comes out of us making bad decisions, not by the adoption of any particular architecture style. Microservices, as any other architecture style, are neutral, optional. Pick them if that fit your needs, simply ignore them if not.

Copyright notice: Featured picture taken from the Haproxy website without permission

Global Software Architecture Summit 2019 – episode I

A picture of Peter Naur by Duo Duo Zang

This is the first episode in a series of posts reviewing the first Global Software Architecture Summit that took place in Barcelona the 10th of October. To see the whole list of episodes, please visit this.

Code is your partner in thought

George Fairbanks
George Fairbanks

George Fairbanks (@Fairbanks) was the first speaker, and introduced to us some ideas which originally appeared in a book by Peter Naur, in 1985, being the most relevant of them the realization that the act of Programming, understood as the overall set of activities involved in the production of software, is by itself a task of invention of theories to understand a problem and finding suitable solutions to address it. Actually, the title of the Naur’s book was “Programming as Theory building“.

To achieve a solution to a given problem, we first invent theories, and the code we eventually write is a representation of that theory. Which is another powerful reason why the code must be as understandable as possible: because the code is the main instrument of communication of that theory to other people, now and in the coming future; a purpose, by the way, that the old-fashioned procedure of writing tidy documentation upfront of the code was unable to fulfill ever.

Computer programs only make sense to the team who have developed it

Peter Naur (1985)

In other words, the code repository is not only where the documentation of the solution, and of the problem it addresses, live. Since that code is also a representation of the theory we invented in the process of understanding the problem and generating ideas to handle it, git is not only in charge of handling a repository of code, but also a repository of knowledge.

Actually, this is an idea which stands out as the pivotal summary of the whole event: the creation of software is an exploration of thoughts inspired by the necessity to address some problem, and it looks, and it should be practiced, more as Science is, via the scientific method. Come up with an idea, produce some experiments to test it, and, if the experiment ends well, make it happen in Production, just before keeping the exploration going on.

Even more, this procedure of coming up with a theory, test it in practice, and eventually put it into production in the form of runnable software, is a long term learning process. As metaphor, we might imagine a wider cycle around the quicker Agile cycles that we are used to call sprints: a process extremely important, in my opinion, to keep Scrum teams from falling to the sickness of delivering new features time and again, but never reaching any goal.

Fairbanks dropped other interesting thoughts, as the concept of the Model-code gap (in essence, the troubles which arise whenever the representation of our thoughts that we put in our code is crooked, or biased, instead of the finest match to them possible), or that documentation is a good place to explain those ideas or decisions which did not get through: precisely because the code does not represent them, it may be a good practice to include them in the code repo as README files, for instance.

In essence, it was a very nice introduction of the main topics that we were going to see showing up throughout the whole event.

Copyright notice: Painted portrait of Peter Naur by Duo Duo Zhuang.