Why I wanted to write this book review

‍

More than a decade ago I was helping in splitting a monolith into microservices. The project, big and financially successful, was in .Net with the database in Oracle. A release would take hours and cause considerable frustration. We were working on Windows OS-powered machines with IIS as a webserver and within an on-premises data center. Fast forward 2 years, and things would look much different: the project was split into microservices, we were developing with Scala on MacOS using AWS as a cloud provider, nginx as a web server, and so on. As a colleague would say to me: “Only my wife and kids are the same; everything else changed.”

‍

In a sense, it was a bit magical: imagine having a huge pile of code now split into small projects managed by several small teams, each with its own pace, release cycle, and backlog. Of course, the road to microservices was not an easy one and a part of the reason was that the microservice literature back then was not as developed as today. We were still learning the best practices of working with microservices and domain-driven design. Things have improved, though, and a very notable book is Software Architecture: The Hard Parts, written by Neal Ford, Mark Richards, Pramod Sadalage, and Zhamak Dehghani.

‍

This is a (sort of a) disclaimer: the point of this article is to give you an idea of what to expect from the book and hopefully make you curious enough to read it (yes, the spoiler is already here: this is pretty much a solid “Read it and read it several times!” recommendation). Some details and explanations are, therefore, intentionally left out to boost your curiosity.  

‍

Who is this book for

‍

You might be tempted to skip this book due to its title if you are not working as an architect but that, I believe, would be a simplification. The book deals indeed with the microservices architectural style but make no mistake: you are likely already using principles described in this book. And if you aren’t now, you will rather sooner than later.

‍

Even if you or your team decides to continue using a monolith or a service-based architecture, it pays to know how to make those decisions: what you gain and what you lose in each case.

‍

Probably one of the most powerful messages coming from this book and a very important reason why every software engineer should read it is this: there is no good way or bad way, there is just a list of trade-offs. We will come back to this but seeing and understanding this message - almost obsessively repeated throughout the book - will make you a better engineer because you will better understand how to make technical decisions. As a funny note, after I finished reading it, I would make a list of trade-offs even for not-so-critical technical decisions because it had become fun to look at problems from different angles.

‍

Some of my favourite parts

‍

In the following, I will discuss some of my favourite parts of the book. Some are bigger topics, and some are things I felt were helpful and interesting to me without a special order.

‍

Before we begin, though, a few things about the book itself. It is split into two roughly equal–sized parts. Part one is “Pulling things apart,” and the second one is “Putting things back together.” Part one is thought to be about structure, and the second one is about communication.

‍

It takes a monolithic project through several iterations - component decomposition and refinement, database decomposition, microservices communication and contracts, and so on – as it splits it into microservices. An example project is the Sysops Squad Saga, which is used to showcase the various problems and solutions. Plenty of imaginary dialogue happens between the architects, database, and software engineers as the project is worked on. Let’s open the book!  

‍

‍

Monolith decomposition strategies

‍

1. Tactical forking

‍

You’ve got a monolith, and you’re not happy. You want to move to microservices because everyone else seems to be doing this. You book a conference room for a full day; you draw bounded contexts (some of them with dotted lines because they don’t scream boundaries quite clearly) and you give the drawings to the existing teams to implement the services. That should do it!

‍

But it is most likely not going to work. Bounded contexts are a good idea and an aim in themselves, but there are important steps in between.

‍

Let’s first be pessimistic: the code is pure spaghetti (e.g., no clear layers, a big bag of dependencies, unclear responsibilities, and so on). There are no clearly distinguishable packages or components, and the code is simply not decomposable. This is a case for tactical forking. With it you are like a sculptor: you make replicas of the code and chip away at the unwanted parts to form services (clone the repo and give each team a copy of the entire codebase). Here’s what the authors say about benefits and shortcomings.

‍

As a benefit, you can count on the teams being able to start right away, and generally, developers find it easier to delete code rather than extract it.

‍

What about shortcomings? The resulting services will still contain a large amount of mostly latent code, and without additional effort, the code quality will not improve (chaos will remain chaos). There could also be inconsistencies between the naming of shared code and shared component files, which will result in difficulties in identifying the common code and keeping it consistent.  

‍

The authors point out that this method is a tactical rather than a strategic approach as its name suggests.

‍

2. Component-based decomposition

‍

Do you see some well-defined (or even loosely defined) components in the code? That is good because you can then use component-based decomposition patterns. What I like in this case is the objectivity and the structured approach. You measure, and then you cut.

‍

High-level, there are 2 big steps (3 if you count easy-peasy):

‍

• apply component-based decomposition patterns in a structured and incremental way to break the monolith into domain services.
• Once the patterns are applied, start working on decomposing data and breaking those domain services into finer-grained microservices as needed.
• easy-peasy, right? 😊

‍

Here is a picture from the book to make the steps clearer (and justify the easy-peasy-ness):

‍

Side note: this figure is on page 138 from 771. Yes, it’s a big book so take it slow.

‍

Off course, I am joking! Each step is complex and requires quite some work, but the authors provide a great walkthrough with the Sysops Squad (the example project used in the book) and explain every step along the way in detail.

‍

You might be asking, "This is a long and tedious process. Is it worth the trouble?” I guess the answer will vary from business to business and budget to budget. The author's approach is structured and incremental, pretty much like a roadmap, making it easier to explain to stakeholders where you’re going and where you currently are.

‍

For example, during the first step, identity and size components, you would gather metrics to analyze the code so that you have a better idea of whether components (defined by authors as a group of files or classes that serve a business or infrastructure function—e.g., the OrderHistory package) are too big (doing too much) or too small (not doing enough).

‍

In the second step, gathering common domain component patterns, the aim is to consolidate common business domain logic that might be duplicated across the application. You would be focusing on reducing the number of potentially duplicate services and distinguishing between shared domain functionality (data formatting and validation, notification) and shared infrastructure functionality (logging, metrics gathering, and security). The name of the game here is common, including what goes into a shared “something”. So, what is that “something”? Should it be a shared library or a shared service? What are the trade-offs for each? (Spoiler alert: they are discussed at length in Chapter 8).

‍

What I find nice is that the authors also looked at tools to help in making the decision between strategies and objectives. Concepts presented include afferent and efferent coupling (incoming and outgoing connections to a code artifact like components, classes, functions, and so on), abstractness (the ratio of abstract artifacts to concrete ones), and instability (a measure of codebase volatility), which help drive a metric-based decision.

‍

3. Fitness functions

‍

I hope you get the idea. It’s not just a fancy flow. There’s real work in each of those steps, and it pays off. The next steps are equally interesting, and you will not get bored learning about them.

‍

You might be telling yourself: this is a huge effort, and it will take quite some time! How do I work on these changes and avoid having the same problems reintroduced while I do them? That’s a great question, by the way! For example, once you have determined the dependency graph (which component is dependent on which), you will likely want to have “something” in place to not have those dependencies broken (e.g., an engineer adds a dependency that breaks certain domain boundaries).

‍

Welcome to fitness functions! This is an architectural governance tool. As fancy as that sounds, it is basically a test that runs in certain places (continuous delivery pipelines or maybe a commit hook for simple functions) that fails if certain conditions are not met. In the dependency problem, they might check that some component does not have a dependency on another component by checking the namespaces (or packages in some languages) where the code resides. Of course, fitness functions can always be useful not only when you are migrating a monolith to microservices, as the authors nicely explain just at the beginning of the book. There are also examples of fitness functions for each step in the code decomposition pattern flow.

‍

4. Service-based as a pit-stop

‍

The second step above mentions domain services as a target just before refinement into fine-grained microservices, so I want to add a small note here: in both tactical forking and component-based decomposition, domain services are the intermediate step, but they can certainly be the final one. That’s because you now have something that resembles a service-based architecture quite a lot!

‍

The plan was all the time to migrate towards a service-based architecture first and from there you move on to a microservices one, should there be a need. A service-based architecture would not require you to do anything about the database schemas, which is great because 1 moving target (only the services) is better than 2 moving targets (services and database(s)). As a bonus, as the authors mention and I tend to believe them, decomposing databases is a pretty big problem on its own! Another important aspect is that a service-based architecture move is a technical one. That is, it “generally doesn’t involve business stakeholders and doesn’t require any change to the organization structure of the IT department nor the testing and deployment environments.”. The corollary is also important; I believe moving to microservices should probably be accompanied by a team structure change, given that now you build it and deploy it instead of (probably) just building it.  

‍

Let’s take a break and stare at a screenshot from the book for a second.

‍

‍

Integrators and disintegrators

‍

The concepts of integrators and disintegrators are probably my favourites in the book. I like them so much that I adapted the questions I ask candidates in technical interviews after them. I usually try to discuss breaking a monolith into microservices and the various reasons for the granularity of those microservices. I once had a very pleasant surprise when the candidate answered with examples from this book.

‍

So, what are they? Disintegrators and integrators are the opposing forces when it comes to the granularity of a service. They provide guidance and justification for when to break a service or go the opposite way.

‍

The authors propose both integrators and disintegrators and discuss them at length. I will detail 3 examples shown in the book and hope you will be curious to find out about the rest.

‍

1. Database transactions integrator

‍

Database transactions are one of the common problems in a system. If an ACID (Atomicity, Consistency, Isolation, and Durability) transaction is required, the services should be consolidated into a single service. The lack of ACID creates a lot of complex and error-prone correcting logic when one of the services fails and could leave the data in an inconsistent state.

‍

Now, the paragraph above is a mouthful so let’s look at one of the examples the authors give. Given 2 services, one managing a customer’s profile (profile database and table belong here) and one managing the customer’s credentials (password database and table belong here), and considering the scenario of a customer registration, if the password service fails, the data is left in an inconsistent state because there is no ACID database transaction between them. The credentials might be in their own service to reduce general access to sensitive data, but in doing so, the inconsistency problem is being created. Thus, if “a single unit of work ACID transaction is required from a business perspective, these services should be consolidated into a single service”.

‍

‍

2. Scalability disintegrator

‍

I find the scalability disintegrator the easier one to relate to. Given 3 services packed together in a service, if one of them has higher scalability needs, it makes sense to move it out into its own service. The equation is pretty straightforward: if the services are together, you need to scale up until the busiest one is not busy anymore, but in doing so, you scaled more (e.g.: a bigger AWS EC2 instance) due to the other 2 which translated to higher costs than if the busiest one would be alone into its service.  

‍

3. Code volatility

‍

Code volatility is the rate at which the source code changes, and I like this one a lot as well. You might test your code before release instead of relying on your customers – I heard of some weird companies doing this (I have to put this note here because this actually happens so often that the joke might not be understood by some engineers: it’s a joke, you need to test your code and in not doing it, you are only making yourself a disservice because those bugs will need to be fixed eventually and you will not even remember what you were doing there in the first place; it is also not that hard and I have some articles you can check out for tips and tricks: https://cdohotaru.medium.com ) - and because those services are packed together, the testing scope is bigger than needed when the rate of change for one of them is higher than for the other 2. In other words, if service A changes weekly, you need to test services A, B, and C because they are glued together. But if service A is isolated into its own service, the testing scope is reduced to service A.

‍

Giving each driver more or less power is not straightforward, and that is why involving the stakeholders to better understand the business is the right approach. You would discuss what is more important for the business: Is overall agility better (maintainability, testability, deplorability)—which translates to faster time to market—or strong data integrity and consistency? Better data consistency or better security? And so on.

‍

Chapter 7 deals with these 3 integrators and disintegrators and more and what I also like is the way it uses the same examples (a notification service made up of 3 notification services) to highlight how to think about the trade-offs: if this integration or disintegration driver is more important – due to business requirements – then this, if this driver is more important, then that. At some point, you might think the authors are making fun of you: “Didn’t you just say we split them? What do you mean we put them back together?” But, no, not really; it is just thinking about a problem from different perspectives. Once you manage to hold in your mind 2 opposing ideas and not go crazy but keep looking for arguments for each side, you are already a much better software engineer.

‍

Miscellaneous likes

‍

1. Database types and ratings for each

‍

Chapter 6 has a section on how to choose a database type. It goes on to explain and rate (some ratings are for ease of learning, ease of data modelling, and consistency) various types of databases, from the relational, key-value document up to more specialized ones like the time-series database.

‍

2. Shared library or shared service?

‍

That is certainly a nice discussion. The implications of one approach over the other are not always intuitive. Knowing the trade-offs of each (how is your shared service scaling, by the way?) is certainly advantageous. Some other concepts in the same chapter 8, are equally interesting.

‍

3. Eventual consistency patterns

‍

Given that microservices and eventual consistency are like twin brothers—they love each other, and they grow together—but things can get hard when they get naughty. The section on the most popular patterns is like a very experienced babysitter who gives you a moment to think of a solution for your problem. If you don’t have kids, you’ll not get this. Just ignore my dad jokes and carry on!

‍

4. Sysops Squad Saga

‍

This is both a like and a dislike, and here’s why: some complex concepts are just that: abstract, complex, and ... have I mentioned complex? The Sysops Squad is the project used throughout the book as an example. It really helps sometimes to translate those abstract and complex topics into solutions discussed by the imaginary engineers involved in the monolith to microservices migration. The book shows examples, and often, the discussions help readers better understand the concepts.

‍

Sometimes though, the discussions are just distracting and don’t bring a lot of practical value. Hell, at some point, those imaginary people start to argue on decision responsibility, and you are left wondering: “Oh, wow! Look at that! Engineers don’t fight only in real life but also in books!”

‍

Closing thoughts

‍

I have to point out that this article touches on just a few chapters in the book, but reviewing such a big book has its limitations. See the link below for a table of contents.  

‍

Software architecture: the hard parts are a solid read recommendation. But it is also much more than that. You don’t read this book and then toss it back in your library to have dust keeping it company. You are more likely to read it several times in your career and to come to it to re-read some sections to remember, for example, what on earth a Fairy Tale Saga is all about.  

‍

Besides the knowledge and experience it contains, the book obsessively repeats the word trade-off (about once every two pages and please consider that some pages contain images!) which makes me think the authors are as traumatized as I am of software engineers who haven’t been trained to think in terms of trade-offs (to all those engineers that worked with me before reading the book and learning about trade-off: sorry, really sorry!). I am very much convinced that training yourself to think in terms of trade-offs (do you see what I’m trading off in this paragraph?) will make you a much better software engineer. So just go and read the book! In the worst case, there will be a trade-off but I’m not sure anymore which ...

‍

‍

Book information:

Title: Software Architecture: The Hard Parts

Authors: Neal Ford, Mark Richards, Pramod Sadalage, Zhamak Dehghani

Release date: October 2021

Publisher: O'Reilly Media, Inc.

ISBN: 9781492086895

Publisher link: https://www.oreilly.com/library/view/software-architecture-the/9781492086888/

‍