Sebastian's blog

Feel the force

Sebastian Hans — Mon, 30 Mar 2026 00:00:00 +0000

An important principle in both martial arts and software development is “feeling the force”. By this I mean sensing the strength and direction of the force an opponent is applying (or is about to apply) as well as the opponent's stability in various directions you might consider applying force in. Or, in software architecture, seeing all the forces at play when preparing for a decision. In both cases, feeling the force enables you to anticipate how it might impact you (in a literal or figurative sense), and allows you to find the best (martial arts or software architecture) move in the situation.

Feeling the force in martial arts

In martial arts, the better you can judge the speed and angle of – and the strength behind – an attack, the better you can position yourself outside the direct line of attack and perform a counter movement that neutralizes it. In software architecture, the better you can judge the social, political, and technical forces affecting the system you are designing, the better you can adapt the shape of the system to go along with or withstand these forces as appropriate.

A classic diagram in martial arts shows how to destabilize your typical two-legged opponent.

“Cross” diagram of an opponent's stance

You apply force perpendicular to the line connecting the two feet. Pulling or pushing in parallel to this line only serves to stabilize your opponent because you end up pushing him onto his own foot, so the force is easily redirected into the ground. However, there is no support on either side of this line, so applying just a little bit of perpendicular force immediately destabilizes your opponent, forcing him to adjust his position or fall over.

Now, in most practical situations, you won't have time to draw a diagram, or to ask opponents to stand still so you can get a good look at their foot position. You have to get the direction right immediately. To get to the point where you can do that, you need to practice. A lot. By doing this, you develop a sense of where the opponent's feet are without looking at them, of the momentum of the opponent's body (which enables you to decide to which side to move), and of the best point in space to position your own body so the opponent has no chance to lean on you to stabilize himself. This is what I mean by feeling the force. You practice until this becomes second nature. At this point, it feels effortless, and it may look like magic to an unskilled observer.

Feeling the force in software architecture

In software architecture, “feeling the force” means being aware of the constraints bearing on decisions you are about to make, of the current trajectory of the system under development and surrounding systems that might potentially have an impact on those decisions, and of the leverage points at your disposal to deal with them. This awareness lets you come up with solutions that fit the constraints and guide the system in the right direction. It can feel as if the architecture “flows” naturally from the context.^[1]

For example: Picture an old enterprise with a heterogeneous IT landscape (partially on prem and partially in the cloud) managed by a strong, centralized IT department, where in-house development is focused on Java/Spring Boot and TypeScript/Angular. The enterprise as a whole is highly focused on compliance such as data protection regulation. Now imagine a situation in which one business unit of this enterprise, a mostly agile bubble in this large, not-so-agile organization, is tasked with developing a new customer landing page with some custom functionality, while it is at the same time trying to build an internal integration platform to support several budding value streams. The landing page is important politically, and there is a deadline – not too tight, but not comfortable either. The business unit has a couple of employees who know just enough Python to be dangerous and has already developed some ideas for building the integration platform in the cloud (even though the requirements are very hazy) before the project has come to the attention of the IT department, which is – of course – stretched for resources. One of the ideas is to use the new landing page as a pilot for the integration platform (which doesn't yet exist).

This is a real situation I found myself in a couple of years ago, when I, as part of said IT department, was in the position to establish myself as a solution architect for this project by dint of being in the right place at the right time. So what does “feeling the force” look like in this situation?

It was clear from the outset that letting the business unit loose with an account on the cloud platform by themselves was not going to work out well. At the same time, we (the IT) didn't have spare resources to staff the project ourselves. Nevertheless, positioning yourself as a blocker is never a good start. So the first thing I did was to talk to them and listen to their ideas. This was to the detriment of other commitments, but it enabled me to establish a friendly working relationship, to gather further information, and to get a feeling for the forces they felt (political pressure, promises they had made to their leadership, motivation to get their hands dirty coding themselves etc). I was observing their stance, so to speak. While this may sound obvious when you read about it (“just talk to them”), other IT architects at the company got quite upset about the plans of the business unit and wanted to have an internal discussion about how “IT should position itself” before confronting the business unit with this position. This is the opposite of feeling the force – trying to react before really knowing anything. Feeling the force requires establishing contact first.

Talking to my colleagues in IT was also part of “feeling the force”. One aspect of this was the self-image of certain colleagues, which boiled down to, “We are the IT experts!” Another was the availability – or rather, unavailability – of spare resources. Some others like the IT tech strategy and security policy I already knew.

After some internal negotiation, I managed to secure a part-time commitment from one of our IT cloud architects for the business unit to ensure a reasonable base setup in the public cloud without any major security issues. I also offered to accompany the project as a solution architect, while at the same time juggling with my other commitments to make this work. The people in the business unit accepted this gladly because I framed it as extending a helpful hand rather than coming down on them with compliance and security regulations. In the eyes of my IT colleagues I was reining in the rogue business unit, which served to appease them.

After some back and forth, we settled on a microservices architecture based on Python for the integration platform part (to enable experimentation by the business unit without too much IT involvement, at least in the beginning) and a backend for frontend built by myself and a fresh graduate of our internal training program on the approved tech stack. This backend for frontend served the dual purpose of

providing a simple facade to the frontend for the functionality it needed and
shielding the potentially not as secure (built by less experienced people using an unfamiliar tech stack) integration services from external access.

I also freed up time by delegating other work so I could take the lead writing the mandatory security concept (including threat modelling and risk assessment) and getting it approved with minimum fuss. In the end, we did manage to get the landing page up in time for the deadline, and development continued within this architectural frame.

How to practice

In martial arts, you learn how to feel the force by practicing the same sequence again and again. The feedback cycle is very quick: you push or pull and see the effect immediately. Did it work? Remember what it felt like and try to reproduce it. It didn't work? Make a slight change to your position, stance, or direction, repeat, and observe the effect. Usually, an instructor will help if you get stuck or have questions.

In software architecture, the feedback cycle is typically much longer, and you get much less repetition. Also, the situation is often not repeatable in the same way – unless you are prepared to discard at least part of the work that has already been done and start over – and how often do you get that chance? Another consequence of this is that there is no instructor who has done the same thing a thousand times before and can tell you what you are doing wrong.

Despite all this, I think you can – and should – practice feeling the forces in software architecture, too. Here are a few ideas:

Close the feedback loop. Although feedback is often slow to come, it is still valuable. Get feedback on your architecture decisions after enough time so that their effects have made themselves felt. In his lightning talk Tech debt nomads and slash-and-burn development, Einar Høst suggests that this time is about 3 years. From my personal experience, I'd say it's more like 2 years for bigger decisions if you are sensitive to feedback. Feedback on smaller decisions can come much faster, but you still have to look for it. It doesn't always smack you on the head. In any case, make sure to look for feedback, and to learn from it. If I leave a team before the trajectory becomes visible, I try to get in touch with someone on the team later and ask them how the architecture has held up. This is not always possible, but when it is, it's always interesting.
Try multiple versions. For smaller decisions like, for example, whether to implement a certain feature using inheritance or composition, it is often quite possible to try both versions and see where this takes you. Writing a little bit of code is cheap, and even if you will discard half of it, what you learned will stay with you and make you a little bit smarter.
Recycle past situations. You may not be able to recreate a situation to practice it again for real, but thinking is cheap. You can always recreate it in your mind and reflect on it. What happened? How did you handle it? What happened as a result? What do you know now that you didn't know then that turned out to be important? How could you have known at the time? In retrospect, what signs were there that you might have missed? Also: What went well, and why? Were you well prepared, or did you get lucky? What could have gone differently, and how would you have reacted to the change – with your knowledge then and with your knowledge now? How could it have gone even better? Asking these questions turns you into your own instructor because having lived through the situation already allows you to see it much clearer, and to come up with your own advice. If you can talk the situation through with someone else (a peer, a mentor), this can help, too, but even if you can't, a little reflection can take you a long way.
Stress-test your decisions. You can exercise feeling the force in your current situation, too, by taking a step back before committing to a decision and asking challenging questions like, “Why are we even doing this?” This is only half a joke. Surprisingly often, the answer is not really clear. But even if it is, you can play around with the parameters the answer likely contains. What if this requirement changes? What if this assumption turns out to be wrong? What would need to happen for this to blow up politically? Etc. Posing these questions can help you feel out the trajectory you are on and bring hidden forces to the surface so you can deal with them.^[2]
Do systems thinking exercises. If you want to get a broader perspective, or if you are new to this and don't have a long history of past decisions to draw on, you can do systems thinking exercises like Ruth Malan's Advent(ure) in System Seeing. The early exercises use generic topics to get you used to various kinds of questions to ask, while the later exercises can be applied to a situation of your own choosing. This can be a work situation, but it doesn't have to be. The advantage of these exercises is that some people make their “solutions” public so you can compare them with your own and with each other and examine the differences. Ruth has links to some examples on her site. My own reflections from last year can be found here.

Conclusion

Feeling the force is an important skill for both martial artists and software architects. Noticing all the forces at play in an encounter enables you to respond effectively, while expending only the necessary effort.

If you get this right, in software architecture as in martial arts, it will feel as if you aren't actually doing very much and things are going fine nonetheless. This might make you nervous. You might feel your training partner is not really attacking properly, or that you should be “architecting more”. But don't be fooled! The goal is not to do much. The goal is to do the right things to ensure the result you want. Indeed, having to exert much effort for little gain is one of the surest signs you are not sensing the forces well (or are not responding appropriately). If you find yourself getting hit again and again despite your best efforts, or if you find yourself mired in complexity despite having already done much architectural work, take a step back and reflect! You might not be taking into account the forces around you. Maybe it is time to practice feeling the force a bit more.

Martial artists practice this skill regularly and with quick feedback that is not easy to ignore (like a fist in your mouth). It's more difficult for software architects due to long feedback cycles and because negative feedback is easier to miss (hopefully!). Nevertheless, you can and should practice feeling the force as a software architect, and after reading this, you don't have an excuse not to. 😁

Footnotes

But consider verbalizing this context and writing it down, e.g., in the form of Architecture Decision Records! Unlike the application of a martial arts technique, which is ephemeral, a software architecture decision is durable and should be documented and understandable later. ↩︎
This is similar to how Residuality Theory tries to come up with better software architectures by applying random simulations of stress to potential decisions, and adapting them to survive these stresses. If you haven't encountered Residuality Theory before, I highly encourage you to take a look. ↩︎

Pattern Radar

Sebastian Hans — Sat, 28 Feb 2026 00:00:00 +0000

In this post, I introduce the idea of the “Pattern Radar” as a means of incrementally improving a legacy code base. If you want to skip right to the practical part, go here.

Intro: working with legacy code

Have you ever had the “pleasure” of working on a code base that has been in development for 15+ years, has had various attempts at improvement perpetrated on it, has survived them all, and is now ready. For. A. New. Feature? To be implemented by you, who just joined the project?

If you haven't, let me give you a taste of what it's like. You start out thinking it can't be too hard; it's only a new attribute in a database entity and a bit of logic over here. Or is it over there? Wait, why are there 5 classes with almost identical names in 3 packages? OK, let's look at the call chain for this one to see where it is used. Invoke the Find Usages function in my favourite IDE, and … what's this? 30 call sites? Erm, the first one doesn't look plausible at all, it's actually calling a completely different service. WTF? Oh, I see, the services are implementing the same interface, and the IDE lists all uses of the interface. So I actually have to look at all of them to see which are the ones I want.

(Fast forward a bit.)

So, I have finally found the correct place to change. I think. Is there a test for this class? Yes, there is. It's an … integration test that connects to an external test database? Hm, OK. What does it do? There are two test methods, one named editObject(), and one named editObjectNegative(). What do they do? They call the editObject() on the service and assert various things. OK, looks like the “negative” one is testing error scenarios, and the other one is testing the happy path, but I'm not entirely sure because some of the assertions seem only tangentially related to the logic under test. My team mates can't remember why they are there either. O. K.

How do they do test setup? Where, for example, is the data coming from? Search, search. Hm, seems to be magically there. A few conversations with veteran team mates later, I know that the test data is maintained in a giant SQL file in a different repository. Now, if I add the data I need there, do I break all the existing tests? No, good, I only broke 50 or so of them.

That's enough of that. This was just to introduce you to the environment that sparked the following idea. Because the thing is, the team working on the application is actually quite motivated. They want to improve the code base. They have already made some attempts at this in the past. For example, I found two different Hibernate entities representing the same class of data, one of which was much cleaner than the other one, which was marked as deprecated. But which one do you think is used by most of the code? Right.

For me as a newcomer, these partial improvements sometimes actually made things worse. Having seen two or three ways of solving the same problem in the (pretty large) code base, how do you decide which one to use when you encounter the same problem again? In the case of the entity class, it was at least clear what the plan was. In other cases, not so much.

So how to improve this?

For some improvements, we wrote backlog items and implemented them in one fell swoop, just to get some big blockers out of the way. For example, we had a mixture of JUnit 4 and JUnit 5 tests in one of the Java backend services, and I used OpenRewrite to convert all of them to JUnit 5 (which worked like a charm, by the way; OpenRewrite is a fantastic tool for this kind of work).

If you can do that, it's great. However, addressing all code quality issues in this fashion is unrealistic. We would be busy for half a year doing nothing but quality improvements – and step on each other's toes for the whole time because most of these improvements, if done in a big bang, touch large parts of the code base, so this would produce an endless stream of merge conflicts. We needed a way to improve the code gradually. This means we needed to ensure

that team members had (i.e., took) time to make improvements during their normal work and
that the team agreed on what actually constituted an improvement.

As to 1., we had already instituted the Boy Scout Rule, “Leave the code better than you found it,” and it was kind of working. If we could just find a way to align on 2., we'd be in a good place. This is where the Pattern Radar comes in.

Introducing the Pattern Radar

The Pattern Radar is inspired by Thoughtworks' Technology Radar. It lists recurring patterns that we find (or would like to find) in our code base, and classifies them according to how we want to deal with them. Our classifications are as follows.

TRIAL: We want to try this. It's not yet clear whether the pattern is suitable for general use.
Expected behaviour: Present trial to the team and together decide how to proceed. Don't apply directly outside of the trial; wait for the experience report. If it turns out not to suit us, we'll roll back the trial.
ADOPT: Tried and tested. We want to apply this broadly.
Expected behaviour: Use when appropriate. Prefer this to patterns in other categories except for special circumstances. Refactor occurrences of PURGE patterns into this when you come across them (Boy Scout Rule).
PURGE: Was used in the past and may still be found in the code base (maybe even broadly), but should be replaced by a different pattern.
Expected behaviour: Don't use in new code! If you come across it, replace it with an ADOPT pattern (Boy Scout Rule).
KEEP: May be found in the code base in certain places. It's OK there, but it shouldn't be used more broadly.
Expected behaviour: Don't use in new code! If you come across it, leave it alone!
BURN: Doesn't occur in the code base, and we don't want to see it ever again.
Expected behaviour: Don't use in new code! If spotted anywhere, replace immediately!

We have not strictly defined what counts as a pattern except that they should be things that are found recurringly in our code base. Some examples are:

Hibernate Typed Queries with JPQL: ADOPT
Hibernate Criteria Queries: PURGE (replace with Typed Queries)
Application logic in mapper classes: PURGE
Composition over inheritance: ADOPT
Testcontainers for DB: ADOPT
Mockist tests (in which all dependencies are mocked): PURGE
Reactive forms (in our Angular frontend): ADOPT

We started by gathering these and other patterns in a table on a wiki page and discussing them one by one. We used the following six columns:

Name: A short name for the pattern (as shown in the examples above)
Classification: TRIAL, ADOPT, PURGE, KEEP, or BURN
Last change: The date when we added or updated the pattern
Description: Description of the pattern in one sentence
Rationale: Brief outline of the reasoning behind the classification (1-3 sentences)
Comments: Additional commentary, e.g., suggested replacements for PURGE patterns or reasons you might not want to use an ADOPT pattern

A full example might look like this:

Name	Classification	Last change	Description	Rationale	Comments
Testcontainers for DB	ADOPT	19.02.2026	Use testcontainers for integration tests that need a database.	We want our tests to be independent from shared infrastructure (pipeline reliability!)
Composition over inheritance	ADOPT	23.02.2026	Reuse code via composition rather than via inheritance.	Composition tends to be more explicit and easier to understand. The code base relies on template methods rather heavily but this has confused devs.	Template methods can still be used in moderation, but we prefer composition.

Benefits

We started this about 2 weeks ago and have already observed a few benefits while discussing the patterns that came up during our initial collection. In some cases, it simply made clear the reason these patterns existed in the code base and helped us align on proper and improper uses.

In other cases, someone described a pattern that irritated them and suggested to purge it, and someone else went, “Oh yes, I've noticed that too, we should probably get rid of that,” allowing us to boy scout them away with a clear conscience.

Once, we discussed a pattern that was very prevalent in the code base and someone suggested a replacement. What a relief! I had been propagating the pattern because consistency in a code base is valuable (sometimes more valuable than small, isolated improvements) and had been unaware that this had been discussed in the past, and the team had already decided to replace it.

Writing down a brief rationale forced us to articulate why exactly a given pattern was to be adopted or to be purged, providing important context, especially to newer team members.

And the discussion of one or two patterns highlighted deep-seated differences in opinion among team members. We still have to resolve these, and I believe this won't be easy, but having put them on the table at least gives us the chance to have a meaningful discussion instead of quietly continuing to work at cross-purposes.

Why not ADRs?

The Pattern Radar captures our decisions about how to evolve our code base. Does this sound similar to Architecture Decision Records? In a way, it is. Actually, we use ADRs, too, but those are more focused on the high-level structures of our system; you know, the things that are hard to change and for the most part only occur once. If we want to change the service decomposition, for example, we will initiate an ADR, discuss options in the team, decide together, finalize the ADR, and then implement it, probably in an incremental fashion, but still in a quite straightforward manner.

In contrast, I envision the Pattern Radar as a light-weight, evolving catalog of the recurring lower-level structures in our code base. To phase out Hibernate Criteria Queries does not feel ADR-worthy. It is a smaller decision that can be implemented in many small, local steps as we go along. The Pattern Radar gives us a place to discuss and record such decisions, and to review them regularly, without introducing much overhead.

Getting started

Setting up a Pattern Radar is very simple. You don't need more than a place where you can write down a simple list or table and share it with your team. This can be a wiki page, a Google doc, a virtual whiteboard, a Markdown file in your source code repository, or wherever your team likes to record things.

Begin by writing down patterns you have noticed in your code base, and ask your team mates to do the same. Then get together and talk about the problems these patterns solve or create, and agree on a classification. That's all you need in the beginning. You can always add more patterns and details or visualizations later.

To actually let the Pattern Radar come to life, however, you also need to adopt the Boy Scout Rule, i.e., agree to make little changes in line with the Pattern Radar as you go along.

What's next?

Currently, our Pattern Radar consists of about two dozen patterns, half of which we have already classified. The format is a simple table as described above. We have not yet created a visualization (the classical radar picture with rings). I'm not sure we will ever do that. It looks nice for the Tech Radar, but as long as we are going to use the Pattern Radar only internally in our team, I don't think it would add much value.

We have not yet defined quadrants because we don't have that many patterns. We will probably do that once we reach a number of patterns where it makes sense to add a bit more structure. Looking at the patterns we have already collected, I can see how quadrants for “backend code”, “frontend code”, and “testing” might make sense. Maybe we'll call them “sectors” or something if we don't get exactly four.

Although we have defined TRIAL, KEEP, and BURN as classifications, we have not used them yet. I imagine TRIAL becoming useful as time goes on and we decide to try new patterns. And as patterns we classified as PURGE slowly vanish from our code base, we might reclassify them as KEEP for patterns that make sense to keep for a limited scope, or as BURN for patterns we have successfully eliminated. I think it will be useful to keep those on the radar lest someone try to introduce them again, but make them easy to filter out so they don't clutter up our table too much.

We will see how it works out. If anything interesting comes of it, I will write a follow-up post.

If you decide to try it out, I'd like to hear about your experience, so please drop me a line!

Reflections on Advent(ure) in System Seeing 2025

Sebastian Hans — Sun, 25 Jan 2026 00:00:00 +0000

These are my reflections on the last Advent(ure) in System Seeing.

Advent(ure) in System Seeing is a series of daily challenges by Ruth Malan, focusing on understanding systems, and playing out from December 1st to December 24th. I enjoyed following along for the second year now and recommend it to anyone who is interested in trying their hand at system thinking.

I posted my results and reflections on Mastodon under #AdventOfSystemSeeing, but thought I'd collect them here for ease of reference – and because Ruth asked me to share a PDF of my zine that was the culmination of last year's effort. You can find it at the bottom of this post.

Thank you, Ruth, for your prompts and the effort you put into them!

Day 1 – Draw a bicycle (and sketch a key mechanism)

Day one was about drawing a bicycle from memory (find the full prompt here).

I did steps 2 (sketch), 3 (explore key capability), and 4 (reflect). Here are my pictures.

My drawing of a bicycle

Focusing on the propulsion system

What I noticed:

I paid more attention to how the parts were connected this time around.
Drawing made me notice I don't know the exact angles between parts of the frame. My drawing looks slightly skewed.
I see what I didn't draw.
Just because you interact with a system often this doesn't mean you understand it.
There's always another layer. I did note the necessity of friction, but I didn't describe how this comes about (gravity, usually).
The system only works at all in connection with its environment. Without the ground, the bicycle doesn't move.

Day 2 – Zoom out to see the world around a bicycle

This was about exploring the context around bicycles (full prompt).

Didn't have much time today, but I managed to knock out a simple context diagram.

Concept map around the topic bicycle

What I noticed:

My map is not hierarchical. There are many connections across contexts.
Some contexts have more depth than others, maybe because I have more experience riding a bike than building one.
Some contexts are more … organized (at scale) than others, e.g., I imagine bike factories to be more strictly organized than how individuals use their bikes recreationally. I had no way to indicate this in the map. Maybe I could have color-coded it?
I realize I forgot to include regulation and inspection (like traffic regulations and the TÜV).

Day 3 – Explore impact of change

Today's task was to draw a Futures Wheel to explore how one change in the context of cycling would ripple out into the environment (full prompt).

Futures Wheel for the change “building more bike lanes”

What I noticed:

More than one first order effect may contribute to a second order effect.
Different first order effects can lead to opposite second order effects.
Second order effects are tricky.

Day 4 – Going behind the scenes

Drawing a rich picture of all that happens when you fill a pot with water (full prompt) was the exercise on day 4.

Rich picture of me filling a pot with water

What I noticed:

I know where the water in Munich comes from, but I don't know the exact mechanics.
The plumber pays taxes in the city, which I didn't draw.
Each of the elements in the picture is a complex system in its own right.
I really don't like drawing that much.

Day 5 – Characterizing systems

This day's task was to design an introductory page of a zine on systems (full prompt). Here's what I came up with.

Zine page on the characterization of systems

I spent waaaaay too much time on this one, but I am pleased with the result.

Deciding what to put on the sheet was the easy part. Coming up with a balanced arrangement was harder, and I didn't quite manage it. That this was supposed to be a zine cover made me want to achieve a much more polished design, which took a long time and a few false starts. I also wanted to tie the concepts together with a unifying theme, the airplane, which worked quite well, I think.

Day 6 – Causal loops with Meadows

This day was about watching part of a lecture by Donella Meadows and taking sketchnotes (full prompt).

This is a very interesting lecture. Here are my notes on minutes 18 to 36. I'll have to watch the whole thing when I have more time. The point that stuck with me the most is that when there are loops within loops, an interesting question is when which of them will dominate the system because this will influence the overall direction the system will take.

Sketchnotes on minutes 18 to 36 of Donella Meadows' lecture, “A Philosophical Look at System Dynamics”

Day 7 – Give causal loop diagrams a spin

After watching Donella Meadows draw a few causal loops, this day was spent on drawing one myself (full prompt).

I chose to explore technical debt. This one is interesting because increasing technical debt initially increases development velocity (that's the whole point), but once a certain threshold is crossed, it begins to decrease it (red arrow). Perceived sucess tends to increase feature priority (over engineering priority), but so does a reduction in perceived success (red arrow). In both cases (initial low technical debt as well as later high technical debt) this can result in a self-reinforcing loop leading to the collapse of the system. Thus, the critical point of leverage seems to be how we approach feature vs. engineering priority in the face of some level of perceived success. At this point, we can break the self-reinforcing dynamic by exercising sound judgement.

Causal loop diagram exporing technical debt

Day 8 – Context changes everything

Drawing a second page for a zine on systems was the task of day 8 (full prompt).

I went with context as the topic and chose a more abstract visualization this time (mostly to keep to the time box).

Zine page on system context

Day 9 – Relationships and interactions

Today, there were two tasks: to watch a video about relationships and exchanges among trees and fungi in mycorrhizal networks, and to draft another zine page on relationships or interactions (full prompt).

I didn't come up with a good focal point for the zine page within my timebox, so I just jotted down relationships and interactions between systems as they occurred to me. If I had more time, I would try to bring them in some kind of order, maybe by similarity or something. What I now notice I missed are indirect relationships, e.g., mediated by some other system.

Sketch of the tree video and draft of a zine page on relationships and interactions

Day 10 – Draw your org 3 ways

As the title says, this day was about drawing an organization I have been part of at least three different ways (full prompt).

I drew my work organization, anonymized so I can share it.

The (almost) classic org chart. As a consultant, I am part of 2 orgs, one of which is my employer (red), and the other one is the customer I am currently working for (blue).

My organization shown as an org chart

The teams, working groups, and communities of practice in which my actual work happens. None of these appear in the official org chart because they are either too small (the Scrum team) or cross organizational borders (the communities).

My organization shown as a Venn diagram of more informal working groups

A calendar view that roughly shows where my time is spent. Unlike the other charts, this one shows the relative importance of various activities. Because I work for a consulting firm, work for the customer dominates my calendar, although I am also engaged in various internal activities.

A calendar view of what I spend my time on

Day 11 – Describe your focal situation

On this day, we moved from generic systems exercises and ruminations about common systems to exploring a specific situation in our own lives (full prompt).

Reflecting on this, I find that deciding on my focal situation for the upcoming days was easy. I've been thinking intensely about the situation the team in my current customer project is facing anyway. On one hand, writing down all the things bouncing around in my head was slightly depressing because it showed how many unresolved issues – major issues – there are. On the other hand, I can see none of them are unsolvable, even if it will take a long time to address them.

Day 12 – Sketch the situation

Today, the task was to either draw a rich picture or an actor map of the focal situation chosen yesterday (full prompt). I chose the actor map.

What I noticed: Drawing the situation – the software system we are responsible for, a number of construction sites within the system, and the actors surrounding and complementing it – made me think a bit harder about where to place each actor in relation to the whole. I realized the motivation of some of the actors is not that clear (to me). Most of the contractors don't seem to have strong motivation apart from ensuring their contracts are extended. This is both a boon and a hindrance. A boon because they are unlikely to oppose potential improvements; a hindrance because they are equally unlikely to contribute much energy. This means what leverage I need I'll have to find in the others, and I “just” have to make the changes I want easy for the contractors to go along with. Well then, problem solved. 😉

Day 13 – What if you do nothing (different)?

The task was to reflect on the “characteristics of the current state of things” of my situation and to think about what was likely to happen if I didn't actively change anything (full prompt).

In my case, the most relevant factors influencing the default future situation likely are:

reverting to the mean,
institutional inertia,
central policy forcing certain issues,
cost of change for established communication patterns,
distribution of experience in the team.

Day 14 – How we got here

Today was about exploring how the situation came to be, either in textual form, or in the form of a graphical history (full prompt).

I tried to assemble a graphical history. This was difficult because the history of the project I'm working on goes back more than 10 years and there have been many changes to the team. The documentation is … not great as far as significant decisions are concerned, so the historical information is disjoint at best, and in large parts unrecoverable. I myself joined 6 months ago and my efforts to uncover the rationale behind certain … interesting decisions were not very successful. This in itself is telling.

Day 15 – Mapping value flows

Today, we focused on value exchanges in the our focal situation, and visualized them in the form of a Value Network Map (full prompt).

Mapping the value flows for my current project was interesting because it is an internal service that provides basic data to many other services. Hence, on the surface, the primary value flows are very simple. But actually, the team providing this service is part of a rather large network of other teams and communities, some of which provide direct support to each other, some are formally responsible for certain aspects (e.g., a centralized architecture team), and some are more informal in nature. It is not always obvious (to me) what each party gets out of the interactions I have observed so far. Putting this on paper clarified some things. I'll have to dig deeper on the others.

Day 16 – Another distillation

Taking a break from our focal situation, this day was about designign another page for the systems zine (full prompt).

I followed the suggestion of illustrating constraints. I found this rather hard – constrained by my drawing skills, time, and mind space more than by the medium.

A zine page on constraints in systems

Day 17 – Systems and boundaries

Back to our focal situation, today's task was to “identify systems, list them, or draw (and name) tne systems, and their interrelationships” (full prompt).

The page I used to draw my diagram quickly got too small. In the end, I touched on everything from the technical components of my system of interest over the team composition, organisational structures of the German government, technical infrastructure down to the electrical grid, my own company, the labor market which enables me to work for it, the contracting system which allows me to work for this customer, the economy underpinning both, the food supply chain (gotta eat something!), the (US) companies providing the software I use to work on the system remotely, and world politics up to the solar system and astrophysics. After all, where would we be without the sun?

Oh, and, yes, we have users, too. 😉

I'm not sure if I should be glad or sad that, tomorrow, I'll probably be focusing on breaking the dependency between our unit tests and a centrally provided test database. 😵‍💫

Day 18 – Exploring capabilities and boundaries

This day's task was to draw the network of capabilities that enable the system identified yesterday “to be viable as a system with that purpose” (full prompt).

Reflecting on this task, I realize that most of the capabilities required for the system to fulfil its purpose are external to the system – or at least out of scope of the team nominally responsible for the system. This has already made some things more difficult than they should be.

Day 19 – Exploring constraints and forces

The focus for today was on action – to identify an area where we wanted to do something (the problem), and to explore relevant constraints, forces, assumptions, and approaches (full prompt).

Reflecting on this task, I can say mind mapping on paper does not work for me. I always end up needing more space in places where I did not plan for it, resulting either in very crammed corners or spaghetti connections. So I made a table instead, with columns for constraints, forces, assumptions, and approaches. I linked constraints with the forces they generate and assumptions I made about them. I marked each force assisting my endeavor with a green plus sign and each resisting force with a red minus sign. I also linked forces and assumptions with possible approaches that build upon them or try to counter them, and I included my assessment of the efficacy of each approach. I did this iteratively, adding entries to all columns as they occurred to me.

This is what it looked like:

Table of constraints, forces, assumptions, and approaches pertaining to a particular problem

Initially, I came up with a lot of resisting forces, until I realized that I myself and my current role in the team could count as forces, too. Now it's even, which matches my gut feeling. It could go either way.

Filling the first three columns enabled me to think about possible approaches in a more structured way. Not all approaches are mutually exclusive, and they address different subsets of the forces. Most importantly, I noticed an important gap: some of the positive forces hadn't been properly utilized yet. Thinking about this enabled me to come up with a new approach that might have the best chance of success.

This may well have been the most immediately helpful exercise up to now.

Day 20 – Reflection

The task was to go back over the work of the past 19 days and extract concepts related to systems; to identify the lenses we've used in system seeing; to come up with questions and a next step; and to think about what topic we would like to add (full prompt).

I drew a concept map.

Concept map of system concepts and lenses

Two things I added to my map that weren't explicitly in focus on days 1 through 19 is seeing what's there and seeing what's missing. Seeing what's there means not being content with the first impression and jumping to conclusions but looking closer for what's really there, e.g., do we really have cause and effect here or is it just correlation? Seeing what's missing means noticing gaps in our understanding or in arguments, and working to fill them.

Day 21 – System concepts, take n+1

Today was about designing another page or two for the systems zine (full prompt).

At this point, I was getting a bit fed up with all the drawing, but in for a penny, in for a pound, right?

Draft of a zine page on boundaries

Day 22 – Shifting outcomes

The task was to draw an Impact Map for a challenge in our focal situation (full prompt).

This exercise felt very similar to “Exploring Constraints and Forces” on day 19. When I squint a bit, the goal here looks like the problem there, the actors here form a subset of the constraints there, the impacts here are roughly equivalent to the forces there, and the responses here look like the approaches there. Assumptions are not explicitly called out here, although some of the impacts I listed are indeed assumptions rather than proven facts.

In order to not just copy everything over, I chose a different goal to explore today.

What's new here is the last paragraph where we ask about side-effects and explore further beyond the right side of the diagram. We could do that with the other one, too, however.

Day 23 – Notice what we add

Focusing on a recent conversation, the task was to write down what was actually said in one column and what I heard/thought in a second column, and to reflect on this (full prompt).

There were definitely “differences between [my] external dialogue and internal thoughts and feelings.” I think this is necessary to a certain degree. In general, you don't want to bleat out the first thing that comes to mind in a professional setting. In the situation I had in mind, I had certain suspicions about the motives of the other party, but this was not the moment to address them. There will be a meeting at the beginning of the new year where the cards will be put on the table. We will prepare for this and hopefully have a fruitful discussion then.

Day 24 — Adventure zine

The final exercise was to turn the zine pages designed over the past days into a physical zine (full prompt).

Doing this on Christmas Eve with 2 kids? No chance! On the 25th, they are busy with their presents, and I have some free time. I admit, I groaned every time I encountered a “make a zine page” exercise because drawing does not come naturally to me, but now I already had 5 pages I wanted to finish it. So I added a title page and created two additional pages to illustrate all the tools we used during the course of our Advent(ure). I am quite pleased with the result. I intend to keep it on my desk as a tangible reminder to practice system seeing more often, and as a quick reference.

Front cover

Pages 1 and 2

Pages 3 and 4

Pages 5 and 6

Back cover

Advent(ure) in System Seeing zine

If you want to print my zine, here is a PDF file with all pages in the correct order: Adventure_in_System_Seeing_2025_Zine.pdf.

Because my printer can only handle A4 pages, the PDF file contains 2 pages. I printed them on A4 paper, cut off the margins, and taped the pages together with transparent tape. The right border of the first page (with the title and constraints pages) must be attached to the left border of the second page (with the context and system pages). The resulting large page can be folded into an 8 page zine by following the instructions from Ruth's prompt.

Have fun with it, and let me know what you think!

Clean the f… up!

Sebastian Hans — Wed, 21 Jan 2026 00:00:00 +0000

My new motto is, Clean the f… up!

The “f” is for files, of course.

By cleaning up, I mean not leaving obsolete, unused, temporary stuff lying around. A house is finished not when the walls are complete but when the scaffolding has been removed and the dirt has been swept up so people can live in it without constantly tripping over rubble. Similarly, software is finished not when the tests pass but when all temporary experiments, exploratory meanderings, and temporary workarounds for unfinished parts have been cleaned up – deleted, maybe archived, but in any case removed from the sight of the maintainer who will have to work with the results.

A refactoring is finished not when the new code is operational, but when the old code has been deleted. Not commented out! You are using a version control system, aren't you? If so, deleted code is not lost anyway. There is no point in unused code cluttering up your workspace.

I've heard people say we should hang on to old stuff because there is no harm in it, and “what if we need it again some time?” Well, let me show you some examples.

Just recently, I wanted to make a change to the monitoring setup at a customer project. The change went smoothly in the dev environment, but deployment failed in the production environment. This is a Kubernetes cluster, and applications are installed via Helm. The Helm charts are pulled from an internal registry. The step that failed was the login to this registry. An hour and conversations with 3 different people later, the problem was solved. One of the people I talked to had sent me down the wrong road by suggesting that the credentials we had configured were obsolete; we should authenticate with this other user instead. Actually, the solution was even simpler. You see, during a platform migration half a year ago, the systems had been wired up in such a way as to render the explicit login unnecessary. It had kept working for a time, but apparently now there had been a permission cleanup, and the login stopped working. Simply deleting the login step made the deployment work again.

Actually, the team knew about this. One of the people I talked to was an engineer on my team, and he gave me the hint that allowed me to fix the problem. Cleaning up the deployment process at the time of the migration half a year ago would have cost 5 minutes of engineering time. They had to change the very same file anyway. Doing it now cost about 2 hours and blocked other work. As I said, …

Clean the f… up!

Another thing I am currently working on is improving the performance and stability of the test suite. This project uses a whole lot of integration tests with a containerized database. The tests are based almost exclusively on a common set of test data in the form of a large (large!) SQL file. The file has gotten pretty big and unwieldy (did I mention it was large?), and if a test fails, finding the relevant input data is a major pain in the ass. It is also slower to load into the database than I would like. I'm pretty sure most of the data in the reference data SQL file is not actually used, but the team has a culture of keeping stuff around “in case it is needed again.” Due to this, we are now at a point where it is difficult to tell which parts are needed and which parts aren't. Keeping something like this manageable is much easier if you don't keep unused stuff lying around, so …

Clean the f… up!

A code base I know uses Hibernate ORM at the persistence layer. One of the entities exists twice in the code base. Both implementations read the same database table, but contain a different set of relations. One of them has been deprecated 2 years ago and sports a comment stating that it has evolved into a god class and should not be used; usages should be replaced by the other class. Can you guess in how many places this class is used versus the other one? Right.

Eliminating usage of the deprecated class is not trivial, I grant you that. The replacement is not a drop-in; you need to refactor any code that uses it, which is not a fun exercise. But you know what's worse than sticking with a badly designed class? Having to deal with a badly designed class plus another class for the same thing in the long term.

Clean the f… up!

One customer uses a vulnerability scanner to scan all containers running on the production Kubernetes cluster. One day two months ago, we got an alert for a dependency. The normal process after receiving such an alert is to update the dependency to a non-vulnerable version. In this particular case, the alert was for an indirect dependency, and there was no clear upgrade path because it was no longer actively maintained. Figuring out what to do took some time. In the end, I noticed that the dependency was only required by obsolete code (a part of the service that was no longer in use). The solution was to remove the dead code – and the dependency, of course. Again, figuring out which parts were really unused and which parts were still necessary was not trivial. It would have been much easier if this had been done at the time when the part of the service had been decommissioned by the people who were working on it. It took me much longer, plus we had vulnerable software in production for no good reason.

Clean the f… up!

Keeping obsolete, unused, temporary stuff around is not free! The cost may not be immediately obvious because leaving it there feels like being faster in the moment. But the cost (in terms of engineering effort) of cleaning up later is almost certainly higher than the cost of cleaning up right away. So do yourself – and your organization – a favour and clean the f… up!

Thank you.

Transcript: Programming as Theory Building and the Not Invented Here Syndrome (english)

Sebastian Hans — Sun, 28 Dec 2025 00:00:00 +0000

This is the translated transcript of my talk “Programming as Theory Building and the Not Invented Here Syndrome” at the InfoDays: Softwarearchitektur 2025. It is a rather direct translation, so expect some rough edges.

(Das deutsche Originaltranskript ist hier.)

Intro

Hello, welcome to my talk “Programming as Theory Building and the Not Invented Here Syndrome”! My name is Sebastian Hans. I have been working in software development for 20 years, most of the time as a software architect and developer in development teams; and this is the setting for the ideas you are about to hear from me.

Currently, I am a Senior Technical Consultant at bbv and accompany software and digitalization projects for our customers. When you do this for a while, sometimes you see things go well, and sometimes they get a little bumpy, occasionally due to technology, but often due to people, too. And that is what we will deal with in this talk, “Programming as Theory Building and the Not Invented Here Syndrome”. In it, I will first briefly introduce the essay “Programming as Theory Building”, then the Not Invented Here Syndrome, and then I will examine the connections and show situations in which they influence the daily life of development teams.

Programming as Theory Building

Programming as Theory Building is an essay by Peter Naur. It is 40 years old now, but its topic is still just as relevant as it was in the 80s. I recommend to everyone to read it themselves some time; there is a lot of wisdom in it. For our purposes here, it is enough to know the core thesis: a program does not consist of its source code, but of a theory. And programming does not consist of translating facts from the real world into source code, but of building this theory. The theory – and thus the program – lives in the heads of the developers.

(A small side note: In the following, I am going to talk about “programs” because Naur's essay uses this term. But everything applies to systems and system compositions as well – just to make that clear.)

Therefore, to really understand a program in this sense, reading the source code is not enough; and neither is looking at the documentation; you somehow have to get at this theory.

The term “theory” that is used here does not only refer to factual knowledge, e. g., what the code does, where to find which functionality and so on, but also the ability to explain it, to justify it, and to connect it to the real world. You can imagine the code to be like the shadow of a real object. Here we see a shadow. If I only look at this, I can see it is a circle. But this on its own does not constitute a theory, because I have no idea what creates the shadow. And there can be quite different theories. It could be a sphere, for example. Or a cylinder. Or a cone or an Easter egg. If I am in possession of the theory, I can explain why the shadow is circular. Let's take the cylinder as the theory. Then the reason is because it is being illuminated along its axis. And if someone comes along and says, “Look, our requirements have changed by 90°, make sure the shadow matches!”, then I only ask for the axis, and if it is the vertical axis, then I say: “OK”, and – boom – I am done. Someone who is not in possession of the theory and only sees the circle cannot know what needs to be done – and they also cannot somehow reconstruct this knowledge from the circle. That simply is not possible.

Whether this theory is present or not is especially noticable when changes or extensions are needed. There are usually many ways to somehow shoehorn in a requirement, but often only a few that fit the theory. If you choose one of those, the change is relatively easy and the result remains easy to change and maintain. If you do it differently, you create an unmaintainable pile of code that might even work if you are lucky.

An example

An example from my own practice: For a few years, I was in a team that was responsible for a payment component. This component provided a unified interface through which apps could process payments from customers. It had different payment service providers connected on the back end for different payment methods. The payment component has a ports-and-adapters architecture, which means, in short, it separates domain logic from technical connections to surrounding systems, which live inside adapters.

At one time, a new colleague joined the team and was given the task of implementing a new, additional payment flow. To do this, he made extensions in the domain and in the adapter for the affected payment service provider. Now I have to say that, for various reasons, the boundary between generic domain logic and specifics for the payment service provider, which belong in the adapter, is not that sharp. Of course, the ports-and-adapters architecture itself is documented, but that does not help you decide exactly which piece of the implementation to put in which box. This is part of the theory that lives in the heads of the developers and that you cannot directly find anywhere else. Long story short, the new colleague did not fit the feature in the way it was intended. He pulled too many adapter specifics into the domain code, making this part too complex, and we then rearranged it together quite a bit to simplify subsequent changes. Doing this, we achieved two things. On the one hand, we brought the changes into alignment with the theory of the payment component and thus quite practically improved the maintainability of the software. And on the other hand, the colleague brought another piece of the theory into his head and can thus better drive the development forward within the bounds of the theory.

So again: The core thesis of Programming as Theory Building is that a program is more than the source code plus documentation. Rather, the program consists of the underlying theory and lives in the heads of the developers. Source code and documentation show, at best, a small part of it.

So much for “Programming as Theory Building”. Let's move on to the other concept, the Not Invented Here Syndrome.

The Not Invented Here Syndrome

Not Invented Here refers to the tendency to reject things that one has not discovered or invented oneself. In the context of software development, these can be tools, processes, programming paradigms, or indeed also a theory that underlies a program. This is not a formally defined term but rather an observation from practice, but there are studies that show that the effect is real. For example, this study from 2014. In it, researchers investigated how organizations conduct crowdsourcing, that is, ask the “crowd” – external people, customers, website visitors, etc. – for improvement suggestions; and in particular, how they then evaluate and implement them. To do so, they measured various types of distance and found, for example, that suggestions that were closer to what the organizations were already doing anyway received more attention than genuinely new suggestions that came from somewhere in the crowd – which were Not Invented Here. What is Not Invented Here is not that interesting to organizations.

I have experienced this myself as well. One of the clients of the payment component (the one I already mentioned) was an SAP system. But it was not connected the same way as all the other clients. When this connection was being planned, there was a dispute between those responsible for the payment component and those responsible for the SAP system because the SAP people said that the flow as the payment component team envisioned it (and as the other clients had implemented it) would not work with SAP; it would contradict the SAP logic. Not Invented Here. We can't do that and we don't want to.

Because this Not Invented Here reaction is usually adverserial, it has a somewhat bad reputation. It is also called “Not Invented Here Syndrome”, and a syndrome is fundamentally something bad. But the Not Invented Here reaction, as I would like to call it more neutrally, does not necessarily have to be bad. A small dose of Not Invented Here is good and even necessary, especially if you take Programming as Theory Building seriously.

Not Invented Here as a sensor

To understand this, I would like to return to the idea of distance used by the study. Very close is almost Invented Here, far away is definitely Not Invented Here. For the purpose of this talk, I will take a closer look at the distance of an idea or concept to the theory of the program. I'll call it “conceptual distance”. The distance is 0 if an idea lies within the bounds of the theory. And if a concept does not fit in with the theory at all, then I say the two have a high conceptual distance. From the perspective of the theory of the program, such an idea would be Not Invented Here.

If you then think of a program – in the sense of theory + code – and an idea for a change, then it would be good if the idea had as little conceptual distance as possible to the theory. Because then it would usually be easy to implement, relatively low-risk, and would leave the program in a good, maintainable state. Whereas a change that was far away from the theory would be rather difficult to implement and error-prone and could potentially destroy the conceptual integrity of the program and thus render it unmaintainable. Hence, it is completely rational to view ideas with a high conceptual distance to the theory of the program more critically than those that are closer to it – which is exactly what “Not Invented Here” is about. From the perspective of the development team, Not Invented Here is basically a protective shield for the conceptual integrity of our theory. Or a sensor that tells me, “Watch out, this is something that does not fit our theory.”

Does that mean Not Invented Here is always good and we should make a habit of rejecting ideas that are Not Invented Here? Of course not. Blanket statements are always wrong. For one thing, there are many different types of distance where one could cry “Not Invented Here”; distance based on group membership, for example. Something like that is usually bad. That is not what I mean here. I exclusively mean conceptual distance.

And there is another reason why I should not categorically reject things that are Not Invented Here: the theory of the program may not be perfect. From time to time, the team learns something new and adapts the theory. This is only possible by integrating ideas that lie outside the bounds of the theory.

Dealing with new ideas

So what do I do if I am a developer in an established development team and someone comes along with an idea or does something in a way that triggers my Not Invented Here sensor? First, I quickly check whether it has triggered for a good reason. Is it the conceptual distance from the theory? If it is not, then maybe it is a false alarm and I should mute it and just continue listening to the idea. But if I find that what I am hearing or seeing really is far away from the theory of my program, then I can rejoice because then the sensor has worked. It has prevented us from blindly putting something into our program that will cause problems later.

The simplest solution is to say, “No, we won't do that.” This saves time and money, and keeps the program tidy and small. But of course you cannot and do not always want to do that, because behind many ideas there is a good intention. Then the task consists of reducing the distance to the theory as much as possible. Is there a way to represent what you actually wanted to achieve with the idea within the existing bounds of the theory? This is the best case; then you should just do it that way. Remember the new colleague in the payment component team. He wanted to build a feature and tried it in a way that did not fit the theory, and the team then remodeled it together to work well within the existing theory.

If we cannot bring the idea conceptually closer to the theory, but we still want to implement it, we can do that, but it will be more work. Because then we have to adapt the theory to the idea. And that means not just writing new code, because the theory is more than just code. We have to bring the whole team along and together work out a new, consistent mental model that reconciles the existing theory and the new idea; and then we can implement that. The main task here is to establish a shared understanding of the new theory. Writing the code to match afterwards is usually much easier.

Another example

I also have an example for that, from the same payment component. We once had to replace a payment service provider. The contract with the old one had expired, a new one had been found as part of a tender process and had to be integrated, without downtime. We already had a way to configure the payment service provider per app, to send the payments for each app to the appropriate adapter. That means, following the theory of the program, we could have switched the apps over via a simple configuration change at the time of migration, and we would have been done. But since many customers were affected, a lot of money was at stake, and big-bang migrations are quite risky, I had the idea of performing the migration per customer instead. This way, we would have much more fine-grained control, and we could significantly reduce the risk.

However, the conceptual distance to the existing theory was relatively large, and so – even though the idea came from me, that is, from within the team – we carefully considered whether to introduce it like that and how to marry this concept of migration per user with what was already there. Ultimately, we decided to completely remove the configuration per app, implement the migration per user, and remove it (along with the old payment service provider) after the migration was finished. The last part may surprise you, but it had to do with the new theory we had formed on the subject of routing payments to payment service providers.

Old theory:

Routing is not really proper logic. It is enough to have a routing table somewhere that can be changed arbitrarily at runtime.

New theory:

Routing is a core function of the payment component, and changing it is a non-trivial undertaking that will probably involve custom logic and edge cases that are best represented in code.

If we had not taken the time to adapt the theory, but simply bolted the migration per user on top … I don't even want to imagine what that would have come of that. Certainly nothing that would have been easy to maintain and extend.

Dealing with existing theories

Let's look at the whole thing from a different perspective, namely from the point of view of someone who wants to or has to deal with someone else's theory. This could be, for example, a piece of software that someone else has developed and where something needs to be changed now. And let's also assume that the conceptual distance to what I am otherwise used to is medium to high. Then I will initially have difficulties understanding the theory behind it. I won't be able to carry out the change to the software in such a way that I can be sure the result will be good. The theory is too far away. In other words, my Not Invented Here sensor is being triggered. But that does not mean I should fall into the Not Invented Here defensive posture. Because the consequence of the Theory Building thesis is: if I don't want a half-baked kludge, then I first have to obtain the theory of the program. I have to get it into my head. How to do that depends on the situation.

Onboarding to an active development team

One possibility is that I am joining an established development team. Then I can assume that the people already have a theory in their heads, but I don't know it yet. I only see the source code at first. Then the first thing to do is to get out of the defensive stance! The most unproductive thing to do would be to say things like, “That's strange! How could anyone come up with such an idea? I would have done this completely differently.” And so on. This would only prevent me from completing my main task, which is to internalize the theory of the program as quickly as possible. What helps is

talking to the team,
reading documentation,
talking to the team,
reading code,
talking to the team,
maybe starting with coding, using pair programming if possible, but at first less to adapt the code, and more with the goal of learning.

Oh yes, and talking to the team.

I've encountered such a situation recently with a service that handles historicized data. In this service, attributes of some entities can change over time and they are stored with “from” and “to” dates. Others are not. Some relationships between entities are historicized, too, and some aren't. I can see in the code that this is so, and the internal wiki contains some amount of documentation, but I only really understood the why behind it in conversation with the team. I could have said directly, “This is all way too complicated, I would have built it completely differently.” Not Invented Here. But that would not have helped me understand the topic.

So whenever I'm new in a team, I switch off the Not Invented Here defensive posture and try to absorb the theory as quickly as possible.

Dealing with legacy software

A different situation would be taking over responsibility for an existing system where nobody is available to answer questions. This can have different reasons. Maybe it is an old system where nothing has had to be done for a long time, and the people with knowledge of it have since moved on. And now there is something to do again and I'm the lucky guy who gets to do it. Or maybe a team exists, but it no longer consists of the original developers but of people who joined later and have not internalized the theory themselves. I know the experience – maybe you know it too – of joining a team that does not know its own system – or only parts of it. Then it gets difficult; then you have to reconstruct the theory. This is hard work.

And with this work, too, my Not Invented Here sensor or the idea of conceptual distance can help. Because it tells me what I still need to investigate. From the start, I already have a certain idea of what the theory could look like, even if it is only my intuition based on my experience with other programs. When I then look at the material that is there – code, diagrams, documentation, whatever – I see things where I think, “Yes, OK, fine.” And I see things where my distance sensor triggers – the one for conceptual distance. Then I think, “Wow, that does not fit at all with what I have pieced together so far!” Some developers then fall into the Not Invented Here defensive stance and grumble about the legacy crap they have to deal with. I take that as a signal to take a closer look, because this is a place where my theory is obviously still wrong or at least incomplete. So I dig in further and try to understand exactly that spot and reconstruct the original theory a bit further.

This works up to a certain point. But usually not completely. When you have reached that point, then you can – and should – begin to rebuild the theory to suit the active developers, to get out of this “I have no idea what I am doing” mode. The result is a theory that is actually alive in the heads of the developers, and you can continue working with it.

An example of this is a developer team that inherited a Java library consisting of five layers. Nobody knew the purpose of these layers, and some developers had exactly this Not Invented Here reaction where they didn't want to touch the thing at all. When I joined the team, we looked at what problems we actually had and where layers helped us and where they didn't; and we have now started to change the code according to our new theory.

Reprise

So you see, there is a connection between the idea of Programming as Theory Building by Peter Naur and Not Invented Here. To summarize briefly, Programming as Theory Building asserts that the essence of programming consists not of writing code, but of building a theory. This theory lives in the heads of the developers and is the actual program. The code is part of it, but it only ensures that the program becomes executable on a machine. Changes to the code that are in harmony with the theory are much easier to carry out and lead to better results than changes that are made without regard to the theory – because the program consists mainly of the theory.

Not Invented Here is a reaction to ideas that come from far away, where the distance can lie in different dimensions. This reaction is often dismissive and counterproductive, but it does not have to be that way.

One dimension that is particularly important for development is the conceptual distance. How far apart are two concepts or ideas or theories? According to the Theory Building thesis, program changes with a high conceptual distance to the theory are precisely the problematic ones. And this is why the Not Invented Here reaction can be useful, because it alerts me to this situation.

When I have learned to get rid of the automatic defensive stance that is often associated with it, and view it as a signal instead, then I am free to consider whether and how I want to reduce the conceptual distance. Maybe I don't want to do this at all and say, “No, I won't do that!” But if I want to do it, I have essentially three possibilities: I can adapt one side or the other, or I adapt both and land somewhere in the middle.

We have looked at a few situations: when onboarding a new developer, the two relevant concepts are the theory of the program and the preconceived ideas and notions in the developer's head. In this case, I usually want the developer to first understand the theory of the program and later bring in their own experience to improve the theory.

With change proposals, two things can happen. Either you manage to implement the requirements within the bounds of the theory of the program – that is the best case – or you actually have to adapt the theory of the program to integrate the solution idea. But then really change the theory, not just isolated pieces of code!

When taking over a legacy system where the theory is no longer accessible, you will often be unable to fully reconstruct it. But you should try, and then adapt the remaining parts to your own ideas and form a new theory so that you can work with it properly.

So, the next time you encounter a Not Invented Here reaction, in yourself or in others, think about what caused it and whether it might not be a signal to engage more deeply with the topic!

Thank you and happy programming! Or theory building!

Transkript: Programming as Theory Building and the Not Invented Here Syndrome (deutsch)

Sebastian Hans — Sun, 28 Dec 2025 00:00:00 +0000

Dies ist ein Transkript meines Vortrags „Programming as Theory Building and the Not Invented Here Syndrome“, den ich auf den InfoDays: Softwarearchitektur 2025 gehalten habe (leicht bearbeitet).

(English translation here.)

Intro

Hallo, willkommen zu meinem Vortrag „Programming as Theory Building and the Not Invented Here Syndrome“! Mein Name ist Sebastian Hans. Ich bin seit 20 Jahren in der Software-Entwicklung tätig, den allergrößten Teil davon als Software-Architekt und Entwickler in Entwicklungsteams, und das ist auch das Setting für die Ideen, die ihr gleich von mir hören werdet.

Aktuell bin ich als Senior Technical Consultant bei der bbv und begleite Software- und Digitalisierungsprojekte bei Kunden. Wenn man sowas länger macht, dann sieht man Sachen, die gut laufen, und Sachen, die holpern, manchmal in der Technik, oft aber auch bei den Menschen. Und mit denen beschäftigen wir uns eben genau in diesem Vortrag „Programming as Theory Building and the Not Invented Here Syndrome“. Dazu werde ich zuerst kurz das Essay „Programming as Theory Building“ vorstellen, dann das Not Invented Here Syndrome und dann werde ich die Zusammenhänge untersuchen und Situationen aufzeigen, in denen die in der Praxis von Entwicklungsteams eine Rolle spielen.

Programming as Theory Building

Programming as Theory Building ist ein Essay von Peter Naur, das mittlerweile 40 Jahre alt ist, aber inhaltlich immer noch genauso gültig wie in den 80ern. Ich kann jedem nur empfehlen, es bei Gelegenheit selbst zu lesen, da steckt eine ganze Menge Weisheit drin. Für unsere Zwecke hier reicht es, die Kernthese zu kennen. Die lautet nämlich, dass ein Programm nicht aus seinem Source-Code besteht, sondern aus einer Theorie. Und programmieren besteht nicht darin, dass man Sachverhalte aus der echten Welt in Source-Code übersetzt, sondern darin, dass man diese Theorie aufbaut. Die Theorie – und damit das Programm – lebt in den Köpfen der Entwickler.

(Kleine Randnotiz: Ich werde im Folgenden immer von „Programmen“ reden, weil Naurs Essay diesen Begriff verwendet. Das gilt aber immer auch für Systeme und Systemverbunde – nur, damit das klar ist.)

Um ein Programm in diesem Sinne wirklich zu verstehen, reicht es also nicht, den Quellcode zu lesen, und auch nicht, die Dokumentation anzuschauen, sondern man muss irgendwie an diese Theorie kommen.

Der Begriff der Theorie, der da verwendet wird, bezeichnet nicht nur Faktenkenntnis, zum Beispiel, was tut der Code, wo finde ich welche Funktionalität und so weiter, sondern darüberhinaus auch die Fähigkeit, das zu erklären, zu begründen und einen Bezug zur realen Welt herzustellen. Man kann sich das so vorstellen, dass der Code wie der Schatten eines realen Objekts ist. Hier haben wir einen Schatten. Wenn ich mir nur den anschaue, kann ich sagen, das ist ein Kreis. Damit habe ich aber noch keine Theorie, weil ich keine Ahnung habe, was den Schatten erzeugt. Und es kann ganz verschiedene Theorien geben. Es könnte zum Beispiel eine Kugel sein. Oder ein Zylinder. Oder ein Kegel oder ein Osterei. Wenn ich die Theorie kenne, dann kann ich erklären, warum der Schatten kreisförmig ist. Nehmen wir mal den Zylinder als Theorie, dann liegt es daran, dass er entlang seiner Achse beleuchtet wird. Und wenn jemand daherkommt und sagt: „Pass auf, unsere Anforderungen haben sich um 90° gedreht, sorg dafür, dass der Schatten dazu passt!“, dann frage ich nur, um welche Achse, und wenn es die vertikale Achse ist, dann sage ich: „OK“, und – zack – bin ich fertig. Jemand, der die Theorie nicht kennt und nur den Kreis sieht, der kann nicht wissen, was zu tun ist – und er kann das Wissen auch nicht irgendwie aus dem Kreis rekonstruieren. Das geht einfach nicht.

Ob diese Theorie vorhanden ist oder nicht, merkt man also vor allem, wenn Änderungen oder Erweiterungen gebraucht werden. Es gibt meistens viele Möglichkeiten, eine Anforderung irgendwie reinzupfriemeln, aber oft nur wenige, die zur Theorie passen. Wählt man eine davon, ist es relativ einfach und das Ergebnis ist auch weiterhin gut änderbar und wartbar. Wenn man es anders macht, erzeugt man einen unwartbaren Haufen Code, der, wenn man Glück hat, funktioniert.

Ein Beispiel

Ein Beispiel aus meiner eigenen Praxis: Ich war ein paar Jahre lang in einem Team, das eine Zahlungskomponente verantwortet hat. Diese Komponente hat eine einheitliche Schnittstelle bereitgestellt, über die Apps Zahlungen von Endkunden abwickeln konnten, und sie hatte hintendran unterschiedliche Zahlungsdienstleister für unterschiedliche Zahlungsmittel angebunden. Die Zahlungskomponente hat eine Ports-and-Adapters-Architektur, das heißt, kurz gesagt, sie unterscheidet zwischen der Domänenlogik und technischen Anbindungen an Umsysteme; die stecken in den Adaptern drin.

Im Laufe der Zeit ist ein neuer Kollege ins Team gekommen und hat die Aufgabe bekommen, einen neuen, zusätzlichen Zahlungsablauf zu implementieren, und er hat dazu Erweiterungen in der Domäne und in dem Adapter für den betroffenen Zahlungsdienstleister vorgenommen. Nun muss ich dazusagen, dass die Aufteilung zwischen generischer Domänenlogik und Spezifika für den Zahlungsdienstleister, die in den Adapter gehören, hier aus verschiedenen Gründen nicht so ganz eindeutig ist. Die Ports-and-Adapters-Architektur ist natürlich dokumentiert, aber das hilft einem nicht bei der Entscheidung, was man genau in welches Kasterl reinsteckt. Das ist Teil der Theorie, die in den Köpfen der Entwickler lebt und die man nicht direkt irgendwo anders herauslesen kann. Langer Rede, kurzer Sinn, der neue Kollege hat das Feature natürlich nicht so geschnitten, wie es gedacht war. Er hat zu viele Adapter-Spezifika in den Domänencode gezogen, sodass es da zu komplex geworden ist, und wir haben dann gemeinsam nochmal einiges umgebaut, um nachfolgende Änderungen zu vereinfachen. Damit haben wir zweierlei erreicht. Einerseits haben wir die Änderungen mit der Theorie der Zahlungskomponente in Einklang gebracht und damit ganz praktisch die Wartbarkeit der Software verbessert. Und andererseits hat der Kollege ein weiteres Stück der Theorie in seinen Kopf gebracht und kann so die Entwicklung im Rahmen der Theorie beser weitertreiben.

Also nochmal: Die Kernthese von Programming as Theory Building ist, dass ein Programm mehr ist als der Quellcode plus Dokumentation. Das Programm ist vielmehr die dahinterstehende Theorie und die lebt in den Köpfen der Entwickler. Quellcode und Dokumentation sind im besten Fall ein kleiner Ausschnitt davon.

So weit zu „Programming as Theory Building“. Kommen wir zu dem anderen Konzept, dem Not Invented Here Syndrome.

Das Not Invented Here Syndrome

Not Invented Here bezeichnet die Tendenz, Dinge abzulehnen, die man nicht selbst entdeckt oder erfunden hat. Im Kontext von Software-Entwicklung können das Werkzeuge sein, Prozesse, Programmierparadigmen oder eben auch eine Theorie, die hinter einem Programm steckt. Das ist kein formal definierter Begriff sondern eher eine Beobachtung aus der Praxis, aber es gibt Untersuchungen, die belegen, dass der Effekt real ist. Zum Beispiel eine Studie aus dem Jahr 2014. In der haben Wissenschaftler untersucht, wie Organisationen Crowdsourcing betreiben, also die „Crowd“ – irgendwelche externen Personen, Kunden, Webseitenbesucher usw. – nach Verbesserungsvorschlägen fragen, und insbesondere, wie sie die dann auswerten und umsetzen. Dabei haben sie verschiedene Arten von Distanz gemessen und zum Beispiel festgestellt, dass Vorschläge, die inhaltlich näher an dem dran waren, was die Organisationen sowieso schon gemacht haben, mehr Aufmerksamkeit bekommen haben als wirklich neue Vorschläge, die irgendwo aus der Crowd kamen, die also Not Invented Here waren. Was Not Invented Here ist, interessiert Organisationen anscheinend nicht so sehr.

Das habe ich auch selbst schon erlebt. An die Zahlungskomponente, die ich schon erwähnt habe, war auch SAP als Client angeschlossen. Aber natürlich nicht so wie alle anderen Systeme. Als diese Anbindung angebahnt wurde, haben sich die Verantwortlichen für die Zahlungskomponente und das fragliche SAP-System in die Wolle gekriegt, weil die SAP-Leute gesagt haben, der Ablauf, wie ihn sich das Zahlungskomponententeam vorstellt und die anderen Clients ihn auch realisiert haben, würde mit SAP nicht funktionieren, der widerspräche der SAP-Logik. Not Invented Here. Das können wir nicht und wollen wir nicht.

Weil diese Not Invented Here-Reaktion meistens negativ ist, hat sie einen ein bisschen schlechten Ruf. Man sagt auch: „Not Invented Here Syndrome“, und ein Syndrom ist grundsätzlich was Schlechtes. Aber die Not Invented Here-Reaktion, wie ich sie jetzt mal neutral nennen will, muss nicht unbedingt schlecht sein. Eine kleine Dosis Not Invented Here ist gut und sogar notwendig, ganz besonders, wenn man Programming as Theory Building ernst nimmt.

Not Invented Here als Sensor

Um das zu verstehen, möchte ich die Idee der Distanz nochmal aufgreifen, die in der Studie angewandt wurde. Ganz nah dran ist fast Invented Here, weit weg ist definitiv Not Invented Here. Für den Zweck dieses Vortrags schaue ich mir mal die Distanz einer Idee oder eines Konzepts zur Theorie des Programms an. Ich nenne sie hier „konzeptionelle Distanz“. Dabei ist die Distanz 0, wenn sich eine Idee innerhalb der Theorie bewegt. Und wenn ein Konzept richtig quer liegt, sodass es überhaupt nicht zur Theorie passt, dann sage ich, die beiden haben eine hohe konzeptionelle Distanz und die Idee wäre aus Sicht der Theorie des Programms Not Invented Here.

Wenn man sich dann ein Programm vorstellt – im Sinne von Theorie + Code – und eine Idee für eine Änderung, dann wäre es doch gut, wenn die Idee eine möglichst geringe konzeptionelle Distanz zur Theorie hätte. Denn dann wäre sie normalerweise leicht durchzuführen, relativ risikoarm und würde das Programm in einem guten, wartbaren Zustand hinterlassen, wohingegen eine Änderung, die weit weg von der Theorie wäre, eher schwierig umzusetzen wäre und fehleranfällig und eventuell die konzeptionelle Integrität des Programms zerstören und es somit unwartbar machen würde. So gesehen, ist es völlig rational, Ideen, die konzeptuell weit weg von der Theorie des Programms sind, kritischer zu betrachten als welche, die näher dran sind – also genau „Not Invented Here“ zu betreiben. Aus Sicht des Entwicklungsteams ist Not Invented Here quasi ein Schutzschild für die konzeptionelle Integrität unserer Theorie. Oder ein Sensor, der mir sagt: „Obacht, hier haben wir was, das nicht zu unserer Theorie passt.“

Heißt das, Not Invented Here ist uneingeschränkt gut und wir sollten Ideen, die Not Invented Here sind, grundsätzlich ablehnen? Natürlich nicht. Pauschalaussagen sind immer falsch. Zum einen gibt es viele unterschiedliche Arten von Distanz, bei denen man „Not Invented Here“ rufen kann, z. B. auch solche, die auf Gruppenzugehörigkeit beruhen. Sowas ist meistens schlecht. Das meine ich hier nicht. Ich meine ausschließlich die konzeptionelle Distanz.

Und es gibt noch einen anderen Grund, warum ich Dinge, die Not Invented Here sind, nicht pauschal ablehnen sollte: Die Theorie des Programms muss ja nicht perfekt sein. Manchmal lernt das Team was dazu und entwickelt die Theorie weiter. Das kann eigentlich nur passieren, indem man Ideen, die außerhalb der Theorie liegen, integriert.

Umgang mit neuen Ideen

Was mache ich also, wenn ich Entwickler in einem eingespielten Entwicklungsteam bin und jemand kommt mit einer Idee daher oder macht etwas auf eine Art und Weise, die meinen Not Invented Here-Sensor triggert? Zuerst check ich kurz, ob er aus einem guten Grund getriggert hat. Ist es die konzeptionelle Distanz zur Theorie? Wenn nicht, dann ist es vielleicht ein Fehlalarm und ich sollte den mal stummschalten und mir die Idee einfach weiter anhören. Wenn ich aber feststelle, dass das, was ich da höre oder sehe, wirklich weit weg von der Theorie meines Programms ist, dann kann ich mich freuen, denn dann hat der Sensor funktioniert. Er hat verhindert, dass wir blind etwas in unser Programm einbauen, das später Probleme verursachen wird.

Das Einfachste ist dann zu sagen: „Nein, das machen wir nicht.“ Das spart Zeit und Geld und hält das Programm schön klein. Aber das kann und will man natürlich nicht immer machen, denn hinter vielen Ideen steckt ja eine gute Absicht. Dann besteht die Aufgabe darin, die Distanz zur Theorie möglichst zu verringern. Gibt es eine Möglichkeit, das, was man mit der Idee eigentlich erreichen wollte, im bestehenden Rahmen der Theorie abzubilden? Das ist der beste Fall, dann sollte man das eben so tun. Wir erinnern uns an den neuen Kollegen bei der Zahlungskomponente. Der wollte ein Feature bauen und hat es auf eine Art und Weise versucht, die nicht zur Theorie gepasst hat, und das Team hat es dann gemeinsam so umgemodelt, dass es im bestehenden Rahmen gut funktioniert.

Wenn wir es nicht schaffen, die Idee konzeptionell an die Theorie ranzubringen, und wir sie trotzdem umsetzen wollen, dann geht das auch, aber es wird mehr Arbeit. Dann müssen wir nämlich die Theorie an die Idee anpassen. Und das heißt, nicht nur neuen Code schreiben, denn die Theorie ist ja mehr als nur Code. Wir müssen das ganze Team mitnehmen, uns gemeinsam ein neues, konsistentes mentales Modell überlegen, das die bisherige Theorie und die neue Idee in Einklang bringt, und das dann umsetzen. Dabei ist die Hauptaufgabe, dieses gemeinsame Verständnis von der neuen Theorie herzustellen. Dann den passenden Code zu schreiben, ist meistens deutlich leichter.

Noch ein Beispiel

Dazu habe ich auch ein Beispiel, und zwar aus derselben Zahlungskomponente. Da mussten wir einmal einen Zahlungsdienstleister austauschen. Der Vertrag mit dem alten ist ausgelaufen, im Rahmen einer Ausschreibung wurde ein neuer gefunden und der musste angebunden werden, und zwar ohne Downtime. Es gab schon eine Möglichkeit, die Zahlungsdienstleister pro angebundener App zu konfigurieren, um die Zahlungen für eine App an den jeweils passenden Adapter zu schicken. Das heißt, der Theorie des Programms folgend, hätten wir zum Migrationszeitpunkt die Apps konfigurativ umstellen können und wären fertig gewesen. Da es aber um viele Kunden und um viel Geld ging und solche Big-Bang-Migrationen ziemlich risikobehaftet sind, hatte ich die Idee, die Umstellung doch lieber pro Kunde vorzunehmen. So ließe sie sich viel feingranularer steuern und wir könnten das Risiko deutlich senken.

Die konzeptionelle Distanz zur bestehenden Theorie war da allerdings schon relativ groß und obwohl die Idee von mir, also aus dem Team selbst heraus kam, haben wir uns genau überlegt, ob wir das so einführen wollen und wie wir dieses Konzept der Migration pro User mit dem verheiraten, was schon da war. Letztlich haben wir uns gemeinsam dafür entschieden, diese Konfiguration pro App komplett auszubauen, die Migration pro User zu implementieren und die nach der Migration zusammen mit dem alten Zahlungsdienstleister auch wieder auszubauen. Letzteres mag überraschen, aber das hatte mit der neuen Theorie zu tun, die wir zum Thema Routing von Zahlungen an Zahlungsdienstleister gebildet hatten.

Alte Theorie:

Das Routing ist eigentlich keine richtige Logik. Da reicht es, wenn man irgendwo eine Tabelle hat, und man kann das Routing praktisch beliebig zur Laufzeit ändern.

Neue Theorie:

Das Routing ist eine Kernfunktion der Zahlungskomponente und wenn man das ändert, ist das ein nicht-triviales Vorhaben, das wahrscheinlich mit Custom-Logik und Edge-Cases verbunden ist, die man am besten mit Code abbilden kann.

Hätten wir uns nicht die Zeit genommen, die Theorie anzupassen, sondern die Migration pro User einfach zusätzlich reingefrickelt … das will ich mir gar nicht vorstellen, was dabei rausgekommen wäre. Bestimmt nichts, was sich leicht weiterentwickeln ließe.

Umgang mit bestehenden Theorien

Betrachten wir das Ganze doch mal aus einer anderen Perspektive, nämlich aus der Sicht desjenigen, der mit einer Theorie von jemand anderem umgehen will oder muss. Das kann zum Beispiel ein Stück Software sein, die jemand anders entwickelt hat und an der es jetzt etwas zu ändern gibt. Und nehmen wir auch mal an, dass die konzeptionelle Distanz zu dem, was ich sonst so gewohnt bin, mittel bis hoch ist. Dann werde ich anfangs Schwierigkeiten haben, die Theorie dahinter zu verstehen. Ich kann dann die Änderung an der Software nicht so durchführen, dass ich mir sicher sein kann, dass das Ergebnis gut ist. Die Theorie ist zu weit weg. In anderen Worten, mein Not Invented Here-Sensor löst aus. Das heißt aber nicht, dass ich in die Not Invented Here-Abwehrhaltung verfallen sollte. Denn die Konsequenz aus der Theory-Building-These ist ja: Wenn ich keine halbgare Frickellösung haben will, dann muss ich mir zuerst die Theorie des Programms beschaffen. Die muss in meinen Kopf rein. Wie das geht, hängt von der Situation ab.

Onboarding in ein aktives Entwicklungsteam

Eine Möglichkeit ist: Ich komme in ein eingespieltes Entwicklungsteam. Dann kann ich davon ausgehen, dass die Leute eine Theorie in ihren Köpfen haben, die ich aber noch nicht kenne. Ich sehe erst mal nur den Source-Code. Dann heißt es als erstes: Die Abwehrhaltung abschalten! Das Unproduktivste, was ich machen kann, ist, bei jeder Gelegenheit zu sagen: „Das ist aber komisch. Wie kann man nur auf so eine Idee kommen? Ich hätte das ganz anders gemacht.“ Und so weiter. Das hindert mich nur daran, meine Hauptaufgabe zu erledigen, die da wäre, möglichst schnell die Theorie des Programms zu verinnerlichen. Dabei hilft:

Mit dem Team reden.
Doku lesen.
Mit dem Team reden.
Code lesen.
Mit dem Team reden.
Auch mit dem Coden anfangen, wenn möglich im Pair-Programming, aber zuerst weniger, um den Code anzupassen, sondern eher mit dem Ziel, zu lernen.

Ach ja, und mit dem Team reden.

Ich hatte das erst vor kurzem bei einem Service, der mit historisierten Daten hantiert. Da können sich Attribute von manchen Entitäten über die Zeit hinweg ändern und sie werden mit „von“ und „bis“ gespeichert. Andere nicht. Und auch Beziehungen zwischen Entitäten sind teilweise historisiert, teilweise nicht. Ich kann im Code sehen, dass das passiert, und es steht auch ein bisschen was darüber im internen Wiki, aber richtig verstanden, warum das so ist, wie es ist, habe ich erst im Gespräch mit dem Team. Ich hätte auch gleich sagen können: „Das ist doch alles viel zu kompliziert, ich hätte das ganz anders gebaut.“ Not Invented Here. Aber das hätte mir nicht dabei geholfen, das Thema zu verstehen.

Also wenn ich irgendwo neu reinkomme: Die ablehnende Not Invented Here-Reaktion abschalten und möglichst schnell die Theorie aufsaugen.

Umgang mit Legacy-Software

Eine andere Situation wäre: Ich übernehme Verantwortung für ein bestehendes System, bei dem niemand mehr verfügbar ist, dem ich Fragen stellen kann. Das kann unterschiedliche Gründe haben. Vielleicht ist es ein altes System, an dem lange nichts mehr zu tun war, und die Leute mit Ahnung sind inzwischen weitergezogen. Und jetzt ist wieder etwas zu tun und ich habe das große Los gezogen. Aber vielleicht ist auch ein Team da, aber es sind nicht mehr die ursprünglichen Entwickler sondern welche, die später dazugekommen sind und die Theorie selbst nicht verinnerlicht haben. Ich habe das schon erlebt – vielleicht ihr auch – dass ich in ein Team gekommen bin, das das eigene System nicht kennt – oder nur Teile davon. Dann wird es schwierig, dann muss man die Theorie rekonstruieren. Das ist Arbeit.

Und auch bei dieser Arbeit kann wieder mein Not Invented Here-Sensor bzw. die Idee der konzeptionellen Distanz hilfreich sein. Die sagt mir nämlich, womit ich mich noch beschäftigen muss. Ich habe ja schon, wenn ich anfange, eine gewisse Vorstellung davon, wie die Theorie aussehen könnte, und wenn es nur meine Intuition ist, die auf meiner Erfahrung mit anderen Programmen beruht. Wenn ich mir dann das Material anschaue, das da ist – Code, Bildchen, Doku, whatever – dann gibt es Dinge, da denke ich mir: „Ja, OK, klar.“ Und es gibt Dinge, da löst mein Entfernungssensor aus – also der für die konzeptionelle Distanz. Da denke ich mir dann: „Hui, das passt gar nicht zu dem, was ich mir bisher zusammengereimt habe!“ Manche Entwickler verfallen dann in die Not Invented Here-Abwehrhaltung und grummeln über den Legacy-Scheiß, mit dem sie sich rumschlagen müssen. Ich nehme das als Signal, genauer hinzuschauen, denn das ist eine Stelle, an der meine Theorie offensichtlich noch falsch oder zumindest unvollständig ist. Also grabe ich mich da rein und versuche, genau die Stelle zu verstehen und die ursprüngliche Theorie noch ein Stück weiter zu rekonstruieren.

Das geht bis zu einem gewissen Punkt. Aber meistens nicht vollständig. Wenn man an dem Punkt angelangt ist, dann kann – und sollte – man anfangen, die Theorie zugunsten der aktiven Entwickler umzubauen, um aus diesem „ich habe keine Ahnung, was ich da tue“-Modus rauszukommen. Dann hat man wieder eine lebende Theorie, die tatsächlich in den Köpfen der Entwickler drinsteckt, und kann damit weiterarbeiten.

Ein Beispiel dafür ist ein Entwickler-Team, das eine Java-Library geerbt hat, die aus fünf Schichten aufgebaut ist. Niemand kannte den Sinn dieser Schichten und da gab es genau diese Not Invented Here-Reaktion von einigen Entwicklern und sie wollten das Ding am liebsten gar nicht anfassen. Als ich dazugekommen bin, haben wir uns angeschaut, welche Probleme wir tatsächlich haben und wo uns Schichten helfen und wo eher nicht, und wir haben jetzt angefangen, den Code gemäß unserer neuen Theorie zu ändern.

Reprise

Ihr seht also, es gibt einen Zusammenhang zwischen der Idee des Programming as Theory Building von Peter Naur und Not Invented Here. Nochmal kurz zusammengefasst, sagt Programming as Theory Building, dass die Essenz des Programmierens nicht das Schreiben von Code ist, sondern das Aufbauen einer Theorie. Diese Theorie lebt in den Köpfen der Entwickler und ist das eigentliche Programm. Der Code gehört auch dazu, aber der sorgt nur dafür, dass das Programm auf einer Maschine ausführbar wird. Änderungen am Code lassen sich im Einklang mit der Theorie viel leichter durchführen und führen zu besseren Ergebnissen als Änderungen, die ohne Rücksicht auf die Theorie vorgenommen werden, weil das Programm eben hauptsächlich aus der Theorie besteht.

Not Invented Here ist eine Reaktion auf Ideen, die von weit weg kommen, wobei die Entfernung in unterschiedlichen Dimensionen liegen kann. Oft ist diese Reaktion ablehnend und kontraproduktiv, aber das muss nicht so sein.

Eine für die Entwicklung besonders wichtige Dimension ist die konzeptionelle Distanz. Wie weit liegen zwei Konzepte oder Ideen oder Theorien auseinander? Laut der Theory-Building-These sind Programmänderungen mit einer hohen konzeptionellen Distanz zur Theorie genau die problematischen. Und deswegen kann die Not Invented Here-Reaktion nützlich sein, weil sie mich auf so etwas hinweist.

Wenn ich gelernt habe, die automatische Abwehrhaltung loszuwerden, die oft damit verbunden ist, und sie stattdessen als Signal zu betrachten, dann bin ich frei, mir zu überlegen, ob und wie ich die konzeptionelle Distanz verringern will. Vielleicht will ich das gar nicht und sage tatsächlich: „Nein, das mach ich nicht!“ Aber wenn ich es machen will, gibt es im Wesentlichen drei Möglichkeiten: Ich kann die eine Seite anpassen oder die andere oder ich tue beides und lande irgendwo in der Mitte.

Wir haben uns ein paar Situationen dazu angeschaut: Beim Onboarding eines neuen Entwicklers sind die beiden Konzepte die Theorie des Programms und die vorgefertigten Ideen und Vorstellungen im Kopf des Entwicklers. Da möchte ich in der Regel, dass der Entwickler zuerst mal die Theorie des Programms versteht und später seine eigenen Erfahrungen einfließen lässt, um die Theorie zu verbessern.

Bei Änderungsvorschlägen kann zweierlei passieren. Entweder man schafft es, die Anforderungen im Rahmen der Theorie des Programms umzusetzen – das ist der beste Fall –, oder man muss tatsächlich die Theorie des Programms anpassen, um die Lösungsidee zu integrieren. Aber dann auch wirklich die Theorie, nicht nur punktuell den Code!

Und wenn man ein Legacy-System übernimmt, bei dem die Theorie nicht mehr greifbar ist, wird man oft die Situation haben, dass man sie nicht ganz rekonstruieren kann. Man sollte es aber versuchen und dann die verbleibenden Teile an die eigenen Vorstellungen anpassen und eine neue Theorie bilden, damit man vernünftig damit arbeiten kann.

Wenn ihr also das nächste Mal auf eine Not Invented Here-Reaktion stoßt, bei euch selbst oder bei anderen, dann überlegt euch mal, was dahintersteckt und ob das nicht ein Signal sein könnte, euch tiefer mit dem Thema zu beschäftigen!

Danke schön und Happy Programming! Oder Theory-Building!

Unit testing vs. integration testing

Sebastian Hans — Tue, 04 Nov 2025 00:00:00 +0000

Unit tests and integration tests are often compared – and usually presented as opposites. In this post, I will examine these two types of tests from a different point of view and show how this perspective can help us write better tests.

A plea for a differentiated view

Unit tests are usually characterized as “small”, “fast”, and “reliable”, while integration tests are typically seen as “large”, “slow”, and “unreliable”. If you compare the test for one method of one class that doesn't do much with the test of the complete system including an external database, this is probably true. In the former case, most developers would probably agree that it is a unit test, and in the latter case, that it is an integration test. This is certainly not wrong. However, it's also not the whole truth. Let's take a look at a test for a Java class that solves the universally beloved FizzBuzz problem.

(All the code from this post is available in the GitHub repo fizzbuzz-testing-example to play around with.)

public class FizzBuzz {
    public List<String> go(int n) {
        return IntStream
            .rangeClosed(1, n)
            .mapToObj(this::fizzBuzz)
            .toList();
    }

    private String fizzBuzz(int n) {
        if (n % 3 == 0 && n % 5 == 0) return "FizzBuzz";
        else if (n % 3 == 0) return "Fizz";
        else if (n % 5 == 0) return "Buzz";
        else return Integer.toString(n);
    }
}

public class Main {
    public static void main(String[] args) {
        var fizzBuzz = new FizzBuzz();
        fizzBuzz.go(100).forEach(System.out::println);
    }
}

class FizzBuzzTest {
    @Test
    void oneIs1() {
        FizzBuzz fizzBuzz = new FizzBuzz();

        var result = fizzBuzz.go(1);

        assertEquals("1", result.getFirst());
    }

    // + several similar test cases
}

Probably everyone would agree that this is a unit test. It instantiates the class, calls a method that doesn't interact with other components, and checks the result, nothing more … but is that really true? What about IntStream? What about String and Integer? What about the Java compiler? And the Java runtime? All of this is necessary to actually execute the method FizzBuzz.go(int). If we leave out any of this, the program is no longer executable. A bug in any of these components could cause the test to fail. The test only runs successfully if all of these components work without error.

Let that sink in for a moment.

“But those things aren't mine! If my test depends on 3rd-party stuff, does that mean this is an integration test!?”

Yes, that's exactly what it means. More precisely: every test is an integration test, because every test integrates something; the only question is what. Thus, every unit test is also an integration test. It integrates everything that is part of the unit and leaves out everything that is not part of the unit.

In the example above, the test integrates the external dependencies I already mentioned, but also the methods FizzBuzz.go(int) and FizzBuzz.fizzBuzz(int). And the methods of the standard library, too.

So the interesting question is …

What is a unit?

For our purposes, a unit is a self-contained functional element with defined dependencies and inputs and outputs. Units can work together as part of a larger unit.

We'll expand the FizzBuzz example a bit to highlight some possibilities:

public interface Output {
    String render();
}

public record NumberOutput(int number) implements Output {
    @Override
    public String render() { return Integer.toString(number); }
}

public enum FizzBuzzOutput implements Output {
    FIZZ("Fizz"),
    BUZZ("Buzz"),
    FIZZBUZZ("FizzBuzz");

    private final String rep;

    FizzBuzzOutput(String rep) { this.rep = rep; }

    @Override
    public String render() { return rep; }
}

public interface Selector {
    Output select(int n);
}

public class Does {
    public boolean divide(int divisor, int n) { return (n%divisor==0); }
}

public record SimpleSelector(Does does) implements Selector {
    @Override
    public Output select(int n) {
        if (does.divide(3, n) && does.divide(5, n)) return FIZZBUZZ;
        else if (does.divide(3, n)) return FIZZ;
        else if (does.divide(5, n)) return BUZZ;
        else return new NumberOutput(n);
    }
}

public interface Streamer {
    Stream<Output> go(int n);
}

public record AscendingStreamer(Selector selector) implements Streamer {
    @Override
    public Stream<Output> go(int n) {
        return IntStream.rangeClosed(1, n).mapToObj(selector::select);
    }
}

public record FizzBuzz(Streamer streamer) {
    public List<String> go(int n) {
        return streamer.go(n).map((output) -> {
            if (output == FIZZBUZZ) return output.render().toUpperCase();
            else return output.render();
        }).toList();
    }
}

public class Main {
    public static void main(String[] args) {
        var fizzBuzz =
            new FizzBuzz(
                new AscendingStreamer(new SimpleSelector(new Does())));
        fizzBuzz.go(100).forEach(System.out::println);
    }
}

The new solution divides the code into several classes with distinct responsibilities. The class FizzBuzz now uses a Streamer that determines what number to start with and in what order to process the numbers. Streamer in turn uses a Selector to determine which number should actually be output. Selector in turn builds on the helper class Does^[1], which handles the divisibility check. The outputs are returned in typed form. FizzBuzz itself now only needs to convert the Stream of Outputs into a list of Strings to keep the interface unchanged.^[2]

However, we've built in one extension: we want to SHOUT “FizzBuzz” in capital letters and introduced an additional mapping in the FizzBuzz class for this purpose.

To plug the individual parts together, we use constructor-based dependency injection. The use of interfaces allows us to easily swap out implementations. It should be noted, however, that this separation of concerns is an internal implementation detail of the FizzBuzz class and has no impact on the interface it exposes to callers or its functionality. Except for the initialization code, the main method looks exactly the same as before.

For the purpose of testing this implementation, what are the relevant units here? The reflexive answer would probably be, “Each class is a unit.” For unit testing, this means we test each class individually and leave its dependencies out of the test. We use Mockito for this and also AssertJ, to simplify list and stream handling in the tests. Here are a few example tests:

@ExtendWith(MockitoExtension.class)
class SimpleSelectorTest {
    @Mock
    private Does does;

    private SimpleSelector selector;

    @BeforeEach
    void setUp() {
        selector = new SimpleSelector(does);
    }

    @Test
    void modNothingYieldsNumber() {
        when(does.divide(3, 1)).thenReturn(false);
        when(does.divide(5, 1)).thenReturn(false);
        assertThat(selector.select(1)).isEqualTo(new NumberOutput(1));
    }

    // …

    @Test
    void mod3AndMod5YieldsFizzBuzz() {
        when(does.divide(3, 1)).thenReturn(true);
        when(does.divide(5, 1)).thenReturn(true);
        assertThat(selector.select(1)).isEqualTo(FIZZBUZZ);
    }
}

This test checks the selection logic. It does so more directly than the test in the previous version because it no longer has to extract the result from a list. The class Does is mocked away. There are separate tests for that one.

@ExtendWith(MockitoExtension.class)
class AscendingStreamerTest {
    @Mock
    private Selector selector;

    private Streamer streamer;

    @BeforeEach
    void setUp() {
        streamer = new AscendingStreamer(selector);
    }

    @Test
    void returnsSelectedOutputsInOrder() {
        when(selector.select(1)).thenReturn(new NumberOutput(1));
        when(selector.select(2)).thenReturn(new NumberOutput(2));
        when(selector.select(3)).thenReturn(FIZZ);

        var stream = streamer.go(3);

        assertThat(stream).containsExactly(
            new NumberOutput(1),
            new NumberOutput(2),
            FIZZ);
    }
}

This test only checks the behavior of our Streamer implementation. The Selector is mocked away.

And last but not least, the test of FizzBuzz checks the behavior of only this class, with the output of the Streamer specified by Mockito:

@ExtendWith(MockitoExtension.class)
class FizzBuzzTest {
    @Mock
    private Streamer streamer;

    private FizzBuzz fizzBuzz;

    @BeforeEach
    void setUp() {
        fizzBuzz = new FizzBuzz(streamer);
    }

    @Test
    void aggregates() {
        when(streamer.go(1)).thenReturn(Stream.of(
            new NumberOutput(1),
            new NumberOutput(2),
            FIZZ));
        assertThat(fizzBuzz.go(1)).containsExactly("1", "2", "Fizz");
    }

    @Test
    void shoutsFIZZBUZZ() {
        when(streamer.go(1)).thenReturn(Stream.of(FIZZBUZZ));
        assertThat(fizzBuzz.go(1)).containsExactly("FIZZBUZZ");
    }
}

The tests all run successfully, and when we execute Main.main(), we see that everything also works together.

The approach “unit=class” (or sometimes also “unit=method”) is widespread among Java developers. I've seen this kind of test (at the class level with mocked dependencies) many times in different projects. And dependency chains are quite common. An example in a microservice based on the ports-and-adapters architecture could be: Web Controller → Driving Port → Application Service → other Application Service → Domain Object → another Domain Object → Driven Port → JDBC Adapter (arrows are runtime dependencies).

Of course, FizzBuzz in its entirety is also a unit. In the first implementation, all the code is in one class; in the second it is distributed across multiple classes, but it still forms a functional unit. In this example, the unit as a whole is quite small and can well be tested in its entirety, but in larger code bases such whole-system tests can be quite slow (looking at you, @SpringBootTest). This is the point at which most developers begin to bandy about the word “integration test”, and negative vibes can be felt.

So, mocking it is. But what happens to tests in this style if we begin to make changes?

A small change

Let's assume that the customer for our industry-leading FizzBuzz solution wants to pave the way for a bright future in which FizzBuzz can not only “fizz” and “buzz”, but also “zoom” and “boom” and much more – and not just for the numbers 3 and 5, but for any other number. To achieve this, we will probably need to combine multiple numbers, or rather their mapped Outputs. This is where our enum-based approach will hit its limits. We would have to add not only ZOOM and BOOM as values, but also all possible combinations. To avoid combinatorial explosion, we introduce a CombinedOutput instead:

public record CombinedOutput(List<Output> outputs) implements Output {
    public CombinedOutput(Output... outputs) { this(List.of(outputs)); }

    @Override
    public String render() {
        return outputs
            .stream()
            .map(Output::render)
            .collect(Collectors.joining(""));
    }
}

And use this in our SimpleSelector:

<         if (does.divide(3, n) && does.divide(5, n)) return FIZZBUZZ;
---
>         if (does.divide(3, n) && does.divide(5, n)) return new CombinedOutput(FIZZ, BUZZ);

And in its test:

<         assertThat(selector.select(1)).isEqualTo(FIZZBUZZ);
---
>         assertThat(selector.select(1)).isEqualTo(new CombinedOutput(FIZZ, BUZZ));

We run the entire test suite and see that all tests pass. Great! That would be that. We're now well equipped for the introduction of ZOOM at 7 or whatever else may come our way. Quickly push to production and the sprint goal is achieved. Or the deliverable for this release can be signed off. Or something like that. In any case, we're done.

PAUSE

Before reading on, take a moment to think about what just broke! Because, yes, something did break. But I didn't lie. All tests ran without errors.

Found the mistake? No? Well, then, let's take a look. What happens when we run Main.main()?

1
2
Fizz

Looking good so far. It's fizzing.

4
Buzz

Buzzing, too.

Fizz
7
8
Fizz
Buzz
11
Fizz
13
14
FizzBuzz

And here's the “FizzBuzz”. But – wait a minute! Shouldn't it be “FIZZBUZZ”? You remember the SHOUTING requirement? And I know for sure there's a dedicated test for it. Where was it again? Ah, in FizzBuzzTest. Let's take a closer look at it now.

    @Test
    void shoutsFIZZBUZZ() {
        when(streamer.go(1)).thenReturn(Stream.of(FIZZBUZZ));
        assertThat(fizzBuzz.go(1)).containsExactly("FIZZBUZZ");
    }

If the streamer returns FIZZBUZZ, “FIZZBUZZ” should come out. So what's the problem? The problem is that in production the streamer no longer returns FIZZBUZZ at all, because we changed the selector to return a CombinedOutput. If the selector were still outputting FIZZBUZZ, the class FizzBuzz would still SHOUT. But it no longer does.

Because we focused each test so narrowly on its respective unit – or what we identified as a unit, i.e., the class – none of them was able to uncover this mismatch. A comprehensive integration test would have helped, but those are expensive and slow and are therefore only executed relatively late in the development cycle – or omitted entirely if the test coverage is already good. And it is good (100%).^[3]

A real example

Of course, this is a contrived example. But the same kinds of errors also occur in real development projects. A case I've personally encountered concerned a duplicate check in a REST interface. There was a test for the database adapter that ensured a DuplicatePaymentException was thrown when a unique constraint was violated during insertion into the database. And there was a test at the API layer that ensured that in case of a DuplicatePaymentTransactionException an appropriate response (HTTP status code 409) was sent to the client. This project relied heavily on mocking to isolate the “units” in tests. Each of the tests looked plausible when viewed individually, and they were all successful. But clients in production kept getting HTTP status code 500 (which indicates an internal server error) instead of the expected duplicate error message, because the DuplicatePaymentException was not caught and handled. I only found the cause when I looked at both tests side by side. That's when I noticed that different (but similarly named) exception types were being used.

The problem of too narrow unit tests

The fundamental problem here is the same as in the FizzBuzz example: the test of one unit makes assumptions about the behavior of another unit and hardcodes them into the test instead of verifying them as well. If the streamer returns FIZZBUZZ, then FizzBuzz does the right thing. But if it doesn't, the test is meaningless. If the adapter throws a DuplicatePaymentTransactionException, the API delivers the correct response. But if it throws a DuplicatePaymentException instead, we lose.

In both cases, we tried to make things independent of each other that are not independent of each other in reality. But if the behavior of one of our unit's dependencies changes, it would certainly be good to have a test that fails if the change invalidates our assumptions.

So back to giant integration tests after all? Not necessarily. Instead, I propose overlapping tests.

Overlapping tests

Such tests are unit tests, too. We just choose the unit a bit differently. So far, we have used exactly one class as a unit, with boundaries upwards (to the caller) and downwards (to the dependencies). If we draw a picture of the scopes of these tests, they are completely separated from each other. In the following diagram, each test frames the classes it covers, i.e., exactly one each. The arrows between the classes represent runtime dependencies.

As we have seen, the problem with this strict separation is that the interactions remains untested. Overlapping tests solve this by expanding the scope of the tests (i.e., enlarging the unit under consideration), so that the interactions between the classes are included in the tests. Figuratively speaking, each arrow is contained in at least one test scope. For our project “Over-engineered FizzBuzz”, it could look like this:

And here is the code:

class SimpleSelectorTest {
    private final SimpleSelector selector =
        new SimpleSelector(new Does());

    @Test
    void oneIs1() {
        assertThat(selector.select(1)).isEqualTo(new NumberOutput(1));
    }

    // …

    @Test
    void fifteenIsFizzBuzz() {
        assertThat(selector.select(15))
            .isEqualTo(new CombinedOutput(FIZZ, BUZZ));
    }
}

Here, Does is no longer mocked away, which even makes the test code shorter.^[4]

class AscendingStreamerTest {
    @Test
    void returnsSelectedOutputsInOrder() {
        var streamer =
            new AscendingStreamer(new SimpleSelector(new Does()));

        var stream = streamer.go(15);

        assertThat(stream).containsExactly(
            n(1), n(2), FIZZ, n(4), BUZZ, FIZZ, n(7), n(8), FIZZ, BUZZ,
            n(11), FIZZ, n(13), n(14), FIZZBUZZ
        );
    }

    private NumberOutput n(int n) { return new NumberOutput(n); }
}

Here, the selector is no longer mocked (and neither is Does – we could mock it, but we wouldn't gain any advantage by doing so). The test must cover all possible outputs that are important to us, which is why it goes up to 15. That's where FIZZBUZZ appears for the first time. This ensures that the streamer works for all Outputs that SimpleSelector returns. In the “mocking” test implementation, this was pointless, since the test itself specified what the streamer returned. It didn't contain the arrow, so to speak. This test does.

@ExtendWith(MockitoExtension.class)
class FizzBuzzTest {
    @Mock
    private Does does;

    private FizzBuzz fizzBuzz;

    @BeforeEach
    void setUp() {
        fizzBuzz =
            new FizzBuzz(new AscendingStreamer(new SimpleSelector(does)));
    }

    @Test
    void aggregates() {
        when(does.divide(3, 1)).thenReturn(false);
        when(does.divide(5, 1)).thenReturn(false);
        when(does.divide(3, 2)).thenReturn(false);
        when(does.divide(5, 2)).thenReturn(false);
        when(does.divide(3, 3)).thenReturn(true);
        when(does.divide(5, 3)).thenReturn(false);

        assertThat(fizzBuzz.go(3)).containsExactly("1", "2", "Fizz");
    }

    @Test
    void shoutsFIZZBUZZ() {
        when(does.divide(3, 1)).thenReturn(true);
        when(does.divide(5, 1)).thenReturn(true);

        assertThat(fizzBuzz.go(1)).containsExactly("FIZZBUZZ");
    }
}

In this test, Does is mocked, so the test doesn't reach all the way to the end of the dependency chain (otherwise it would be a complete integration test). However, it must reach far enough to cover the interesting behavioral differences in the dependencies. In our case, this means the selector must be included, since it determines which Outputs can arrive in FizzBuzz. This way, we get a test that fails when we change the selector result from FizzBuzzOutput.FIZZBUZZ to CombinedOutput and thereby break our SHOUTING feature.

Exactly how to draw the scope boundaries for the tests to find a good middle ground between sharply delineated (too narrow) tests and all-encompassing integration tests is a matter of experience. However, there are a few useful heuristics.

A dependency with a stable interface can be mocked without giving up too much safety. The interface of Does is very stable since it is based on mathematical rules, and can therefore be mocked without running the risk of being surprised by changes. In this concrete example, this isn't actually all that useful, since the implementation is fast enough and we don't really gain anything through mocking. But if we imagine a web interface that's slow and a bit unreliable, but has a stable interface contract, it makes much more sense.

On the other hand, mocking a dependency with interesting behavior should not be taken lightly. In the case of FizzBuzz, the Selector implementation contains the most interesting behavior (selecting what to output). As you can see in the picture above, we could mock it in FizzBuzzTest and still have all arrows covered by at least one test. We do not, however, because its behavior arguably has a larger influence on the operation of FizzBuzz than, e.g., that of the Streamer implementation. By mocking the Selector implementation, we would run the risk of missing interesting interactions. Replacing FizzBuzzOutput.FIZZBUZZ with CombinedOutput in the Selector would still cause AscendingStreamerTest to fail, so we would notice that this change has some knock-on effects. But we would not directly see the most interesting one.

A larger example

The principle of overlapping tests can be applied not only to unit tests within a single piece of software, but also to integration tests between multiple services. An example of its successful application was an integration of multiple services with payment service providers via a dedicated payment component.

Each service had tests for itself, of course, and the payment component also had tests for itself. Large integration tests, where a service was integrated with the payment component and the payment service providers, were difficult because the connection to the payment service providers' test environments wasn't particularly stable and therefore such tests were often disrupted by temporary problems. That's why we, as developers of the payment component, agreed with the service teams that they would use fake payment methods for the majority of their tests. These fake payment methods were not hooked up to the payment service providers. This way, service tests could proceed without disruption. Since the test scopes overlapped, as shown in the following diagram, it was still ensured that the system as a whole worked as intended.

In this diagram, tests shown in orange were the responsibility of the payment component team, and tests shown in blue were the responsibility of the service teams.^[5]

In addition to the tests shown in the diagram, there were further overlapping tests within the payment component. And each building block in the diagram also consisted of multiple units, which were tested partly with sharp boundaries and partly overlapping. Overall, the payment component achieved very good test coverage this way – much better than could be achieved with completely isolated unit tests, even supplemented with “large” integration tests.

Conclusion

Unit tests and integration tests cannot be sharply separated from each other – and are certainly not mutually exclusive concepts. Units can be of different sizes and contain sub-units. On the one hand, a unit is distinct and separated from other units (at the same level of granularity), and on the other hand, it integrates its own sub-units. Considering only units at one level (e.g., only unit=class) and testing them in isolation leads to test gaps and consequently to bugs.

Overlapping tests close these test gaps and at the same time help to avoid elaborate, slow and expensive all-encompassing integration tests.

Happy testing!

Footnotes

The names Does and divide may seem a bit strange, but they enable the client code to say if (does.divide(3, n)). This is a trade-off I'm willing to make. Anyway, it doesn't have a bearing on the point of this post. ↩︎
I use record instead of class for SimpleSelector, AscendingStreamer and FizzBuzz for syntactic brevity only; not because I think these classes make good records. ↩︎
Yes, the error would have been noticed if we had immediately deleted the superfluous enum FIZZBUZZ. Then FizzBuzzTest would no longer have compiled. But in large projects changes often aren't rolled out atomically across the entire code base, and it is common for old classes or values to stay around for a while. ↩︎
The test of Does itself remains the same. It doesn't have any dependencies anyway. ↩︎
In the diagram, the payment service providers lie only partially in the test scope, since their test environments come with various limitations that make it impossible to cover 100% of the functionality in integration tests. Yes, this caused problems. Yes, we also tested in production. ↩︎

Excellence is a habit

Sebastian Hans — Tue, 21 Oct 2025 00:00:00 +0000

In martial arts training, you spend a lot of time repeating certain movements, sequences, and reactions over and over again. When learning a defense, one partner often executes a predetermined attack, and the other one practices the defensive technique. Usually (hopefully!) your training partner is not really trying to hurt you. Also, very few people would be able to go full-throttle for a whole training session, even if they wanted to. As a consequence, practice is mostly done at some level of reduced force.

Now, the thing is, deflecting a blow dealt by someone who isn't really trying to hurt you and who is also conserving strength is relatively easy. You don't really need much in the way of technique for that. Unfortunately, this means that it is very easy to become negligent. You don't step as far aside as you should because you don't feel the need. After a few repetitions, your partner starts to punch slightly to one side because your blocking technique is going to push the arm over there anyway, right? And your technique gets sloppy without you noticing because, well, it seems to work, doesn't it? You are standing at the wrong angle but it doesn't matter because your partner is focusing on the (slightly off) punch and doesn't realize you are in reach of their other arm now. And neither do you. Let's call this way of training “sloppy mode”.

This is all well and good as long as you keep training in this way. But what happens when we add pressure, e.g., with a belt examination? I have trained with a lot of youths who have confidently told me, “In the examination, I'll try harder/step farther/block better!”

Let me tell you what usually happens. When the pressure is on, people do what they have always done, except sloppier and with more force. Will they hit harder? Yes, they probably will. Will they block with more force? Yes, they probably will. Will they find the correct distance if they didn't before? No, they won't. Will they block more accurately? No way! Will they position themselves better? Forget it! Will they fumble, forget steps, miss their aim? Depends on how much they have practiced. I repeat: Under pressure, people do what they have always done, except sloppier and with more force.

This is because under pressure, there is no time to think, so you fall back to habits. Your mind goes blank, and your body takes over and performs the movements it remembers. You don't think, “Oh, wait! This time, I wanted to do it differently!” Instead, you think, “Oh, shit!” Or, “What am I doing?” Or you don't think at all. If you stop to think, you are already down. When the pressure is on, you don't think. You rely on habit.

The whole point of training is building habits that work well in the kind of situation you are training for. This means that you must strive to execute your defense flawlessly, performing the correct movements while maintaining good form each and every time. Even if your partner's attacks are lackluster. Even if you could – in the training situation – defend yourself by just waving your arms in front of your face. And even if you yourself are tired. You may train at a lower energy level. Energy will increase under pressure. But you must not train sloppily! If you build a habit of sloppiness, you will be even more sloppy when it matters – when your are under pressure.

If, however, you make a habit of moving correctly – even when it is not strictly needed – you can rely on this habit to defend yourself when it matters.

Or, in other words, you build excellence by putting in the effort when it doesn't matter – because excellence is a habit.

What does any of this have to do with software development? Well, have you ever thought to yourself, “Do I really need a unit test here? Ah, no, it's obviously correct.” Or, “This method looks long. Maybe I should split it? … Naw, too much work, and it's not too bad.” Or, “Why is the linter complaining? Stupid distraction! I'll just silence the warning for now.” Or maybe you didn't think too hard about corner cases because it got too complicated and when are those going to occur anyway?

Sometimes you just want to get something done and gloss over the details. And it works. You get it done and move on. Often, you can get away with skimping on code quality in the short term because there are no immediate repercussions. And implementing the happy path gets you quite far without thinking too hard. And everyone is happy, right? Because, when you get down to it, most code isn't that critical. So what if it isn't as clean as it could be, not as robust as it could be, not as [insert your favourite quality here] as it could be?

To my mind, this is akin to practicing martial arts in sloppy mode. Yes, it works … for now. There are no immediately visible negative consequences. And it is less work. It may even feel more efficient. Writing one less “useless” unit test. Not getting hung up on finding the perfect class name. Copying and pasting instead of extracting common logic because it's faster in the short term. Coding the parts users want, not getting distracted by trivialities. Getting things done!

But – and this is a big BUT – what happens when it does matter? When bugs creep in because of untested “obviously correct” code? A real example: in one of my past projects, I reviewed a pull request by a colleague and noted there was no test for a certain class he had changed. He replied that the class was just there for uniformity with the rest of the implementation and was only passing through calls to a delegate. Such a class was not worth the effort of a unit test in his view. The class did have one if statement, however – and as it turned out later, he had erroneously reversed the condition. With a unit test, he would have noticed this himself – and earlier – saving us time.

What if many small “insignificant” slow-downs add up to a performance nightmare? I once spent a lot of time speeding up a performance-sensitive application by 250%, by “just” eliminating unnecessary copying of data structures.

What if the edge cases you didn't care enough about to handle correctly suddenly start to occur en masse in production and lose you business? When I was responsible for a payment service, we were seeing errors in the logs that indicated there was a problem with customers who had not been active for a year or longer. At first, we thought it was just an edge case that was badly handled, but in reality, this edge case was occurring at a much higher rate than we would have thought, causing payment problems for real customers.

If you program in sloppy mode, your system may work as long as nothing untoward happens. If you have few users and your software is uncritical, you can fix issues as they come up with little ill effects. But it all gets much worse when we add pressure. Pressure on the system or pressure on you.

Increasing load, increasing transaction volume, increasing numbers of users put pressure on the system. Things are more likely to break when they are not built well. Performance problems manifest when the load increases. If you have a million customers, even an edge case that only occurs 0.1% of the time will affect 1000 customers. In software that is under active development, each over-complication, each misnamed entity, each code duplication increases the chances of defects due to misunderstandings or oversights, and also increases the effort required for even minor changes. This is when you need engineering excellence. Or when you notice the lack of it. And when you do, it is already too late. You would have to have built in the quality before – while it wasn't yet needed.

And then there is pressure on you: when an important deadline looms, when you need to incorporate new features faster than your competitors, when you need to scale up quickly, when the excrement hits the ventilator in production and you need a fix now, this puts pressure on you. And under pressure, you do what you always do, except sloppier and with more swearing.

If you want high quality under these circumstances, you need to have built up excellence … before. Because if you haven't developed the habit of programming with high quality while it wasn't strictly necessary, why do you think you will be able to do so now? You build excellence – the habit of excellence – by putting in the effort when it doesn't matter. And you reap its benefits when it does.

This should have been the end of my post, but between finishing and publishing it, I went to my Aikido dojo, and my teacher said something that fits perfectly. He was admonishing us to perform big movements and not leave out intermediate steps, saying,

“If it's there, you can always leave it out later. If nothing's there, you can't leave anything out.“

Think about that.

This post is part of my series on martial arts and software development.

Don't argue with the compiler

Sebastian Hans — Tue, 23 Sep 2025 00:00:00 +0000

One of the lessons I learned from both martial arts and software development is to find solutions within myself rather than wasting time blaming someone else for my problems.

When practicing a defense against a punch, if I'm hit, the defense didn't work. Simply put, that's all there is to it. Beginners sometimes complain, “But I did everything right!” They usually didn't; otherwise, the defense would have worked. Sometimes, they start blaming their partner – or even the floor (“The edge of the mat was sticking up!”). However, this misses the point. If I'm hit, I'm hit. And that's my problem to solve.

Sometimes, I practice with one partner and my take-down is working really well, and then we switch partners, and suddenly I am hit every time. The temptation to blame the new partner can be really great. After all, it worked before, right? Why did the new partner attack differently? Shouldn't they do it like the first one? I have learned to resist this line of thinking and to focus on what I need to change to cope with the way the new partner is attacking.

By now, I am an instructor myself, and there is a demonstration I like to give when I detect this spirit creeping into the training. I call out someone and tell them to punch me. Then, while they are getting ready, I say, “Ah, no, we'll do it differently. Lie down on the mat, please.” Given the situation (I'm the instructor; I'm clearly trying to demonstrate something; I'm asking nicely), they always do. Then I say something like, “If the attack comes like this, you don't need the technique.” Silence. And usually embarrassment on the part of the person lying on the ground. “Don't get me wrong – cooperation is good! We need to cooperate when we are training. And thank you for not hitting me! But if we want to practice a defense against a punch to the chin, we need a punch to the chin, not a partner lying down, not a partner holding their hand in front of my face, not a partner punching to the side.”

This demonstration only works because in this situation the person doesn't really mean to attack me. An attack is something I don't control. If the other guy does exactly what I want, that's not an attack. And if I can talk them into changing their punch to make it easier for me, well, that's not an attack either. I have to deal with the attack the way my opponent delivers it. And if I'm hit, it is I who needs to change something.

In software development, there is a very similar situation. If the compiler throws an error, my code doesn't work.^[1] Beginners sometimes complain, “But I did everything right!” They usually didn't; otherwise the code would have compiled. Sometimes, they start blaming the compiler (“It must be right! Why does the stupid compiler keep complaining!?”). However, this misses the point. If it doesn't compile, it doesn't compile. And that's my problem to solve.

Situations like this occur again and again. I'm sure everyone who has been programming for any length of time will recognize it. Why, just this week I had a problem compiling a Java program which used a third party library. The compiler complained that it couldn't find the class from the library I needed. For a second, I thought, “It must work! I did everything right!” But only for a second. Then I looked at my own changes again, and lo and behold! I had messed up the dependency declaration.^[2] I fixed it, and it worked.

Arguing with the compiler is a waste of time – as is arguing with my opponent.^[3] If I get an error, or if I get hit, I use this as feedback to fix my code or my defense.

Yes, compiler bugs do exist – but unless you are a compiler developer or using some niche language with only a handful of users, they are rare. In 30 years of software development, I've had a production problem due to a compiler bug exactly once, about 15 years ago. (It was a C program, and the bug only appeared with compiler optimization turned on, i.e., in production builds.) In all other cases, the problem was in my own code so diverting my energy to searching somewhere else would just slow me down. And even in the case that the problem really is elsewhere, rigorously validating my own code and thinking enables me to better diagnose the issue, which helps, e.g., when submitting bug reports. So, even though sometimes, very rarely, it is the compiler that's wrong, I always look at my own code first. And second. And third.

Likewise, in martial arts training, the attacker sometimes really is wrong. Defense techniques are highly situational, and if your training partner has not been paying attention, they may inadvertently change the situation enough for the defense not to work as shown. A good instructor will notice this and correct the form of the attack – and also explain the situation and show what you would need to change to deal with the other form. This is part of the instructor's job. As a defender, you should not try to correct your partner but instead focus on your own form and improve your capacity for dealing with attacks as they come. Arguing with your partner does not lead to progress.

So this is what I learned: To make progress, when I encounter a difficult situation, I look to myself and don't blame the other party, be it a compiler or a training partner. I treat the difficulty as feedback and use it to improve my own game. There is the odd exception, but looking for that shouldn't be my first reaction. Not even the second. By focusing on my part, I make progress much faster.

Footnotes

The same goes for interpreters, of course. For simplicity's sake, I'll just use the word ‘compiler’. ↩︎
The project uses Maven, and I had added the dependency to the wrong section of pom.xml – <dependencyManagement> instead of <dependencies>. ↩︎
In the training setting! Of course, if you can talk a real attacker down before you come to blows, that's much better! Because software development is inherently a learning process, it has much more in common with martial arts training than with the application of martial arts in combat or self-defense. ↩︎

Software development and martial arts

Sebastian Hans — Sat, 30 Aug 2025 00:00:00 +0000

I began my martial arts training when I was in school, at about the same time as I got my first computer, my father's old Schneider Euro PC. It came with GW BASIC, which got me into programming. Both activities have been part of my life since then. In martial arts, I started off training Goshin Jitsu, a little-know style similar to Jujutsu, then added Judo and Shotokan Karate before settling on Aikido. In programming, I progressed to QBasic, dabbled a bit in assembler, then got into C and C++ and tried countless other programming languages. After school, I studied computer science and began my career as a professional software developer.

So I've been practicing both martial arts and software development for about 30 years now, and I think I can say I have made a fair amount of progress since then – in both areas. Over the years, I have come to notice a kind of synchronicity of progress in my martial arts and software development skills. There were periods where progress stalled and I was frustrated because I felt that I was stuck. There were periods where I was preparing for a martial arts examination at the same time as I was preparing for a salary negotiation. And the major successes (taking over a martial arts class, leading a development team) came remarkably close in time, too.

On the one hand, this came unexpected because on the surface, typing code seems to be very different from throwing people to the ground. On the other hand, there obviously is a common element – me – and my personal development showing in both areas is not that surprising. Or is it? Why should progress I make in my martial arts training affect how I approach software development and vice versa? This only makes sense if there are other commonalities which make experience gained in one area somewhat transferable to the other one.

Thoughts about these commonalities have been bouncing around in my head for quite a while now, and, beginning with this blog post, I am going to write them down (in no particular order). I don't know how this is going to turn out – if my ramblings will be comprehensible to anyone other than me – but hey, it's my blog, and I can post whatever I want, so here you go.

I'll start off with the observation that developing software is not only about typing code; and martial arts is not only about throwing people to the ground, either. These are just the externally observable behaviours of software developers and martial artists, respectively. In reality, both software development and martial arts are about dealing with people. We develop software for people. To do this successfully, we need to understand their needs, not just the stated requirements. Talking to customers is as much part of software development as writing code is.

The preface to the classic computer science text Structure and Interpretation of Computer Programs says that

“programs must be written for people to read, and only incidentally for machines to execute.”

Again, this highlights the importance of people to the software development effort – this time, the developers themselves. You cannot successfully develop software by only thinking about the program text and the machines that will execute it. Future developers (including your future self) are the main audience of the code, and you need to anticipate how they are going to interpret (and change) what you wrote.^[1] As you get deeper into software development, you realize that dealing with the machines is actually the easy part. The much harder and, for better or worse, much more high-leverage part is dealing with the humans involved: customers, other developers, other roles in the organization.^[2]

Likewise, martial arts (in the sense of Budō) are about dealing with people, and by “dealing”, I don't mean the purely mechanical exercise of deflecting a blow by an opponent or applying leverage to down someone twice your weight. Ideally, martial arts are about ending conflict. As Ueshiba Morihei, the founder of Aikido, tells us in The Art Of Peace,

“The real Art of Peace is not to sacrifice a single one of your warriors to defeat an enemy. Vanquish your foes by always keeping yourself in a safe and unassailable position; then no one will suffer any losses. The Way of a Warrior, the Art of Politics, is to stop trouble before it starts. It consists in defeating your adversaries spiritually by making them realize the folly of their actions. The Way of a Warrior is to establish harmony.”

This requires seeing beyond the blow to the person dealing it, their motivation, emotional state, and capabilities. Merely deflecting a blow rarely solves a problem, and throwing your opponent to the ground only replaces one problem with another. (What will you do next?) Properly ending conflict is not achieved by fighting against an enemy (although ensuring survival is certainly necessary) but rather by engaging with the other party. Just as developing software benefits from engaging with your stakeholders rather than decoupling from them as far as possible.

As a boy, I've always been very good at the “hard” skills (logic, hitting an opponent, concise writing, hard/blocking techniques, …) and, I am afraid to say, bad at the “soft” skills (empathy, connecting with an opponent, active listening, soft/flowing techniques, …). I learned about the importance of connecting with people slowly over time. As I practised accepting attacks more softly, I came to realize that the same softness helps when accepting code reviews. When I learned about the “Yes, and” rule in a business context, I linked this mentally to the martial arts principle of using the attacker's energy instead of stopping it, strengthening the idea in my mind for use in both contexts. Synchronicity.

To be continued.

Further posts on this topic:

Footnotes

Incidentally, this is why I prefer to talk about “developing” software rather than “writing” software or programs. In the context of programming, “writing” is often confused with “typing”, even though pretty much nobody would equate those two words when talking about the production of other texts like a novel. ↩︎
I could go on, mentioning Peter Naur's Programming as Theory Building, Conway's Law, the difficulty of making, communicating and enforcing architecture decisions, etc., but suffice it to say, software development is a people business. ↩︎

Legacy software

Sebastian Hans — Tue, 29 Jul 2025 00:00:00 +0000

Why is it that no one actively writes legacy software? I mean, the world is full of it. I'm sure anyone who has been developing software professionally for a while will agree, but I haven't met anyone who has told me the code they had just written was legacy code. How come there's so much of it then? And what is legacy software, anyway?

Michael Feathers, the author of the book Working Effectively with Legacy Code^[1], defines legacy code as follows:

Legacy code is code without tests.

While pithy, I've always found this definition slightly unsatisfying. If I add a single test, is my software then non-legacy? Or does the definition apply to lines of code so that a code base with 90% test coverage contains 10% legacy code? Can software only ever become non-legacy with 100% test coverage? This doesn't feel right.

It would also mean that I could prevent my code from ever becoming legacy if I “just” wrote enough tests. I don't know about you, but I have certainly encountered code in my career that had tests yet I would still classify as “legacy”.

Steve Smith offers another definition on the Modern Software Engineering channel, citing Olly Shaw:

It's any software system with a high cost of business change. And as demand for business change increases, it can't respond quickly and safely. Change can't be sustained, something somewhere will break.^[2]

This definition is a lot broader because there may be any number of reasons why code can be hard to change, with missing tests being one of them. I mostly agree. Being hard to change is certainly a property of legacy software. Is every piece of software that is hard to change “legacy”, though? Business change comes in many forms, and no piece of software can be changed equally cheaply along all possible dimensions. If the business pivots 180 degrees, and this requires architectural changes to the software (which are typically hard to do), is this enough to declare the software “legacy”? I think not. I think this definition is too broad.

It is also merely observational, offering no guidance on what to do about it. Is it hard to change? Then it must be “legacy”. Now what? Feathers' definition has the advantage of being actionable. Write tests, and your system becomes less “legacy”.

Neither of the definitions addresses the question of how software can become “legacy“. If someone writes a piece of software now, and someone else considers it “legacy” later, what has happened in the meantime to make it so?

Let's take a look at the story of Mel^[3]. I encourage you to read the whole thing – it's a great story. But if you don't, here's the relevant part: It's about a programmer, Mel, who wrote very tricky code for a drum-memory computer. When pressured to make a change he felt was unethical, he inadvertently put a bug in it, but he liked the behaviour so much he refused to change the code afterwards. His successor, tasked with fixing the bug, struggled considerably and eventually gave up. (It reads much better in the original, really, go read it! You'll understand.)

I'm sure anyone reading this story would agree that the code was hard to change. The story doesn't mention it, but I think we can safely assume Mel didn't write any unit tests, either. Was his program legacy software? As long as Mel was in charge of it, I'd say it wasn't. When the need for a business change arose, he was able to make it easily enough, and he would have been able to fix the bug equally cheaply if he had wanted to. It was only after Mel had left and his successor had been put in charge of the program that difficulties appeared. If someone asked you to fix a bug in this program, I bet you'd consider it legacy software, whether there were any tests or not.

But the program itself hasn't changed at all! So its “legaciness” can't really be a property of the code, can it?

Enter Peter Naur. In his classic essay “Programming as Theory Building”^[4], Peter Naur presents the view that a computer program is much more than just its code:

In terms of Ryle’s notion of theory, what has to be built by the programmer is a theory of how certain affairs of the world will be handled by, or supported by, a computer program. On the Theory Building View of programming the theory built by the programmers has primacy over such other products as program texts, user documentation, and additional documentation such as specifications.

It is this theory that enables programmers

to explain the operation of the program to themselves and others,
to justify the structure of the program,
and to make business changes in a way that maintains conceptual integrity and thus long-term maintainability.

Since this theory lives primarily in the heads of the programmers, Naur describes the life cycle of a program with the notions of life, death, and revival, where:

The death of a program happens when the programmer team possessing its theory is dissolved. A dead program may continue to be used for execution in a computer and to produce useful results. The actual state of death becomes visible when demands for modifications of the program cannot be intelligently answered.

Does this sound like legacy software to you? It certainly does to me. If someone else later takes over responsibility for the program, they will not be able to respond to the demand for business change quickly and safely (as Shaw put it). The code hasn't changed, but the theory behind the program has been lost. It is not known to the successor, just as Mel's theory was not known to his successor. This is because the code (and its associated documentation) may have been handed over, or down – the word “legacy” is really appropriate here.

I therefore propose this definition^[5]:

Legacy software is any software whose theory is not known to those responsible for it.

This definition is consistent with Shaw's in that loss of the theory makes the software hard to change, but it also tells you something about how this comes about and, by extension, what to do about it: keep the theory alive. Naur describes how this can be done:

For a new programmer to come to possess an existing theory of a program it is insufficient that he or she has the opportunity to become familiar with the program text and other documentation. What is required is that the new programmer has the opportunity to work in close contact with the programmers who already possess the theory, so as to be able to become familiar with the place of the program in the wider context of the relevant real world situations and so as to acquire the knowledge of how the program works and how unusual program reactions and program modifications are handled within the program theory. […] The most important educational activity is the student’s doing the relevant things under suitable supervision and guidance. In the case of programming the activity should include discussions of the relation between the program and the relevant aspects and activities of the real world, and of the limits set on the real world matters dealt with by the program.

Naur does not explicitly say how long this period of “supervision and guidance” is supposed to take or how to recognize that the theory has been successfully transferred. My own experience with taking over responsibiltiy for existing software shows that about 2-3 months of active guidance seem to be enough for me to get the gist of it, and I take about 5-6 months to reach a point where my questions to former team members are mostly answered with something like, “Your guess is as good as mine.” This is when I consider my “apprenticeship” over and begin to drive implementation decisions from the theory I have built in my own mind.^[6] However, the TTT (time to theory) seems to vary wildly between developers, and getting there at all requires a willingness to grapple with and internalize a theory that is not your own. I have worked with colleagues who had not internalized the key concepts of the software after 2 years on the team. Since they were otherwise quite smart guys I have to conclude they simply didn't want to.

If you find people who are willing to absorb the theory and give them enough time to actually learn from the old programmers, you can keep the theory alive and thus avoid the sofware becoming “legacy“. If you don't, the theory will get lost. This can happen quite suddenly if the team is replaced completely, or slowly over time if the theory erodes due to insufficient absorption by new team members (whatever the cause).

Looking back at Michael Feathers' definition of legacy code as “code without tests”, and applying these thoughts to it, we can seen how tests can help to keep a code base alive. For one, good tests not only describe what the code does in a technical sense (e.g., “skip credit card check if the ‘preAuth’ flag is set”) but also the intent behind that (e.g., “no long-running checks during pre-auth”). They encode part of the theory – in an executable, testable way. And I bet when Michael begins implementing tests for a legacy system, he uses them not only to ensure technically that the behaviour does not change during refactoring, but also to rebuild the theory of the system in his mind.

So why does no one actively write legacy software? Because it's not legacy while they are writing it. The theory of the program is alive in their minds. It only becomes “legacy” when it becomes a legacy, i.e., when the code is passed on to other developers without also passing on its theory. Unfortunately, passing on the theory is not easily done and takes more time and effort than is usually available, which explains why there is so much legacy software around.

Consider the code you are currently writing. It isn't legacy yet. What's your plan for keeping it that way?

Footnotes

Feathers, Michael. Working Effectively with Legacy Code. Pearson. 2004 ↩︎
Smith, Steve [Modern Software Engineering]. (2025, July 16). Learn To LOVE Your Legacy Code. ↩︎
Crawford, Matt. The realest programmer of all. Newsgroup: net.jokes. November 20, 1984. Retrieved July 19, 2025. ↩︎
Naur, Peter. Programming as Theory Building. 1985. Retrieved July 19th, 2025. ↩︎
I first came up with the definition “Legacy software is any software the theory behind which has been lost,” but nowadays it is possible to vibe code software without building a theory in the first place. Following the Theory Building View of programming, this produces instant legacy software, so I had to change the wording a bit to account for this effect. ↩︎
I see a connection there to the concept of Shu Ha Ri, but this post is already long enough. Maybe I will go into it some other time. ↩︎

Endless possibilities (a socio-technical API pattern)

Sebastian Hans — Thu, 26 Jun 2025 00:00:00 +0000

Thinking about socio-technical API patterns some more, another one occurred to me. I'm calling it Endless Possibilities.

This one plays out over a long(ish) time frame. It starts out as a fairly simple API designed by some producer to satisfy the needs of a single consumer, or perhaps a small group of consumers. Let's assume the API is well-designed, maybe even co-designed between producer and consumer teams. It covers a small number of use cases and is well-tested and well-documented.

Then life happens. A new consumer is onboarded. Their needs align well with the established API, except for one small variation. The producer team addresses this by adding an optional property to one of the requests, with a backwards compatible default behaviour.

When another consumer is onboarded, their special needs are addressed in a similar manner. Maybe it is not a new property this time, but an additional enum value on an existing property. This new option conflicts with the first optional property, but this is not a problem because each consumer will only be using one of them. The API will return a helpful error message if a consumer tries to use both of them together.

Later, one of the existing consumers wants to implement a feature that requires a slightly different workflow, and the producer team needs to decide how to accommodate this: add another specialized option to the existing API methods? Special-case the API for the consumer? Implement new API methods instead? Or make the API more generic so it can handle both the old and the new workflow, distinguishing them by some other criteria? The team decides to take the latter route because this will enable not only the new workflow, but open up some additional possibilities the team feels will be interesting for some consumers. They add some documentation on how to choose between the old and the new workflow.

The lead developer of the provider team leaves the company (for unrelated reasons). Unfortunately, no one else on the team possesses quite the same level of domain expertise and historical knowledge of the technical intricacies. When the next feature is requested by a consumer, the provider team comes up with a solution that involves adding feature flags to 3 different API methods – to be used in a certain combination. One of the flags conflicts with an option added earlier – but only in certain edge cases. No one on the team knows with any amount of certainty if these are plausible scenarios. It may even be unknown whether the older option is still in use by some consumers. The team updates the documentation to indicate which scenarios are supported, but it remains partly ambiguous.

As time goes on, the API grows even more, gathering further options. Exhaustive testing of all combinations of workflows and optional behaviour becomes intractable. Natural churn in the provider team leads to knowledge loss. New features and changes are implemented in ways that almost, but not quite, fit the original intention. For consumers, the API begins to feel bloated and the documentation does not offer strong guidance on how to choose between all the options. Onboarding requires close collaboration with the provider team because the documentation alone is no longer sufficient to understand supported usage patterns and expected behaviour.

And there you have it: an API of Endless Possibilities. It has so many optional features and ways of using it that not even the provider team can answer questions about its behaviour outright. Instead of knowing, they have to do research in the code and maybe perform exploratory testing on their own API. It has lost coherence and gained sharp edges. Some combinations of features are well-tested and explicitly supported, others are untested, some are explicitly forbidden, some do work but have non-intuitive consequences. If bugs crop up, they are difficult to diagnose because it is unclear what the intended behaviour even is.

The evolution from a clean, well-designed API to an API of Endless Possibilities is a slow one. There is no clear inflection point, no moment in time you could point to and say, “This is where it went wrong.” Because of this, this pattern is difficult to avoid, but you can watch for warning signs such as:

The API has multiple options that interact in non-obvious ways.
The API has options that are incompatible for non-obvious reasons.
Options that make sense together from a business perspective are only partially implemented (some combinations are missing).
The API has multiple functions (methods, endpoints, …) that need to be invoked in a certain order – and the order depends on some option(s).
When adding a new option, it is difficult to decide how it should interact with existing options.
Certain aspects of the behaviour depend on both options sent by the consumer and configuration on the provider side.
The team cannot answer a question about API behaviour without research.
The team cannot answer a question about API behaviour without knowing which customer the question is about.

Here are some suggestions how the provider team can slow – or maybe even avoid – the slide into Endless Possibilities:

Keep the API focused on a single purpose. Ensure that everyone on the team knows this purpose and has a sense of what fits this purpose and what does not. This helps to avoid feature creep.
Resist the temptation to use options to implement features that alter the structure of the interaction between consumer and provider; provide a new API for the new interaction pattern instead. A clear separation avoids nasty edge cases that may manifest when mixing interaction patterns.
If you do add features/options, make them orthogonal to existing features/options. This makes them easier to reason about.
If this is not possible, at least clearly document the supported combinations and their semantics – and emit helpful error messages if a consumer tries to use an unsupported combination rather than allowing undefined behaviour (because of Hyrum's Law.
If a consumer does have special needs that cannot be addressed in a straightforward way, it may be better to just hardcode the desired behaviour for this consumer instead of trying for a generic implementation that adds unnecessary complexity for everyone.

Of course, this is easier said than done, and it becomes more and more difficult the further an API grows and the more consumers with diverse use cases it collects. Keeping such an API from devolving into Endless Possibilities requires strong governance with a focus on long-term maintainability. If the provider team fails to establish this, it will reach a point of no return, and it becomes practically impossible to get out of this pattern in an evolutionary way. Then, the only remaining option is revolutionary, i.e., start with a new API design from scratch – and establish stronger governance this time.

The API Leach and the API Standard

Sebastian Hans — Fri, 02 May 2025 00:00:00 +0000

A few days ago, Einar W. Høst published a blog post, Socio-technical API patterns, in which he examines some socio-technical questions about APIs and describes a handful of recurring patterns in the interaction of APIs and the people designing, providing, and using these APIs. He calls these socio-technical API patterns. I quite liked the post. If you haven't done so, go read it! (I am also going to refer to some of his patterns later on, so I suggest reading his post before mine.)

In this post, I would like to add two patterns I have personally observed: the API Leach and the API Standard.

The API Leach

The API Leach is more of an anti-pattern. It comes about when a consumer is unable or unwilling to talk to the provider, but still needs the provided service. The consumer notices an interface exposed (but not advertised) by the provider and starts programming against it without the provider being aware of this. Maybe the interface is an internal API, or maybe it is not intended as an API at all, but as a UI for human users. Scraping web pages for data is one instance of this pattern.

Since the interface is not intended for consumption by external programs, the unwitting provider does not provide documentation, service levels, or support channels to the consumer team. The interface behavior is reverse engineered and may change at any time as the provider makes changes to the system. The interface may disappear without notice, too.

For the consumer team, this is a hard spot to be in. Reverse engineering an undocumented interface is inherently hard and error-prone, and once they are done, the team must be ready to deal with unexpected behavior (and failure) at any time – be it due to misunderstanding during the reverse engineering phase or to technical or functional changes. In all probability, they won't have a test environment, either, and need to test against the production interface with production data, which can severely limit the range of tests that are viable.

The provider team is mostly unaffected, but may notice some oddities:

unexpected usage patterns (because the consumer's needs probably differ from those of the provider team, or from regular human users),
an increased error rate (the “error” part in reverse engineering by trial and error),
increased load (which can, in extreme cases, even dominate the total load),
strange support requests (by the consumer trying to get information from the provider without revealing what they are doing).

These may cause the provider team to notice that something is up. And, of course, the producer may get yelled at after performing breaking changes if the consumer feels entitled enough.

Once the producer learns of the existence of the consumer, what happens next depends mostly on the power dynamics between the two parties. The interface may become an official API; the consumer may be shut out; both parties may negotiate a different API.

Another factor that determines the consequences is the intention of the consumer. The API Leach pattern is often used because the consumer is trying to accomplish something the producer would not be inclined to agree with – or at least has no incentive to support. In these situations, there may not be a good alternative, and if the leach is discovered, the provider will probably attempt to shut down the “integration”.

The pattern can also be established inadvertently, however, e.g., if the consumer fails to contact the producer due to lack of time or knowledge. In these cases, the situation can be resolved by establishing communication and explicitly agreeing on a different integration pattern.

It can also be the result of a team trying to put a Millstone on another team, but failing because they don't have the necessary clout. Sometimes, downstream failures due to changes by the unwilling provider can gather enough management attention to rectify the situation – or at least to make the Millstone possible after a period of leaching, which might be an improvement for the consumer, if not for the provider.

The API Standard

Using a documented API Standard is another pattern that extends beyond the merely technical considerations into the socio-technical domain. Using a standard decouples provider and consumer. The standard was designed for a particular purpose and is adopted by both provider and consumer, presumably because their needs align with this purpose. Collaboration on the API design is minimal because most of it is defined by the standard.

Most standards say little about operational aspects, however, so the service level offered by the provider most likely depends on other factors. Therefore, this pattern is usually combined with other patterns.

In conjunction with the Millstone, where a software team is forced to provide an API to the consumer without having an intrinsic motivation to do so, the API Standard alleviates the problem of coupling between the provider's internal data model and the API because the provider will typically have to implement some kind of mapping anyway. It is very unlikely that the internal model corresponds to the standard already. Thus, the Standard establishes a stable API contract that would be difficult to enforce otherwise. It does not solve the problem of "best effort" SLAs, though.

When combined with the Mountain, where the consumer is dependent on a provider with little to no influence on the API design and the API lifecycle, the Standard provides a measure of protection from the effects of the Volcano (the provider unilaterally deciding to discontinue – or massively alter – the API). If this happens, there will often be other providers using the same standard that can be moved to with little changes to the application code. If a data migration is necessary, the Standard may or may not be of help, depending on whether it specifies export/import functionality.

The Standard does probably not help with the Rapids, where an API is located in the middle of a value stream, but it may help with the Sock Puppet, where a team is talking to itself through an API. When a team manages multiple services, probably the requirements are not equally unique for all of them. There may well be services that perform standardized functions for which usage of a standardized API can reduce the cognitive load on the team and simplify onboarding for new team members.

A disadvantage of using an API Standard manifests if the needs of the consumer diverge from what the standard provides over time. In this case, you have the choice of forcing your needs into the corset of the standard (often suboptimal), augmenting the API with non-standard extensions (may work, but limits the usefulness of the standard), or ditching the standard altogether.

The API Leach may benefit from a provider that adheres to an API Standard (or maybe human interface guidelines), too, because this reduces the chances of breaking changes – although there is no guarantee that the provider will keep to the standard indefinitely – or even that the provider implements the standard correctly. Thus, the Standard does not solve the Leach's fundamental problem, but it may reduce the pain somewhat.

Conclusion

In this post, I wrote down two additional socio-technical API patterns. The API Leach describes the situation where a consumer taps into an “unofficial” interface of an unwitting provider. The API Standard describes how usage of an … API standard affects various other patterns.

If you have observed other patterns, I would appreciate it if you shared them.

Incremental danger

Sebastian Hans — Wed, 23 Apr 2025 00:00:00 +0000

I'm all for incremental development. Incremental development is great. It reduces risk by dividing complex changes into small – less complex – parts. It allows for early delivery of value. It enables learning and discovery. It simplifies development by allowing you to focus on one part of the problem in turn. That's good, right?

Yes, it is, but there is a very important caveat: you have to make sure the parts fit together – and they actually solve the problem in the context where it needs to be solved!

Let's look at a simple example that was developed incrementally.

A simple problem

A team was in the process of containerizing their service. As a result, they had been generating container images at a relatively high rate. This was noticed by the administrator of the central image repository, and the team was asked to delete images they no longer needed and to institute some kind of policy to limit the number of images. The team decided to implement an automated clean-up job to delete obsolete images. Due to the team's (mostly) roll-forward strategy in case of problems, most old images would never be needed again and could be deleted. It was deemed enough to be able to roll back for 7 days or 10 images (a somewhat arbitrary number). These were the images they wanted to keep. Everything else could be deleted.

At the time, the rate of deployment varied between a few times a week and once every few weeks, tending towards the longer time scale^[1]. Revisions (images) could spend a few days or in some cases weeks (for complex features) in the test environment before being deployed to production.

Let's follow the developer – call him George – tasked with implementing the clean-up job.

Increment 1

George implements the first version of the clean-up job this way:

enumerate all images,
filter out those younger than 7 days (the ‘keep list’),
delete the rest.

This works fine, and the job is scheduled to run nightly.

Before reading further, please take a moment to think about this implementation. What do you think will happen given the workflow described above?

Time to think …

Do you see the problem?

George doesn't. He thinks everything is fine.

But then, after a few nights, the job deletes the image that is currently running in production because it is older than 7 days. Fortunately, the team notices this before it turns into a serious problem. If the runtime platform had decided it needed to pull the image again due to a restart, this could have resulted in an outage!

Increment 1.1

Regenerating the deleted image is easy – just run the CI pipeline again.

George then fixes the clean-up job by looking up which image is running in production, taking its timestamp, and calculating 7 days backwards from there instead of from the time the job is run.

This is a bit awkward to test because there no longer are old images, but the code looks OK, so the fix is deployed, and production images are safe again.

Increment 2

George then implements keeping at least 10 images by skipping deletion if the ‘keep list’ from above has less than 10 entries.

Again, before reading further, take a moment to think about this solution. What do you think will happen?

Time to think …

At first glance, this implementation does satisfy all constraints. The image running in production is preserved, images younger than 7 days are preserved, and at least 10 images are preserved. So it's good, then?

Except preserving images is not the point of the exercise. The team had this before the clean-up job was introduced and they never deleted anything. The main goal is to delete stuff.

How does this solution delete images? It will delete images older than 7 days if there are at least 10 images younger than 7 days^[2]. This means the job will only delete anything if the team has deployed to production at least 10 images over the past 7 days, i.e., on average at least 2 deployments per day^[3]. Given the actual deployment frequency of the team, the job will never actually delete anything.

Increment 2.1

This fact is discovered during a code review, and George fixes this by changing the logic. The job now looks at each image, going backwards in time from the production image, and adds images to the ‘keep list’ until a time difference of 7 days or the number of 10 images have been reached. All images older than that are deleted.

Now the job actually deletes old images while preserving those it is supposed to preserve.

Retrospective

To recap, the team started off with a simple requirement: don't keep too much old stuff, but keep enough to be able to roll back in case of trouble. They formulated this in terms of two rough rules, and George went off to implement them incrementally.

The first increment deleted too much. The reason was a misunderstanding of the requirement: being able to roll back for 7 days after a production deployment versus keeping images younger than 7 days from the perspective of now. Realizing that there is a significant difference hinges on the understanding of the context the solution has to work in (the deployment frequency and the duration between image generation and deployment in production). Focusing solely on the task of delivering the first increment, George forgot to consider this context – and with a little bit of bad luck could have caused a production outage.

When taking on the 10 day ‘keep’ rule, George again focused solely on the current task, and did not consider how his implementation would interact with the previous increment and the context (the team's deployment frequency). This time, his ‘solution’ failed to do anything at all.

Conclusion

This example shows how easy it is to lose your place if you consider only the step right in front of you. While incremental development allows you to focus on one small change at a time (and this is one of its best features), this does not mean you should run around with blinders. You still need to be aware of the bigger picture to ensure your one small change actually does what it is supposed to do – and doesn't do what it is not supposed to do.

This bigger picture includes

what is already there (for the second increment, this is the first increment),
the context in which the solution will operate (in George's example, the team's deployment process and cadence),
the intent behind it all (here, to delete unnecessary stuff).

When I develop incrementally, I ask myself this question for each increment:

When this is integrated and running in the target context, will the solution as a whole support the original intent?

Hopefully, the answer is “yes”. But if it isn't, I can still celebrate, because I have caught a flaw before it could cause damage. And I now have the opportunity to rethink my approach and do better.

Footnotes

It had gone down somewhat from the peak during initial experimentation ↩︎
Always counting backwards from the image currently in production. ↩︎
No deployments on weekends. ↩︎

On planning, part 5: exploration

Sebastian Hans — Mon, 17 Mar 2025 00:00:00 +0000

This is the fifth and last post in my series about planning in the context of agile teams.

Part 1 is concerned with why you should plan at all.
Part 2 describes my iterative approach to end-to-end planning.
Part 3 shows how I do forwards and backwards planning.
Part 4 demonstrates how I use the critical path.

In this post, I will explore … exploration.

Happy path planning

We'll start with a very simple plan. The following picture shows how to get from Krottenkopfstraße in Munich to the hospital Klinikum Großhadern.

A very simple plan showing a route through Munich.
Drawn using map data by OpenStreetMap contributors.

You start off going north, turn 5 times, and you are there. The street names are on the plan. What more do you need?

If everything goes well – nothing. If everything goes well, you will get from here to there using only this plan. If everything goes well. How likely is that? In this particular example, and if you are walking, it is pretty likely. If you are going by car, it's a little less likely. There may be construction work blocking a street, traffic jams, or you might not find a parking space once you are there. What are you going to do then?

This plan is fragile in the sense that it works while you can follow it strictly, but if you deviate from it even slightly, it can no longer tell you what to do. And this is a very simple plan for a very simple ‘project’. If your undertaking is more complex – say, like a software development project –, something is bound to go wrong, where ‘wrong’ means deviating from the happy path.

Now, you might say, this is a stupid example. Obviously, I got the route from OpenStreetMap, and why would I blank out everything but the chosen route? Why not show the whole map so I can immediately see alternative routes if I need them? And you'd be right. Here is the map, including the route.

A much more useful and resilient plan.
Map data by OpenStreetMap contributors.

With this map, my plan becomes much more resilient: if I deviate from the route, the map allows me to find back to it or to quickly choose a new route to the target if this seems more appropriate. If getting from here to there is in any way important, I'd be stupid not to use the map if I have it available.

But how often do we have such a map when embarking on a software development project? The plans I have seen often resembled the first picture much more than the second. This is what I call ‘happy path planning’. It presupposes that all will go well and makes little provision for unexpected obstacles.

Ask, ‘What can go wrong?’

If and when I make plans, I always think about things that could go wrong. I explore what could go wrong and think about appropriate responses and alternative ways forward.

I make a point of not being content after devising the first possible plan that could work out (if everything goes well), but to think about other options we have and risks associated with each of them. For example, I can do even better in the routing example above by including the timetable of the subway line U6 in my plan because it could take me most of the way if I wanted to. It adds another option.

I examine the dependencies among tasks and possible orderings. If doing the tasks in one order results in a bottleneck (especially one at a later stage of the project), and doing them in another order increases flexibility, I prefer the latter.

In effect, I am building a map that not only shows one single path but a web of paths, so if some obstacle blocks our way, a solution becomes apparent quickly by looking at the alternatives I have considered. Having this map enables me to respond to change better. (Remember this? It is one of the values from the Manifesto for Agile Software Development.)

This doesn't mean that I don't have a preferred way forward. I do plan a route – a ‘happy path’ – on this map. But – and this is a big ‘But’ – I am prepared to respond to changing circumstances much better than if I had only considered the happy path.

Also, simply knowing that we have fallback options provides much-needed ease of mind in times of high pressure. And this ease of mind sometimes allows us to perform feats that are impossible when we are stressed out. ‘We would like to get this subtask done before the deadline, but we could deliver it later if necessary,’ can take out so much pressure. So maybe having this plan is what allowed us to succeed, even though – on the surface – we did not actually ‘need’ any of those alternatives.

Here we are with general Dwight Eisenhower again:

Plans are useless but planning is indispensable.

Having planned this way (and having built a map around our plan), we are prepared for whatever life will throw at us, regardless of how much of the initial plan actually works.

Conclusion

That's it for my ‘planning’ series. It has gotten much longer than I had planned (hah!).

In conclusion, yes, I think planning is important, even for agile teams, especially if they take on larger pieces of work, have to deal with deadlines, or depend on external resources which may not be easily available. I use forwards and backwards planning to come up with an end-to-end plan that I refine continuously. When dealing with deadlines, I use the concept of the critical path to ensure my plan fits the available time. And I draw up not only the happy path where nothing must go wrong, but I also explore risks and alternatives to be prepared for change because everything flows.

Color scheme switching for WezTerm

Sebastian Hans — Sat, 15 Feb 2025 00:00:00 +0000

I really like WezTerm, a cross-platform terminal emulator and multiplexer I have been using for some time now. What particularly drew me to it was the combination of

cross-platform support,
tabbing,
support for panes,
ligatures, and
powerful customization with Lua.

WezTerm also supports color schemes, and I use one based on the beautiful Selenized color palette, in particular Selenized black & white. While WezTerm comes with a lot of color schemes, at the time of writing, support for Selenized Black and Selenized White is only present out of the box in the nightly builds (via Gogh). This is why I wrote my own color scheme definitions and put them in ~/config/wezterm-color-schemes so they can be picked up by WezTerm. Here they are.

Selenized black & white

Selenized Black.toml:

[colors]
ansi = [
   "#252525",
   "#ed4a46",
   "#70b433",
   "#dbb32d",
   "#368aeb",
   "#eb6eb7",
   "#3fc5b7",
   "#777777"
]
brights = [
   "#3b3b3b",
   "#ff5e56",
   "#83c746",
   "#efc541",
   "#4f9cfe",
   "#ff81ca",
   "#56d8c9",
   "#dedede"
]
background = "#181818"
foreground = "#b9b9b9"
cursor_bg = "#83c746"
cursor_border = "#83c746"
cursor_fg = "#181818"
selection_bg = "#b9b9b9"
selection_fg = "#181818"

[colors.indexed]

[metadata]
aliases = []
name = "Selenized Black"

Selenized White.toml:

[colors]
ansi = [
  "#ebebeb",
  "#d6000c",
  "#1d9700",
  "#c49700",
  "#0064e4",
  "#dd0f9d",
  "#00ad9c",
  "#878787"
]
brights = [
  "#cdcdcd",
  "#bf0000",
  "#008400",
  "#af8500",
  "#0054cf",
  "#c7008b",
  "#009a8a",
  "#282828"
]
background = "#ffffff"
foreground = "#474747"
cursor_bg = "#008400"
cursor_border = "#008400"
cursor_fg = "#ffffff"
selection_bg = "#474747"
selection_fg = "#ffffff"

[colors.indexed]

[metadata]
aliases = []
name = "Selenized White"

Switching between color schemes

This is all well and good. However, another thing WezTerm is missing out of the box is an easy way to switch color schemes. I mostly use dark colors, but sometimes I want to put a screenshot of my terminal in a document and want it to look bright, or I am sharing my screen with someone who can read better when the text is dark on light, so having a convenient way to switch betwwen dark and light modes is useful to me.

Fortunately, WezTerm is highly configurable, so I was able to implement the toggle myself. Here is a minimal, but heavily commented, WezTerm configuration that shows my solution. I have included links to the relevant WezTerm documentation so you can read up on what each of the parts does if you wish.

Standard preamble:

local wezterm = require 'wezterm'
local config = wezterm.config_builder()

Look up home directory to construct some paths (this is standard Lua):

local home = os.getenv('HOME')

WezTerm will look for color scheme files in the directories in color_scheme_dirs:

config.color_scheme_dirs = { home .. '/config/wezterm-color-schemes' }

We will persist the name of the current color scheme in ~/.color-scheme:

local color_scheme_file = home .. '/.color-scheme'

We will toggle between these two color schemes:

local default_color_scheme = 'Selenized Black'
local alternate_color_scheme = 'Selenized White'

In my case, these are the custom schemes described above, but they don't have to be. You can name any color scheme WezTerm is able to find, custom or built in.

Function to load the name of the current color scheme, falls back to default_color_scheme if the file is not present:

local function load_color_scheme()
  local color_scheme = default_color_scheme
  local file = io.open(color_scheme_file, 'r')
  if file then
    color_scheme = file:read('*a')
    file:close()
  end
  return color_scheme
end

Function to save the name of the current color scheme:

local function save_color_scheme(color_scheme)
  local file = io.open(color_scheme_file, 'w+')
  if file then
    file:write(color_scheme)
    file:flush()
    file:close()
  end
  return color_scheme
end

Make WezTerm use the saved color scheme by assigning it to color_scheme:

config.color_scheme = load_color_scheme()

This is an event handler for the custom event toggle-color-scheme. It determines which scheme to switch to and saves it.

wezterm.on('toggle-color-scheme', function(_window, _pane)
  local color_scheme = load_color_scheme()
  local new_color_scheme = default_color_scheme
  if color_scheme == default_color_scheme then
    new_color_scheme = alternate_color_scheme
  end
  save_color_scheme(new_color_scheme)
end)

I use Ctrl-b as the leader key, i. e., as a prefix for my custom keyboard shortcuts:

config.leader = { key = 'b', mods = 'CTRL', timeout_milliseconds = 1000 }

Now we configure the key binding to toggle the color scheme. Specifying both LEADER and CTRL makes it so I have to press the leader (Ctrl-b), followed by Ctrl-c.

config.keys = {
  {
    key = 'c',
    mods = 'LEADER|CTRL',

Use Multiple to trigger two actions when the key combination is pressed:

    action = wezterm.action.Multiple {

First action: fire the event that will save the new color scheme:

      wezterm.action.EmitEvent 'toggle-color-scheme',

Second action: make WezTerm reload its configuration, thus activating the new color scheme:

      wezterm.action.ReloadConfiguration,
    },
  },
}

Finally, return the configuration to WezTerm:

return config

And here is the entire thing:

local wezterm = require 'wezterm'
local config = wezterm.config_builder()
local home = os.getenv('HOME')

config.color_scheme_dirs = { home .. '/config/wezterm-color-schemes' }
local color_scheme_file = home .. '/.color-scheme'
local default_color_scheme = 'Selenized Black'
local alternate_color_scheme = 'Selenized White'

local function load_color_scheme()
  local color_scheme = default_color_scheme
  local file = io.open(color_scheme_file, 'r')
  if file then
    color_scheme = file:read('*a')
    file:close()
  end
  return color_scheme
end

local function save_color_scheme(color_scheme)
  local file = io.open(color_scheme_file, 'w+')
  if file then
    file:write(color_scheme)
    file:flush()
    file:close()
  end
  return color_scheme
end

config.color_scheme = load_color_scheme()

wezterm.on('toggle-color-scheme', function(_window, _pane)
  local color_scheme = load_color_scheme()
  local new_color_scheme = default_color_scheme
  if color_scheme == default_color_scheme then
    new_color_scheme = alternate_color_scheme
  end
  save_color_scheme(new_color_scheme)
end)

config.leader = { key = 'b', mods = 'CTRL', timeout_milliseconds = 1000 }
config.keys = {
  {
    key = 'c',
    mods = 'LEADER|CTRL',
    action = wezterm.action.Multiple {
      wezterm.action.EmitEvent 'toggle-color-scheme',
      wezterm.action.ReloadConfiguration,
    },
  },
}

return config

Saving this to your home directory as .wezterm.lua sets you up to toggle between light and dark mode by pressing Ctrl-b Ctrl-c.

Using this combination of persisting the color scheme and config reload means that the change applies to all open WezTerm panes and windows (because they all belong to the same WezTerm instance).

Switching the current pane only

I also wrote a script to change the color scheme for the current pane, but this one only works for my custom color schemes because it reads the TOML files and outputs the escape sequences to change the terminal color palette. The script expects to live in the same directory as the TOML files and requires toml-cli to work. It does not toggle between two color schemes, but expects a color scheme name as an argument. Here it is:

#!/usr/bin/zsh
if [ -z "$1" -o -n "$2" ]; then
  echo "Usage: $0 COLOR_SCHEME_NAME" >&2
  exit 1
fi
SCHEME="$1"
HERE="$(dirname "$0")"
TOML="$HERE/$SCHEME.toml" 
if ! [ -f "$TOML" ]; then
  echo "Color scheme not found at $TOML" >&2
  exit 1
fi

function color() {
  case "$1" in
    0|1|2|3|4|5|6|7) toml get -r "$TOML" "colors.ansi[$1]" ;;
    8|9|10|11|12|13|14|15) toml get -r "$TOML" "colors.brights[$(($1-8))]" ;;
    *) toml get -r "$TOML" "colors.$1" ;;
  esac
}

echo -en "\\e]4;0;`color 0`;1;`color 1`;2;`color 2`;3;`color 3`;4;`color 4`;5;`color 5`;6;`color 6`;7;`color 7`;8;`color 8`;9;`color 9`;10;`color 10`;11;`color 11`;12;`color 12`;13;`color 13`;14;`color 14`;15;`color 15`\\a"
echo -en "\\e]10;`color foreground`;`color background`;`color cursor_bg`\\a"
echo -en "\\e]17;`color selection_bg`\\a"
echo -en "\\e]19;`color selection_fg`\\a"

It have not yet created a key binding to invoke this script, but it should be possible with a bit of Lua code (emit a custom event and handle it by executing the script in a subprocess, making sure the output goes to the correct pane).

Where to now?

For now, I am happy with my setup. I can toggle between dark and light modes with a keybinding. Possible extensions I have thought about, but not yet implemented are:

toggling the current pane via a key binding,
cover more color options with the color schemes (e. g., tab bar colors),
integrate color schemes into the launcher menu.

Mouse follows window GNOME extension

Sebastian Hans — Sat, 25 Jan 2025 00:00:00 +0000

tl;dr

I wrote a GNOME extension to make focus follows mouse play nicely with keyboard window manipulation shortcuts. the Mouse follows window GNOME extension. This post gives a bit of background and walks you through the implementation.

Building stuff to make my life easier is fun! You can do it, too!

The problem

When I changed my employer at the beginning of the year, I was happy that they let me have a laptop with Linux on it. It came with Ubuntu 22.04 preinstalled, which is not the most recent version, but since this is the only Linux distribution and version supported by the IT department, I'm living with it. Ubuntu 22.04 comes with GNOME 42.9 by default. At first, I thought I might install i3wm, which is what I use on my personal machine, but actually, GNOME isn't too bad, and some things just work out of the box, so I decided to stick with it.

One thing I really like about i3wm, a tiling window manager, is the ability to shuffle windows around with keyboard shortcuts. And focus follows mouse. I really can't use a desktop environment where I have to click on windows to focus them. I want to point my mouse at a window and start typing into it. When using the laptop keyboard, this is just a quick movement with my thumb over the touchpad.

Luckily, GNOME supports focus follows mouse and has shortcuts for what they call ‘split left’ and ‘split right’, which means make a window cover the left (or right) half of the screen, as well as maximizing and minimizing windows and moving them between monitors. So far, so good.

Unfortunately, these two features do not play well together. When I am working in a window on the left half of the screen and split it to the right (e.g., because I want to look at something in a window underneath the active window), the mouse pointer stays on the left half of the screen, and the focus switches to whatever is underneath it now. This is typically not what I want. What I really want is for the mouse pointer to move with the window – and the focus to stay in the window. This is the behaviour of i3wm, and I want to replicate it in GNOME.

Also, sometimes splitting a window to a half of the screen that is already occupied by another window results in the moved window being hidden behind the other window. I guess the compositor doesn't change the z index when moving a window this way. This annoys me, and I want to fix this, too.

The solution

After some searching around, I decided that what I needed was a GNOME extension that watched for these window ‘split’ operations and would then warp the mouse pointer so it stayed in the window.

Another option would have been to implement the ‘split’ operation myself and move the window and the mouse pointer together. I may still do that because it would allow me to add custom window positions, but for the moment, left and right half are enough for me, and I want to keep things simple.

Before I come to the implementation, there is one important detail to consider: the system is running Wayland.

Interlude: Mouse pointer warping in Wayland

Wayland is more secure than X11, and one consequence of this is that moving the mouse pointer around programmatically is not that simple. Basically, you have to emulate an input device, which is a bit more work than I wanted to take on for such a small feature.

Fortunately, others have already done the work. There are tools that do exactly this. They emulate input devices and generate input events. One such tool is ydotool. Another is input-emulator. Both offer a CLI that accepts coordinates and moves the mouse pointer.

For my use case, ydotool would have been better because it supports absolute positioning: give it an x/y coordinate, and it will warp the mouse pointer to this position. When I tried ydotool, however, it always moved the mouse pointer to the left margin of the screen. The vertical position (y) was correct, but x was always 0, no matter how I called it. When I tried ydotoold, a daemon belonging to ydotool that provides a persistent virtual input device in the background, it segfaulted on me. Rather than trying to analyze this, I decided to go with input-emulator instead, which did work. The only downside is that input-emulator only supports relative positioning, i.e., I cannot simply say, ‘go there’. I have to calculate the target position relative to the current position of the mouse pointer.

Now that I know how to control the mouse pointer, I can implement the extension.

The implementation

You can find the implementation at github.com/sebhans/mfw-gnome-extension. It is JavaScript and basically consists of a little boilerplate and a single class with about 100 SLOC.

The idea is to track all window geometries (positions and sizes) and which window is focused. When the geometry of the focused window changes and the change is larger than 50 pixels, we assume the window has been moved by a keyboard shortcut^[1] and check the position of the mouse pointer. If the mouse pointer is still inside the window, we do nothing (since the window will still have the focus, there is no need to make the pointer jump around). If the mouse pointer is outside the window, however, we move it to the center of the window. And we also ensure that the window is in the foreground to get rid of the ‘hidden window’ phenomenon mentioned above.

Watching for position/size jumps instead of just checking whether the window covers half the screen lets the extension handle other window movements that would result in focus loss, like unmaximizing a window and moving a window to another monitor, without extra effort.

Here is a walkthrough of its methods (leaving out some debug statements).

constructor() {
  this.windowCreatedSignal = null;
  this.windowSignals = new Map();
  this.windowGeometries = new Map();
  this.focusedWindowId = null;
}

The constructor initializes the data structures we are going to use:

windowCreatedSignal and windowSignals will keep information about the signals we use to track windows. More on this later.
windowGeometries will keep track of the geometries (positions and sizes) of all windows.
focusedWindowId will track the ID of the currently focused window.

enable() {
  Util.spawn(['/usr/local/bin/input-emulator',
              'start', 'mouse',
              '--x-max', '5000',
              '--y-max', '5000']);
  this.windowCreatedSignal =
    global.display.connect(
      'window-created',
      this._onWindowCreated.bind(this)),
  this._connectAllWindows();
}

This method is called when the extension is enabled. It sets up input-emulator to emulate a mouse. This starts a background process that handles the virtual device and allows us to perform pointer movements later on. The parameters --x-max and --y-max specify the maximum coordinates to use. The default is too small for my monitors. 5000 for each is somewhat arbitrary, but I use the laptop in various setups with external monitors with different resolutions, and 5000 works for me.

The method also registers _onWindowCreated() as a signal handler that will be called whenever a new window is created (so we can track it) and immediately begins to track all already existing windows with the helper method _connectAllWindows().

_connectAllWindows() {
  global.get_window_actors().forEach(actor => {
    this._connectWindowSignals(actor.meta_window);
  });
}

_connectWindowSignals(window) {
  if (this.windowSignals.has(window)) {
    return;
  }

  let signals = [
    window.connect(
      'focus',
      this._onFocusWindowChanged.bind(this)),
    window.connect(
      'position-changed',
      this._onWindowChanged.bind(this)),
    window.connect(
      'size-changed',
      this._onWindowChanged.bind(this)),
  ];

  this.windowSignals.set(window, signals);
}

Together, these two methods set up signal handlers for every existing window. As a result, the handler _onFocusWindowChanged() will be called whenever another window receives the focus, and the handler _onWindowChanged() will be called whenever a window changes its position or size. The size is relevant because the window could already be at the correct position, but still lose focus, e.g., because it changes from maximized (at position 0/0) to unmaximized, still at position 0/0, but much smaller, so that the mouse pointer is now outside the window.

The signals returned by the connect() method are saved in the map windowSignals so we can clean them up later (as is the global 'window-created' signal in the enable() method above).

disable() {
  if (this.windowCreatedSignal) {
    global.display.disconnect(
      this.windowCreatedSignal);
    this.windowCreatedSignal = null;
  }
  this._disconnectAllWindows();
  this.windowGeometries.clear();
  Util.spawn(['/usr/local/bin/input-emulator',
              'stop', 'mouse']);
}

_disconnectAllWindows() {
  this.windowSignals.forEach(
    (signals, window) => {
      signals.forEach(signalId =>
        window.disconnect(signalId));
    }
  );
  this.windowSignals.clear();
}

The method disable() is called when the extension is disabled. It is supposed to free all resources used by the extension, which in our case means disconnecting all signal handlers we set up in enable() and _connectWindowSignals() and stopping the input-emulator background process responsible for our virtual mouse.

So far, so good. The extension will now come up, register all necessary handlers to be notified of window changes, and shut down gracefully. But up to now, it can only handle windows that existed when the extension was enabled (which will probably be none if the extension is loaded at the start of the GNOME session). We need to change that. This is what the _onWindowCreated() handler is for.

_onWindowCreated(display, window) {
  switch (window.get_window_type()) {
    case Meta.WindowType.MENU:
    case Meta.WindowType.DROPDOWN_MENU:
    case Meta.WindowType.POPUP_MENU:
    case Meta.WindowType.TOOLTIP:
    case Meta.WindowType.NOTIFICATION:
    case Meta.WindowType.COMBO:
      return;
  }
  this._connectWindowSignals(window);
  this.windowGeometries.set(
    window, window.get_frame_rect());
}

This handler is called whenever a new window is created. We want to set up our window handler for this new window, too (and remember its geometry while we are at it). There's just one minor complication: there are several types of windows, and not all of them are relevant for us. In my first experiments, I didn't pay attention to the window type and was surprised when the mouse pointer jumped around while I tried to delete a row in a table in LibreOffice Writer via the context menu. I don't know exactly why, but menus, tooltips and the like tend to trigger our ‘jumping window’ reaction, and since shooting those kinds of windows around the screen via keyboard doesn't make sense anyway, we may as well filter them out right from the start.

_onFocusWindowChanged(window) {
  this.focusedWindowId =
    window.get_id();
}

This handler is called whenever a window receives the focus and just remembers its ID for later.

_onWindowChanged(window) {
  let frame = window.get_frame_rect();
  if (window.get_id() != this.focusedWindowId) {
    this.windowGeometries.set(
      window, frame);
    return;
  }

  let oldFrame =
    this.windowGeometries.get(window);
  if (oldFrame &&
      this._hasWarped(frame, oldFrame)) {
    this._ensureMouseIsIn(window);
  }
  this.windowGeometries.set(
    window, frame);
}

This handler is called whenever a window changes its position or size. We only need to do anything interesting if it is the focused window (this check is what we kept track of this for). If it is not, we just remember its new geometry and return immediately.

If it is, we check to see whether the window has jumped (‘warped’ I called it in the code) and if so, we call a helper method to move the mouse pointer if necessary. And we also remember its new geometry.

const WARP_DISTANCE = 50;

_hasWarped(frame, oldFrame) {
  return Math.abs(frame.x - oldFrame.x) >= WARP_DISTANCE
      || Math.abs(frame.y - oldFrame.y) >= WARP_DISTANCE
      || Math.abs(frame.width - oldFrame.width) >= WARP_DISTANCE
      || Math.abs(frame.height - oldFrame.height) >= WARP_DISTANCE;
}

_hasWarped() implements our warp detection. We consider the window to have warped if any of its coordinates or dimensions has changed by at least our 50 pixel threshold.

_ensureMouseIsIn(window) {
  const frame = window.get_frame_rect();
  let [mouse_x, mouse_y] = global.get_pointer();
  if (mouse_x >= frame.x &&
      mouse_x < frame.x + frame.width &&
      mouse_y >= frame.y &&
      mouse_y < frame.y + frame.height)
    return;

  window.activate(global.get_current_time());

  let target_x = frame.x + frame.width / 2;
  let target_y = frame.y + frame.height / 2;
  let dx = target_x - mouse_x;
  let dy = target_y - mouse_y;
  Util.spawn(['/usr/local/bin/input-emulator',
              'mouse', 'move',
              `${dx}`, `${dy}`]);
}

This is the real meat of the extension. This function ensures that the mouse pointer is in the newly warped window. First, we check whether the pointer is still inside by comparing its coordinates with the area covered by the window^[2]. If so, there is nothing to do and we return immediately.

If the coordinates of the mouse pointer are outside the window, we first activate it, which includes bringing it to the foreground to that it cannot be hidden behind another window. Then we calculate the target coordinates (where we want the mouse pointer to be) at the center of the window and the difference of this position to the current position of the mouse pointer (how far we have to move it). And lastly, we call input-emulator to perform the actual movement.

That's it for the code. I noticed some improvements I could have made while writing this post^[3], but I'm letting it stand for now. I've been using the extension for a whole workday now, and I'm pretty pleased with it.

Limitations

I wrote this extension to scratch an itch with minimal effort. As a result, it is narrowly tailored to my use case. It is also the first GNOME extension I have ever written, which is certainly not without consequence. These are the limitations I am currently aware of:

GNOME 42.9
I wrote this for GNOME 42.9 because that is what I have on my work laptop. Extensions work differently in GNOME >= 45, so the extension probably requires changes to be useable with recent GNOME versions.
Hard-coded path to input-emulator
I know how I set up input-emulator on my laptop, so this is not a concern for me. Making the path configurable (or even bundling the tool with the extension) would make reuse simpler^[4].
Interference
The extension's use of input-emulator might interfere with other uses of this tool. I don't use it for anything else, so this doesn't bother me.
Memory leak
As of now, I haven't been able to find a signal that tells me when a window has been destroyed, so windows are never removed from the maps windowSignals and windowGeometries. This means that these maps will accumulate obsolete entries, resulting in a memory leak. I have not yet managed to create enough windows for this to be a noticeable problem, but I still don't like it. I'm sure there is a way to be notified on window destruction, I just haven't found it yet.
Pointer bumping into screen edges
Sometimes when moving a window to another monitor, the mouse pointer bumps into the edge between the monitors and does not reach the target coordinate. I assume this is because my laptop screen and the external monitor have different resolutions and thus the combined desktop is not a perfect rectangle, which means that there are pairs of coordinates that cannot be connected by a straight line without leaving the desktop. I further assume that the mouse emulation performs pointer movements by simulating mouse movements in straight lines, which does not work in these – hah – edge cases. I have some ideas for how to address this, but this happens rarely enough that I'm not yet sure whether it is worth the bother.

Resources

Here is a collection of links I found useful while I tried to get the extension working (in addition to those mentioned above):

GNOME extension documentation (legacy version for GNOME <= 44)
API documentation for GNOME applications in JavaScript
Especially the Meta.* classes
gnome-shell-touchpad-window-move – as a working example of a GNOME extension; I didn't use any code from there, but I got a feel for how extensions work

Conclusion

While the extension is very basic and limited in several ways, it has improved my daily life a lot – and I write this on the first day of using it. I'm glad I went to the trouble. Now that I know how GNOME extensions work, I'm probably going to tweak my desktop even more.

I firmly believe that computers are there to support people and not the other way around. If my interaction with the computer is not satisfactory, it is the computer that has to change, not me. And the best thing about being a developer in an open source environment is that I can change it.

And so can you. What's your itch, and will you scratch it?

Footnotes

Experimentation has shown that the changes when dragging or resizing a window with the mouse are reported at a pretty fine granularity, even when I am moving the mouse as fast as I can. In practice, 50 pixels work very well as a threshold. ↩︎
For the window frame, x and y are the coordinates of the top left corner, so x+width and y+height are the coordinates of the bottom right corner. ↩︎
Use const more, for example, and apply the window type filter to the initial connection, too. ↩︎
But not trivial. input-emulator requires access to /dev/uinput, which is restriced to root by default. See the instructions coming with input-emulator for how to set this up. ↩︎

The Euro pallet

Sebastian Hans — Sun, 29 Dec 2024 00:00:00 +0000

My wife brought home a Euro pallet from work^[1]. She thought we could use it as an elevated platform for stuff in our cellar that shouldn't get wet in case our cellar got flooded. It was a good idea because there actually was a large number of flooded cellars in our neighbourhood this year. (Not ours, though.)

Anyway, we didn't immediately have something to stack on it, so we put in in the garage. Here it is.

The Euro pallet in my garage.

It has been leaning there ever since.

So what?

Today, while carrying some boxes into the cellar^[2], I was slightly annoyed by the pallet because it was partially blocking the passageway beside the car. This got me thinking about the pallet: maybe I should put it to use? Find space for it in the cellar and stack some boxes on it? Not today. I had other things to do, and the boxes I was carrying went onto the shelves anyway.

The pallet remains in the garage, and I will probably get annoyed at it a few more times. Will we ever put anything on it? Maybe, maybe not. Doing so would require us to reorganize the space in our cellar (and, yes, tidy it up a little). So it is likely that we won't get around to it for some time. Maybe never. Or at least not before something happens that causes us to throw it out, which amounts to the same thing.

But – and this is a big but – if we don't actually put something on it, the pallet does not have any value since its intended purpose was to protect our stuff from flooding. If our cellar were to be flooded now, the pallet would protect exactly … nothing.

Worse, it even has negative value because it gets in my way when I'm doing things like carrying boxes around. This was certainly not what we thought when my wife brought the pallet home. We thought it was a good idea. And it would have been if we had followed through. But we didn't. And right now I am busy writing a blog post when I could be in the cellar instead, realizing the value of the pallet.

On the other hand, I don't really know what specifically I should put on the pallet. So maybe a better action would be to throw it away. This at least would deliver value immediately by reducing my annoyance.

As it is, we put work into a good idea (bringing the pallet home) but failed to get to the point where we would have gotten value out of it (by having stuff protected), resulting in low-grade annoyance and extra work instead (having to lift boxes over the leaning pallet and taking special care lest it trip me up).

Software Euro pallets

Stuff like this happens in software, too. When I joined the team responsible for an internal multi-client capable payment API, the component had a database table that assigned clients to payment service providers. The intention was to simplify changes should they become necessary. Reassignment could be done at runtime by simply changing the rows in the table. In practice, all clients were assigned to the same payment service providers. There was only one exception, and it required extra implementation, so it was not a simple runtime reconfiguration.

Testing the code responsible for the client/provider assignment required us to implement fake payment service provider adapters because we didn't have any alternate payment service providers that could be reasonably substituted, even for test purposes. Implementation, test – and maintenance – of this feature was not zero-cost.

The first situation where we actually could have used it was the replacement of our payment service provider for SEPA direct debit^[3]. We could have implemented a new adapter for the new provider and switched our clients over one by one by changing entries in the database table. However, by this time, we had almost half a million customers using SEPA and wanted more control over the migration process. So we implemented a switch per user instead. And removed the client/provider assignment for good.

This feature was the software dev counterpart to the Euro pallet: it was a good idea, but we didn't get any real value out of it and still had to deal with the overhead it introduced, causing low-grade annoyance and extra work (testing, maintenance, additional conceptual work for every other feature to ensure it played well with this one). The best course of action was to remove it.

How about you?

Do you recognize this situation? Did you ever implement something that seemed a good idea but remained unused and thus generated zero or even negative value?

What would you have to do to get the value you originally thought you would? Are you actually going to do it?

Which features are in your software now that would increase the software's value if they were removed?

What is your personal Euro pallet?

Footnotes

It would have been thrown away otherwise, so she was allowed to ‘rescue’ it. ↩︎
You can see part of the door to the cellar in the background of the photo. ↩︎
I spoke about this on last year's WeAreDevelopers DevOps Day in my talk Migrating half a million users to a new payment service provider (free account required to watch). ↩︎

On planning, part 4: the critical path

Sebastian Hans — Tue, 12 Nov 2024 00:00:00 +0000

This is the fourth post in my series about planning in the context of agile teams.

Part 1 is concerned with why you should plan at all.
Part 2 describes my iterative approach to end-to-end planning.
Part 3 shows how I do forwards and backwards planning.
Part 5 argues for more exploration.

In this post, I will explore the concept of the critical path.

It doesn't fit!

So I have come up with a rough plan using forwards and backwards planning, and I have tried to map it onto a timeline. Surprise, surprise! It turns out that the work does not fit neatly between the fixed points of my plan. What now? Obviously, we can play the game of ‘does this here task really take two days, can't we get it done in one?’, but this is not really interesting, so let me jump straight to the point where we realize that we simply cannot make it fit by arguing about our estimates. Enter the critical path.

The critical path

The critical path is a project management method that helps you deal with time pressure by focusing on the tasks that are ‘most blocking’ project completion. After the first plausibility check, I already have estimates for the duration of each step, and I also know the dependencies between steps. This allows me to identify the longest sequence of steps between the start and the goal^[1]. This sequence is called the critical path because any delay here will directly affect our chances of reaching our goal before the deadline. Tasks that are not on the critical path have some ‘wiggle room’ because delays will not immediately threaten the deadline. After all, the tasks on the critical path will take longer anyway.

For the migration of the payment component to a new payment service provider (PSP) we did last year, the critical path identified by forwards and backwards planning looked roughly like this.

This is a simplified view of our migration steps between now and the time where our contract with the old payment service provider will end. Obviously, ‘let services test’ should come after ‘test’ because we need to have a stable (well-tested) implementation first, before we ask all services who depend on the payment component to begin their integration tests.

As you can see, our critical path is much longer than the time between now and our deadline, so we have a problem. Fortunately, there are some tactics that can help. I'm going to show three of them: moving tasks beyond the critical path, risk mitigation by kicking off external tasks early, and decoupling work.

Moving tasks beyond the critical path (the easy part)

Often, when there is a deadline, not everything needs to be finished before the deadline. In our PSP example, the deadline stems from the contract with the old PSP running out. By the time the contract ends, all payments must be routed to the new PSP, but the clean-up work can happen after the deadline. So we can move this part off the critical path simply by refining our understanding of which tasks absolutely need to be done before the deadline to ensure success and which tasks can be postponed if necessary.

We were able to move ‘remove implementation for old PSP’ beyond the deadline because it is not really critical.

This maneuver reduces the time pressure, but as you can see in the diagram, the project still does not fit into the available time frame.

Mitigate risk by kicking off external tasks early

If the project depends on external contributions, especially ones where we cannot be too sure of timely delivery, we can mitigate risk by ensuring that preparatory tasks are done early and the external tasks are not blocked. Then, the team can continue with work that does not depend on the external contribution. When the team is at the point where they need those results, chances are much higher that they will be ready.

In our example, there were two tasks which depend on 3^rd parties to collaborate with us. The first was providing access to the new PSP's test environment. When we first engaged with the PSP, they offered to conduct a few workshops to clarify the technical details and then give us access to their test environment. Recognizing that this would introduce an unwelcome delay before we could get our hands dirty^[2], we asked them to provide API access right away. They complied, speeding up onboarding significantly.

The second task not directly under our control was the testing the services built on top of the payment component would need to do. We worked hard to keep the change as low-profile as possible, but nonetheless there were some integration tests that had to be done and time had to be allocated for them. To minimize the risk of delay associated with this test phase, we started to engage with the service teams long before the tests were to take place. This did not shorten the test phase or allow us to bring it forward, but it ensured that the teams were ready to test when we needed them to.^[3] This engagement was done in parallel to the development work. Thus, even though it added work to the plan, it did not make the critical path longer.

Organizing access to the test environment early allowed us to bring the whole plan forward. Communicating the change and our timeline to the services ensured that their test activities would start on time, mitigating the risk of delays at this late stage.

Decoupling work

Sometimes, tasks depend on parts of other tasks. Example: During our PSP migration, we needed to let our API consumers test their systems against our new implementation. This integration test depended on our new implementation being available in our integration test environment.

Looking at the plan as visualized above, it seems as if we had to have our implementation finished before our API consumers could begin with their tests. These two tasks are serialized on the critical path.

However, in reality our service apps only had a dependency on certain parts of the new implementation, namely registration of new payment methods and payment authorization. Background processes including deferred captures and refunds were decoupled from the direct customer experience. By splitting our implementation task cleverly, we had the relevant parts in our integration test environment earlier, so the rest of the implementation could proceed in parallel with the integration tests, saving us precious time. In fact, the integration tests were off the critical path entirely. At no point did we have to stop progress while waiting for the test results.

By splitting the implementation and test tasks, we decoupled the integration tests of our services from implementation tasks that did not impact them, allowing implementation of those parts and integration tests to run in parallel.

Moving tasks beyond the critical path (the hard part)

In a similar vein, we can sometimes move tasks beyond the critical path by investing a litte extra effort to carve out and postpone uncritical parts.

In our PSP example, we would migrate users to the new PSP in batches, beginning some time before the deadline At first glance, it seems we needed to have the complete set of functionality (registration, authorization, deferred payment (capture), and refund) implemented for the new PSP by this point in time. But actually, what really mattered were registration and authorization because these were part of the checkout flow in our apps and thus a direct part of the user experience. Payment capture, on the other hand, was a deferred batch process. If the capture functionality had not been ready at the start of the migration, most users would not even have noticed because all that would have happened was that their bank accounts would have been debited a few days later. So we split our implementation into critical and uncritical^[4] tasks and prioritized them in such a way that we could have gone live with a partial implementation.

Refund was less critical still. Firstly, we didn't need to refund before we could capture. Secondly, refunds are rare when compared to captures, so far fewer users would be impacted if refunds were delayed.

Relaxing the requirement that capture and refund functionality must be finished before starting the migration finally made the plan seem plausible and even introduced some slack which would allow us to deal with unforeseen problems.

In the end, we did have capture and refund finished before we started the migration. Sometimes, not every risk you planned for actually materializes. But there were some minor things (nice-to-have features in the user interface and some hairy edge cases) that we did implement only after the migration had already started. So while the diagram above does not accurately depict our whole plan, it does show the kind of thinking that went into it.

Conclusion

Thinking about the critical path – the longest sequence of steps necessary to reach a goal – is a valuable exercise when dealing with deadlines because it highlights potential blockers and enables you to plan ahead and address them before they go ‘boom’.

Moving tasks beyond the critical path means realizing that not everything that needs to be done necessarily needs to be done before the deadline. In some cases, this may be obvious, in others less so. Sometimes, steps we initially think of as atomic are really composites of critical and less critical tasks, and splitting the latter out can make the difference between a doable project and a death march.^[5]

Mitigating risk by kicking off external tasks early entails identifying tasks that are not under our control but have the potential to derail our efforts when not done in time. Then we can arrange our plan so these tasks can start as early as possible and anything under our control that could possible block them is out of the way.

Decoupling work means thinking hard about the real dependencies between tasks and, if necessary, splitting them so other tasks are not blocked by – for them – unnecessary activities. If well done, this enables parallelization, which can speed up a project dramatically.^[6]

My next (and final) post of the series on planning addresses exploration of risks and alternative paths.

Footnotes

I'm going to use the start and the goal in the rest of my post for simplicity's sake, but a critical path can be identified between any two fixed points like intermediate milestones or multiple deadlines, and the following tactics will work just as well. ↩︎
There is no substitute for interacting directly with a new API. ↩︎
We also tested our changes rigorously ourselves before deploying them in the test environment accessible to the services. I mention this because, sadly, this does not seem to be the norm. Apparently, the service testers were used to broken releases and were thus understandably nervous about the short period of time we had given them to perform their tests. Our good preparation payed off, and we had positive test results in record time. ↩︎
Not that our background payment processing wasn't critical from a business perspective. It absolutely was. But it was not critical in the sense of the critical path. I.e., while we had to have it implemented before the project was done, and we certainly would have liked to have it finished in time for the migration start, this wasn't strictly necessary for the project to be successful. ↩︎
Note, however, that there is also a big difference between taking on technical debt to make a meaningful deadline and paying it off after the deadline has been successfully met on the one hand, and firefighting mode where we focus only on critical issues and never get to really finish anything because there is constant pressure with artificial deadlines on the other hand. The first is what I have in mind here. The latter leads to burn-out pretty quickly. ↩︎
Sometimes, splitting tasks takes extra effort, and we need to examine whether the benefits outweigh the costs. This is the case if (as in our example) the parallelization potential is large enough and if there are people available to take advantage of it. If nobody is actually taking on the tasks thus unblocked – congratulations! you have just made the project take longer. ↩︎

On planning, part 3: forwards and backwards planning

Sebastian Hans — Tue, 15 Oct 2024 00:00:00 +0000

This is the third post in my series about planning in the context of agile teams.

Part 1 is concerned with why you should plan at all.
Part 2 describes my iterative approach to end-to-end planning.
Part 4 explores the concept of the critical path.
Part 5 argues for more exploration.

In this post, I will explore what I call forwards and backwards planning.

As I wrote previously, before I can begin planning in earnest, I need to determine my current position and my goal (what ‘done’ looks like), and it is helpful to reflect on the means I have at my disposal to reach my goal. For the purposes of this blog post, I will assume I have done this groundwork already.

Planning forwards and backwards

I typically use this kind of planning for projects with deadlines because it helps me think about the dependencies between tasks and ensures that there are no gaps at the beginning and end of my plan. (See my previous post on why this is important.)

Planning forwards and backwards consists of three steps: planning forwards, planning backwards, and matching up both ends of the plan. Whether I start with planning forwards or planning backwards does not really matter, but matching comes last, obviously.

Planning forwards

Planning forwards, I think about what I can do right now that will bring me closer to my goal. I list possible steps to take immediately, then steps I can take afterwards, and so on. I write the steps down as a simple list, in the order in which they need to be done, taking care not to leave any gaps. Each step must depend only on steps appearing before it in the list. If there are any intermediate tasks^[1] to be done before the next step can be taken, these also go onto the list.

If steps can be done in parallel because they are independent of one another, I make a note of this to be considered later on, but I don't usually draw up a dependency graph or a Gantt chart or anything remotely complicated at this stage.

If progress depends on a 3rd party X to do or deliver something, this is also a step. The only difference is that I won't be the one doing it, and thus I will treat it as a black box and not iteratively refine it as I will do with other steps. However, I still need to think about the necessary preconditions for X to actually do their thing! Waiting for X while being unaware that X needs something from us before they can start is a failure mode I have seen time and time again. Because of this, whenever I add such a ‘3rd party step’ to my list, I try to put myself into their shoes and think about what I would need to get started in their place. If there is anything they might need from me (some artifact, information, or maybe just a trigger), I make sure to add it to my plan before the ‘3rd party step’.

If something needs to be done at a certain point in time, or cannot be done before a certain point in time, I note this when adding the step to my list. This is critical information because it constrains how I can move steps around later when trying to fit the plan to the timeline. All preceding steps must be finished before this step is due, and none of the following steps can start before this point in time.

Planning backwards

Planning backwards, I start with the desired end state and ask myself what needs to have happened so that we can say, ‘Yep, we are truly done now.’ This is the last step. And then I ask myself what needs to have happened so that we can take this last step. This becomes the next to last step. In this fashion I work backwards from the goal, always thinking about the preconditions that need to be true for the current step to be taken and the steps necessary to establish those preconditions, kind of like traveling backwards in time.

Often, there is more than one precondition or thing that has to be done first. This is OK. I write all of them down and pick one to continue the time travel, usually the one that seems most important to me. But I make sure to come back to the others later and deal with each of them. After all, they all are necessary to reach my goal.

After planning forwards and backwards for a while, possibly alternating between the two perspectives, each of the two plans will probably have developed several branches like this.

Planning forwards and backwards, two plans emerge, each with possibly multiple branches of linked action steps.

If the project is complex enough for the dependencies to be non-obvious, I might actually draw a diagram like this, otherwise, I'll make do with a linear list and keep the dependencies as notes (or sometimes in my head).

Matching up

At some point, the forwards plan and the backwards plan will begin to overlap, and it is time to match up the plans by connecting the branches of the forwards plan to the branches of the backwards plan. While doing this, several scenarios can appear:

A step appears in both the forwards plan and the backwards plan. This is the simplest scenario, where I can just merge them.
A step in my forwards plan is a direct prerequisite for a step in my backwards plan. Then I can connect the steps directly. I still need to check whether the step in my backwards plan has other prerequisites I need to connect.
A step in my forwards plan is a prerequisite for a step in my backwards plan, but I cannot go directly from here to there. In this case, I need to amend the plan with the necessary intermediate tasks. If they are not obvious, I can reenter forwards or backwards planning mode for a while until I have figured it out.
If a step in my backwards plan has prerequisites that are not fulfilled by my forwards plan, these can either be filled in (see previous point), or the forward plan is missing an important aspect entirely, in which case I will need to fill it in, either by planning forwards, or by continuing backwards planning until I eventually reach the area charted out by my forwards plan^[2].
If a step in my forwards plan seems to lead nowhere, i.e., it cannot be connected to my backwards plan, I need to ask myself whether it is really necessary to reach my goal. Maybe it is a side track I can eliminate or pursue elsewhere, or maybe I forgot something in my backwards plan, in which case I need to reexamine it critically.

The forwards plan and the backwards plan are now linked to form one plan. It contains all necessary steps and no unnecessary steps.

When I have matched up my forwards and backwards plans, I can be reasonably sure I have not forgotten anything important. At this point, I have a complete end to end plan, but I have not put it onto a timeline yet.

Timing it

For projects with deadlines, I put those down first, along with any other known time constraints such as the earliest possible start date or availability periods of critical resources. Then I try to distribute the steps of my plan around those ‘fixed points’ in a way that fits these and any other constraints (like ordering) I have identified during planning.

Once the initial timeline is complete, I do a plausibility check: is it realistic that we can do all this between now and the deadline? To do this, I need to estimate the duration of each task^[3]. If at all possible, I do this together with the people who are going to do the work.

And now?

Unless our schedule is very relaxed, the first naïve attempt at laying out my plan on the timeline will probably not look so good. If it does, all the better. If it doesn't fit, I continue by examining the critical path … critically. More about this in my next post.

Footnotes

I'm going to use ‘step’ and ‘task’ interchangeably. ↩︎
Which might be right at the start ↩︎
Note that estimating the duration is different from estimating effort. In practice, I have often found it easier to answer a question like, ‘Can we do this in a week?’, than a question like, ‘How many hours is this going to take?’ However, you have to take wait times into consideration. For example, if one of the steps is a review of a design document, you have to account for roundtrip times and the possibility of multiple rounds of review. ↩︎

It depends … too much

Sebastian Hans — Fri, 20 Sep 2024 00:00:00 +0000

tl;dr

This is a rant about indecisiveness.

Let's go!

We all know the software architect's (or engineer's) favorite answer is, ‘It depends.’ And there are good reasons for this. It really all depends – on so many things. When coming up with solutions to problems, there are many factors to consider, so any blanket statement, given without knowing the context, is likely to be wrong. Sometimes horribly so. Every software engineer (and architect) should periodically remind themselves that the key to good decisions is considering the context, dealing with conflicting forces, and carefully weighing benefits and risks. The best answer always depends. It is so. Really. And yet… I sometimes feel we have taken it too far.

While hammering this lesson in (and it is an important lesson!), we have somehow elevated ‘It depends’ to dogma, repeat it like a mantra, have come to the unconditional belief that everything always depends, and, crucially, that knowing the answer is impossible. Because how could it be possible? If we accept that there are unknowns – known unknowns and even unknown unknowns – and that ‘it depends’^[1], how can we be certain that any solution will be right? We can't. Not really. So we hedge. ‘It depends’, we cry, and somewhat smugly retreat to the position of ‘not knowing.’ This is a safe position because we can delay by asking for more context. Which is fine, I guess, if time is not an issue.

But … wait a moment! All of my architectural work is done in the context of a particular organization, most of it in the context of a particular team. I – and the collegues I work with – already know the context. An external consultant might get away with ‘it depends’ as an answer to an architecture question, but we won't. I won't let us. Here are some real conversations I have had with team mates in the last year:

‘It depends on how many API consumers we have.’ – ‘We have 5.’

‘We don't know if team A is using this function.’ – ‘They are. We had a meeting about it, and we can see it in the logs.’

‘But if team B wants to change this in the future!’ – ‘They didn't manage to change the last 3 things they wanted to change in their legacy system. They could't if they wanted to. So they won't.’

Yes, in general, it might depend, but in our particular case, this uncertainty does not exist because we already have the answers. Sometimes, it seems to me we have cultivated the attitude of ‘not knowing’ to a point where anyone purporting to have an answer is immediately suspect.

And another thing: dev teams don't have a monopoly on uncertainty. Everything is uncertain. But we usually don't qualify each and every answer we give. Here are some conversations I didn't have in the last year:

‘Will you be home for dinner?’ – ‘It depends. If I am run over by a bus on my way home, I might not be.’

‘Can we watch the show later?’ – ‘It depends. It might be difficult if the monitor blows up.’

‘Can you make it to the meeting?’ – ‘It depends. If the S-Bahn is punctual – no problem.’

OK, I'll grant you the last one. You get my point, however. At some point, we should be certain enough that we no longer need to qualify our statements. Yes, team B might still want – and manage – to change something, just as I might be run over by a bus. And if they do, we will have to change our plans, too. But this is true in any case. We have to work with what we know. This is not to say we shouldn't make an effort to find out things we don't know or to question our assumptions. We should. But at a certain point there are diminishing returns to further inquiry and questioning our knowledge. A point, at which ‘it depends’ stops being useful. A point where we need to decide.

Then we must do so, being fully aware that later on it may turn out we were wrong after all. But we cannot know that because … it depends.

Footnotes

It doesn't even matter what ‘it’ is because everything always depends. ↩︎

Engineering, scientifically

Sebastian Hans — Fri, 16 Aug 2024 00:00:00 +0000

This is an answer to Colin Breck's post Is the Scientific Method Valuable in Engineering?, so you might want to read this first. I liked this post very much because it set me thinking, even though I do not agree entirely with its conclusions. In it, Breck argues that the scientific method is not useful in software engineering because the goal of engineering is different from the goal of science.

In science, we form a hypothesis and design experiments to falsify that hypothesis. In engineering, we establish a process, and apply methods rooted in science, mathematics, and statistics, to ask questions of the system, run experiments, and improve the process.

He concludes that the scientific method is not applicable to software engineering because of those differences, and draws on examples from Bryan Cantrill's talk Things I Learned The Hard Way and on Dave Farley's book Modern Software Engineering to support his arguments.

I will use the same sources and offer a different lens through which to view the interplay between software engineering and the scientific method. Why interplay? Because I think the scientific method is a necessary, but insufficient ingredient to effective engineering. You cannot say, ‘I use the scientific method, therefore I am doing software engineering’, but you also – in my view – cannot claim to do software engineering without using the scientific method in places. You may, however, be using it implicitly much of the time, which can make it a little harder to see. It mainly shows up when someone is not doing it. If this sounds a little cryptic, bear with me; I will explain.

First, however, let's recap what the scientific method is. I will take the easy way and just use the same quote from Robert M. Pirsig's summary of the scientific method from Zen and the Art of Motorcycle Maintenance that Breck used in his post:

(1) statement of the problem, (2) hypotheses as to the cause of the problem, (3) experiments designed to test each hypothesis, (4) predicted results of the experiments, (5) observed results of the experiments, and (6) conclusions from the results of the experiments.

Note that this is a purely declarative definition. It presupposes the existence of a problem without caring where it came from. It assumes that there are hypotheses without saying anything about how they come to be. It demands experiments without giving guidance on how to design them. And so on. Breck's 3^rd footnote contains some quotes to this effect. His – and Bryan Cantrill's – conclusion seems to be that the scientific method requires you to guess (because each hypothesis is nothing other than a guess), and guessing is bad because there are other methods out there that better support goal-oriented problem solving.

So let's take the examples in his post and see if we can discover the scientific method in there!

Debugging, scientifically

The first example is Cantrill talking about his approach to debugging, and how he hates working with ‘hypothesis-centric engineers’:

I know this! It’s this! Oh, it wasn’t that, it’s this!

I can relate to that. Working with someone like that drives me crazy, too.

Cantrill suggests asking questions instead of guessing, and to make an effort to answer them by examining the state of the system. The answers to those questions allow you to narrow your bug search until you have found the bug. This is great advice, in my opinion, and a very good talk overall; I encourage you to watch the whole thing.

Anyway, if you do it his way, there's no guessing, no hypotheses, no scientific method, right?

Wrong! The way I see it, what he is doing is giving advice on how to do step 2 of the scientific method, coming up with a good hypothesis. Because at this point, it is still a hypothesis. He is going ‘I know it's this!’, but after a lot more analysis than the engineers he is complaining about, so the likelihood of his being right is much higher than a random guess. But there is no fix yet, so the problem is not solved, is it? What's next? Fixing the problem. Assuming it is a simple bug, identifying the bug roughly equals fixing it^[1]. At the end of this stands the hypothesis, ‘This was the bug, and what I just did fixes it.’

Step 3 of the scientific method says you should design experiments to test the hypothesis. In my book, this means you should have at least one test that fails before the fix and succeeds after the fix. For reproducible bugs, this should be easy, for others it can be hard, but the scientific method does not require that it be easy.

The prediction of the result (step 4) is included in each test case in the form of assertions. If the prediction turns out to be incorrect, the test fails. Running the test, and observing the result, is step 5.

Step 6: draw your conclusions. If the test succeeds, you will probably conclude that you were right and have fixed the bug. Breck mentions that in science, ‘a lot of effort often goes into explaining the errors in measurements.’ This is what you do if the test fails, too. You examine the test – maybe it contains an error? If not, you have invalidated your hypothesis. That is, you go, ‘Oh, it wasn't that!’ And you are back to step 2, coming up with a new hypothesis.

In conclusion, Cantrill's critique of using the scientific method for debugging seems to me to be really a critique of engineers who churn out ‘fixes’ without bothering to come up with a good hypothesis first.

I use the scientific method for debugging, and many good software engineers do, even if they do not think about it as such.

Improving performance, scientifically

The second example in Breck's post is performance optimization. Here, too, the critique seems to be mostly directed at people who jump to conclusions instead of following a systematic approach of coming up with well-grounded hypotheses, and my answer is the same.

Let's reduce it to bullet points this time:

Statement of the problem: Where exactly is the performance insufficient? If you skip this step, Donald Knuth might want a word with you^[2].
Hypotheses as to the cause of the problem: I agree full-heartedly that it is a bad idea to guess randomly. Use the profiler, measure where CPU time is spent, identify areas for improvement, come up with hypotheses of the form, ‘Unnecessary string copies in the flubber function cause 50% of the CPU load.’
Experiments designed to test each hypothesis: Implement the fix, and carefully design performance tests (this is a rabbit hole in and of itself).
Predicted results of the experiments: ‘The test will run twice as fast after the fix as before.’
Observed results of the experiments: Run the test. Uh, oh, the test shows CPU load was reduced only by 10%.
Conclusions from the results of the experiments: Examine the test (maybe we measured it wrong?), keep the fix (10% is better than nothing), put some more work in?

Again, I see the scientific method at work when I expand my view beyond the problem of coming up with a hypothesis.

Improving continuously, scientifically?

Breck uses manipulation of process variables in a chemical reactor as an example. I'm not sure about this one. If the goal is simply to observe the system behavior under different conditions, this sounds more like fiddling with the knobs and seeing what happens. No scientific method, there; I'll grant you this.

However, I think that as soon as you try to formulate a theory of how the system works and improve it on this basis, this sounds suspiciously like scientific territory again, and you might benefit from a more rigorous application of the scientific method. After all, is, ‘If we lower the inlet temperature by 1 degree, this will lower our energy costs by 2% and not affect the output quality’ not a testable hypothesis?

Changing software, scientifically

Colin Breck cites Dave Farley and comes to the conclusion that, while his book Modern Software Engineering promotes an ‘empirical, scientific approach’, it does not actually advocate for strict application of the scientific method. I have not read Dave Farley's book (sorry), but have heard him talk live and on YouTube and thus gotten a slightly different picture.

About the tests in a continuous delivery pipeline, Breck writes:

But these tests are not being used to disprove a hypothesis or discover a fundamental truth of the system, they are being used to test the invariants of the system and provide the guardrails for continued experimentation, discovery, refinement, and iteration.

And in a footnote:

It could be argued that the fundamental truth is evaluating if the software is always in a releasable state. But tests never perfectly replicate production environments and we need to continue to use empirical engineering techniques to evaluate software once deployed to production systems.

On the first point, I remember Dave Farley saying that each commit codifies the hypotheses that

the commit does what it is supposed to do, and
the commit does not break any part of the system.

And, yes, the tests are used to disprove these hypotheses. Regression tests try to disprove the hypothesis that we did not break anything (another way of saying the invariants of the system are still intact), and newly added tests try to disprove that the commit does what it is supposed to do.

The part about the tests not replicating production exactly is a digression. Other scientific experiments are not perfect either, see the observer effect. This is one reason why experiments/tests can only ever invalidate the hypothesis, not prove its correctness. ‘Tests are not perfect‘ does not imply the scientific method is not in action.

Breck describes the difference between science and engineering as follows:

Engineering is different in that we are not working to prove or disprove a hypothesis, but rather iteratively and empirically improve a process by applying proven methods.

I agree with this, and also with science and engineering having different goals. We can still apply the scientific method in engineering, where appropriate. I see it not as an either/or, but rather as the scientific method supporting the engineering processes.

Necessary, but not sufficient

The scientific method is not all there is to software engineering. We need to do a lot more to deliver good software, as Colin Breck, Bryan Cantrill, and Dave Farley have pointed out.

Some of this plays out within certain steps of the scientific method^[3], some of it is outside the scope of the scientific method^[4]. But there are places where the scientific method is applicable, and using it is indispensable.

Consider these examples of leaving out some steps of the scientific method:

Committing a change that does not address a stated problem?
Why are you committing the change at all, then? Where is its value?
Committing a change without having at least regression tests (experiments)?
You cannot tell whether the change was successful – very risky.
Not including assertions in your tests (= not having predicted the result of the experiment)?
Such tests are useless.
Not running your tests, or not looking at the results?
You may just as well not have tests.
Fixing a bug without testing the fix (includes steps 3-5)?
How do you know that you fixed the bug?
Having a test fail and not act on the result?
Why bother writing and running it then?

I do not see which part you can leave out and still claim to be doing software engineering.

Closing thoughts

The scientific method is more than guessing and testing randomly. It includes other aspects not mentioned in the simplified summary we used for our discussion, like inductive reasoning to come up with good hypotheses. But even if reduced to guessing and testing, it acts as an important safeguard against bad solutions and non-solutions. After all, even the engineers Cantrill complained about noticed that, ‘Oh, it wasn't that!’ Presumably, they had run experiments to determine this.

Going about debugging in this way is not efficient^[5], but it is certainly safer than doing it completely unscientifically. What do you think the result would have been, had those engineers not bothered to test their ‘fixes’?

At the same time, the scientific method alone is not enough to drive software development. We need those repeatable processes. We need to ask good questions, use architectural principles like coupling and cohesion, automate our testing, and work in small steps. But we need the scientific method, too.

And being strict about it is not necessarily slow. I can do the scientific method in a few minutes:

Select a small ticket from the backlog (contains the statement of the problem).
Write a little code and the first test. Since my test includes an assertion, this covers the hypothesis (the code), the experiment (the ‘given’ and ‘when‘ parts of the test), and the prediction of the result (the ‘then’ part of the test).
Run the test, observe the result.
On success, celebrate and commit (which triggers another round of experiments); on failure, go back to 2.

For me, the scientific method, translated into software development terms, means:

You should know what you want to achieve before you start changing anything!
Always write tests for your changes!
No tests without assertions!
Actually run the tests (new and old), look at the results, and act on them!

I, for one, would not want to have it any other way.

Footnotes

If it is not a simple bug, the hypothesis ‘It's this!’ may expand into multiple hypotheses on how to fix it (which makes the process more difficult, but does not change anything substantial, so let's ignore this rabbit hole). ↩︎
‘Premature optimization is the root of all evil.’ — Donald Knuth ↩︎
Like asking questions of the system for coming up with good hypotheses when debugging. ↩︎
Like establishing a repeatable process, and the sense of quality Breck mentions in his post – determining what is good enough. ↩︎
The scientific method does not require efficiency. ↩︎

On planning, part 2: iterative end-to-end-planning

Sebastian Hans — Wed, 31 Jul 2024 00:00:00 +0000

This is the second post in my series on planning.

Part 1 is concerned with why you should plan at all.
Part 3 shows how I do forwards and backwards planning.
Part 4 explores the concept of the critical path.
Part 5 argues for more exploration.

In this post, I show how I create an end-to-end plan using iterative refinement.

Ground work

The primary purpose of planning is to prepare me to reach a goal. For this, I need to know – or figure out – three things: where I am now, what the goal is, and what means to getting closer to the goal I have at my disposal.

Knowing my current position is essential because only then can I put my position in relation to the goal and determine the direction in which I will need to go. How do I get to Reykjavik? If I am in Keflavik, I will probably get a car and drive along the coast. If I am in Nuremberg, I will probably fly. How do I replace my payment service provider (PSP) with another one? The answer depends heavily on the structures and processes that are already in place.

Knowing my goal is important, because, well, if I don't know where I want to end up, there is no point in preparing to get there, is there? In practice, the goal is not always well-defined from the start. I may, for example, know that I want to visit Iceland, but not yet have a specific town or place in mind. Or I may know that I want to replace my PSP, but not yet have a detailed picture of the end state. This is fine. I can narrow down the goal during planning. But it still needs to be specific enough so I can tell when I have reached it.

Knowing the means at my disposal greatly aids in planning because it allows me to focus on possible paths forward as opposed to getting hung up on impossible ones. Means include actions I can take, time, money, work capacity, skills, material, and tools, all of which are usually limited in some way. I have observed that we (myself, but also colleagues) tend to gravitate towards our respective default methods when solving problems and forget that others also exist. As the saying goes, if you only have a hammer, everything looks like a nail. But even if the toolbox is full, if the hammer is lying on top because it is the most used tool, people will still squint very hard to make their problem look like a nail – rather than search through the toolbox. A software engineer, for example, may default to adding code for any problem, forgetting that sometimes organizational changes can work better. Explicitly considering what I have available before entering focused planning mode can open up alternative (and sometimes better) paths I would not have thought about otherwise. It can also prevent me from pursuing impossible plans once I discover what means are not available to me. It may, for example, be impossible to get additional resources for a project because of budget constraints.

Once I have a more or less complete understanding of my starting position, my goal, and available means of getting there, I can begin planning in earnest.

In the following I am going to use the examples from above. For the traveling example, let's say I want to travel from an address in Nuremberg, Germany, to a hotel in Reykjavik, Iceland, on a certain date. For the IT example, let's say I want to replace my current payment service provider A with the new PSP B, and this must be finished 6 months from now.

(Side note: I have never been to Iceland, so the details I am going to add to this example may not be factually correct, but they should serve to highlight the kind of thinking that I put into planning. I have, however, done a payment service provider migration. You can watch my talk about it at the WeAreDevelopers DevOps Day 2023 [free registration required].)

In the case of traveling to Iceland, for example, the means available do not include direct flights from Nuremberg to Iceland. They do include various indirect means of travel.

In the case of the PSP migration, the means available include a development team (partially occupied with other activities going on at the same time), our infrastructure, and the new PSP.

End-to-end planning

I start planning with a rough outline of the whole journey from my starting position until the project is done, i.e., I have reached my goal completely. Then I iterate on it by refining parts of the journey, validating assumptions, identifying risks, and examining alternative paths.

This is a core part of my planning strategy, and it bears repeating because I have so often seen people miss this.

I always plan the complete journey from start to end. There must be no gaps in the middle of the path. Steps may be roughly specified, i.e., details may be left out; steps may be unvalidated; their feasibility may be unclear; they may be fraught with risk; but they must not be missing.

There must be no gaps between steps in my plan.

As long as my plan contains gaps, it is not really executable because I cannot move from one step to the next directly. And worse, there is no way to know whether it is at all possible to make it executable. Small gaps may contain huge amounts of hidden complexity. Making the effort to close those gaps has the potential to expose issues that, if unaddressed, may sink my project. By exposing them, I make them addressable.

Traveling example:

I'm going to fly from Nuremberg to Keflavik airport via Frankfurt and drive from Keflavik to Reykjavik with a rented car.

Notice the gap in the middle? How am I going to get from Keflavik airport to the car rental? Forgetting this step may not be a problem if the car rental I end up using is directly at the airport, but it may turn out badly if there is some distance to cover. A more complete plan would be:

I'm going to fly from Nuremberg to Keflavik airport via Frankfurt, get to a car rental somehow, rent a car, and use it to drive from Keflavik to Reykjavik.

It is not yet clear which car rental or how I am going to get there, but it is clear that this step is necessary and will have to be dealt with at some point.

A first attempt at planning our PSP migration could look like this:

We are going to implement the API of the new PSP, migrate users from the old to the new PSP in batches, and disable the connection to the old PSP.

However, an important step is missing: before we can actually migrate users, we need to change the general terms and conditions to refer to the new payment service provider. Thinking of this early is particularly important because there are legal deadlines to consider which may have knock-on effects on the following steps. A more complete plan could look like this:

We are going to implement the API of the new PSP, update the general terms and conditions, migrate users from the old to the new PSP in batches, and disable the connection to the old PSP.

I always plan the complete journey from start to end. How to get startecd is a point that is often glossed over, but it is a critical part. If the journey does not start where I am now, if I cannot make the first step, there is not going to be a second one, not to mention the last one where I reach my goal. Getting started can take a lot of time and effort which, when not taken into account, can nix my whole plan.

Cannot start if the first step cannot be made from where I am.

Look again at my traveling plan and try to see what is missing at the beginning!

I'm going to fly from Nuremberg to Keflavik airport via Frankfurt, get to a car rental somehow, rent a car, and use it to drive from Keflavik to Reykjavik.

My journey begins at an address in Nuremberg, not at the airport. Getting to the airport is the first step. Going by subway, this step can take up to an hour. I will probably have luggage to check in, which will take time, too. So, an even more complete plan would look like this:

I'm going to walk to the subway, take the subway to Nuremberg airport, check in, fly from Nuremberg to Keflavik airport via Frankfurt, get to a car rental somehow, rent a car, and use it to drive from Keflavik to Reykjavik.

Now, for the PSP example:

We are going to implement the API of the new PSP, update the general terms and conditions, migrate users from the old to the new PSP in batches, and disable the connection to the old PSP.

What is missing at the beginning? Can we start implementing the API right away? No, we cannot. First, we need to clarify API usage for our use cases with our technical contacts at the new PSP, and we need access to the PSP's test environment. Only then can we begin to implement the API. Starting from where we are, the plan thus looks like this:

We are going clarify API usage with our technical support contact, and the new PSP is going to give us access the their test environment. Then we will test our scenarios manually, before implementing the API of the new PSP. We will update the general terms and conditions, migrate users from the old to the new PSP in batches, and disable the connection to the old PSP.

I always plan the complete journey from start to end. When is a project done? Over the years, I have seen various attempts to define done-ness, and claims that some project or other was done, and most of them excluded certain aspects – sometimes for expediency, sometimes because the person claiming done-ness had only limited responsibility, e.g., a developer who considered a task done as soon as the code was written. I think such narrow definitions of done do not make sense in the context of teams with true end-to-end responsibility.

For me, the answer is simple: a project (task, user story, whatever) is done when there is nothing left to do. Done is ‘done’ in the perfect tense, which means the doing has taken place in the past. If part of the doing is still in the future, it is not done. Is it tested? If not, it is not done. Is it deployed? If not, it is not done. Is it released to users? If not, it is not done. Is it documented? If not, it is not done. And so on.

I'm only done when I'm at the goal, not near the goal.

In my opinion, as soon as you start to distinguish between ‘done’ and ‘done done’ and maybe other kinds of ‘done’, you enter a zone of semantic diffusion where it is no longer possible to talk about when something is really finished. After all, when ‘done’ no longer means the doing lies in the past, what does ‘finished’ mean?

Consequently, when planning, I take care to not leave gaps at the end, either. It is easy to miss important bits that happen after the ‘main part’, leading to problems later on. Be careful not to fall into the Pareto trap, where the final (unplanned for) 1% take 99% of the time!

The key question I always ask is, ‘If we have done all this, have we reached our goal completely, or is there still something to do?’ If there is, I add it to the plan and repeat the question until the gap between the end of the plan and actual goal completion has been closed.

I'm going to walk to the subway, take the subway to Nuremberg airport, check in, fly from Nuremberg to Keflavik airport via Frankfurt, get to a car rental somehow, rent a car, and use it to drive from Keflavik to Reykjavik.

OK, but what about the hotel I actually want to get to? Can I drive there or is it in a pedestrian area? Does it have parking space? What about returning the rented car? Let's complete the plan:

I'm going to walk to the subway, take the subway to Nuremberg airport, check in, fly from Nuremberg to Keflavik airport via Frankfurt, get to a car rental somehow, rent a car, and use it to drive from Keflavik to the hotel in Reykjavik. I will park the car in the hotel garage and return it to the rental company's Reykjavik branch the next day.

Now for the PSP migration:

We are going clarify API usage with our technical support contact, and the new PSP is going to give us access the their test environment. Then we will test our scenarios manually, before implementing the API of the new PSP. We will update the general terms and conditions, migrate users from the old to the new PSP in batches, and disable the connection to the old PSP.

Once all users' payments are being processed by the new PSP, are we not done? From the point of view of the users, yes. But not from the point of view of the development team. Payment integration is not a one-shot fire-and-forget project; it is something we will continue to maintain and extend for a long time. The integration component must be maintainable to remain economical, which, among other things, means not keeping dead code around. So there is at least one additional step to do:

We are going clarify API usage with our technical support contact, and the new PSP is going to give us access the their test environment. Then we will test our scenarios manually, before implementing the API of the new PSP. We will update the general terms and conditions, migrate users from the old to the new PSP in batches, and disable the connection to the old PSP. Then, we will remove all code dealing with the old PSP.

Only then are we really done.

Now I have a rough, but complete plan. It covers the whole journey between our starting point and the ultimate goal. While it is probably not ready for execution, it will serve as the basis for refinement and further discussion.

The first rough draft of my plan will probably be wrong, but it has an important property: it is complete. I have found it much easier to do planning with colleagues once I have this scaffolding in place because it serves as an anchor in discussions about feasibility, details of a particular step, risks, and possible alternatives.

There is a meme on the internet, that goes something like this. ‘If I don't get an answer to my question on a forum, I simply post a wrong answer with a different account, and someone will immediately jump in to correct me.’

Relevant xkcd:

What do you want me to do? LEAVE? Then they'll keep being wrong!

The same goes for plans. If I need input from stakeholders who are reluctant to provide information or just not strongly motivated to take time to support the project, showing them a complete, but unvalidated plan more often than not prompts them to correct me.

Similarly, willing contributors who have not yet been deeply involved in the project are often missing some context. For them, too, it is easier to join in – and provide valuable input – if they can see how the issue under discussion fits into the big picture. When discussing a particular part of our journey, I often get questions like, ‘Yes, but what happens next?’ or, ‘What do we do with this result?’ Because the plan is already complete, I always have an answer. Sometimes, the answer is satisfactory. Sometimes, preceding or following steps will need to be adjusted as a result of the discussion. Sometimes, additional steps need to be introduced, or steps need to be reordered or merged. The impact of the step under discussion on the plan becomes clearer if we discuss it in the context of the complete plan rather than looking at each step in isolation.

In a way, this is like continuously integration-testing code changes. If I make an incompatible change to the code of a component in a project with good integration tests, a test will break and thus prompt me to restore compatibility (by either making my change compatible, or propagating my change to the dependent code). Likewise, if I change a part of my plan so it no longer fits with the rest, knowing how the part is linked to the whole enables me to make the necessary changes so the plan stays coherent.

It is OK for the plan to evolve unevenly, i.e., for some parts to be pretty clear while others are still hazy. How detailed each part needs to be depends on what I want to get out of my plan. If I do not have a hard deadline, it's probably enough to focus on the next couple of steps and keep the rest of the journey in the back of my mind to spot any potential problems. Using my example of dealing with a legacy system from the previous post, the plan could look like this.

The next step is known in detail; the way to circumvent the legacy system constraints is reasonably clear;
beyond that, it gets hazy (but still complete).

If I do have a hard deadline, I need to get to a granularity that allows me to roughly estimate each step's duration to ensure my plan fits plausibly into the remaining time period. One way to do this is to plan forwards and backwards (see my next post for more details).

If there is a dependency on some third party that needs to be integrated into my plan, I may need to go deeper on the affected parts of my plan than I otherwise would to ensure a smooth integration.

In any case, the important point is, as long as I have an end-to-end plan, I can iteratively refine each part of the plan as much as is useful, and having the end-to-end view enables me to spot changes that affect other parts of the plan and deal with them accordingly.

Conclusion

I find it important to do planning end to end, i.e., from my current position until I have reached my goal completely, and to do it iteratively because

the resulting plan is actionable immediately (begins where I am right now);
this approach enables effective collaboration;
it allows me to ‘integration test’ changes to the plan, ensuring consistency;
when I'm done, I'm done.

I use this approach for major endeavors, especially risky ones, projects including deadlines, and if there are dependencies to consider.

In my next post, I will write about how I use forwards and backwards planning to meet deadlines.

On planning, part 1: why?

Sebastian Hans — Sun, 30 Jun 2024 00:00:00 +0000

tl;dr

Planning is important even for agile teams.
Planning must be done by the people who are going to do the work, because the most valuable part of planning is the process.
If done right, planning helps you to avoid doing stupid things, prevents blockers, makes you faster, and enables better decisions.

Intro

I am a planner. Have been one since my childhood. I plan everything, from my meals to software development projects. I even plan the unplannable. I have children – the most unpredictable things on this planet – and still make plans. Sometimes they even work. 😃

In my first job as a professional software developer I worked in product development. The company sold market data distribution software to financial institutions. Development was financed in part through licensing fees, but mostly by customers paying us for extensions (new functionality and new interfaces). As these were all fixed price contracts, planning (and its ugly twin, estimation) was vital because any mistakes were going directly to our bottom line. We became really good at it as a result.

Nowadays, I work in agile teams, and there is a strong headwind against planning anything. I get where this is coming from, and I myself plan much less now than I did when calculating fixed price quotes 15 years ago. However, I still think planning is a key skill to be successful in the long term.

In recent years, I have noticed a certain bafflement among my fellow software developers – and not just them – when the need arose to make plans that extended more than two weeks or so into the future. I think this is a shame. So I decided to put some of my thoughts on planning in writing in the hope that someone might find them useful.

In this post, I will examine the question of why we should plan. In a follow-up post, I will write a bit about how I approach planning.

Why plan?

The Manifesto for Agile Software Development values

Responding to change over following a plan

and this is often taken as an argument against planning. It is not, for two reasons. The first is mentioned right in the next sentence (emphasis mine):

That is, while there is value in the items on the right, we value the items on the left more.

This sentence is just as much part of the agile manifesto as the proceeding ‘X over Y’ statements, but often ignored. Yes, there is value in following a plan – and thus, in having a plan –, even if responding to change is more important.

For the second reason, I turn to general Dwight Eisenhower, who gave us this insight:

Plans are useless but planning is indispensable.

While one should certainly be cautious about applying military doctrine to peaceful endeavors like software development (wait, why are you laughing?), it is certainly true that warfare is highly dynamic and much older as a discipline, and thus might have something to teach us about dealing with volatile situations.

So, why are plans useless?

Presumably because, as Helmuth Graf von Moltke has it,

No plan survives contact with the enemy.

Enemies will not adhere to your plan. To the contrary, enemies will try their very best to destroy your plan, and battle field commanders who try to stick to their plans in the face of intransigent enemies will probably lose.

As to the applicability of this adage in software development, have you ever had the opportunity to

… watch the first user interact with the UI you carefully designed?
… support a team who is trying to use an API you specified?
… undertook to finish a task today, only to be interrupted by a (real or imagined) emergency and making no progress whatsoever?

Case closed.

In effect, Eisenhower, von Moltke, and the agile manifesto are all saying roughly the same thing, which is that the value of having or following plans is limited, the difference being a matter of degree, not of principle. I assume this is due to the fact that enemies actively trying to foil your plans make them really useless, whereas the chances of success are significantly higher in software development, where it is usually ‘just’ life getting in the way. The value of following a plan as long as it remains practicable lies in increased efficiency. More on this below.

Why is planning indispensable, then?

After all, if planning is the activity that generates plans and plans are of limited value, how come planning is deemed to have a higher value than the plans it generates? The reason is simply this: the primary purpose of planning is not to generate plans.

Wait, what?

I repeat, the primary purpose of planning is not to generate plans. The primary purpose of planning is to prepare you to reach a goal. While planning will typically produce one or more plans, those are almost incidental. The real value of planning lies in the hard thinking that goes into it and in the shared understanding developed among the planners. To come up with a plan, the planners need to examine the current situation and clarify the goal. They need to explore options, assess risks, and devise countermeasures. They will come up with ideas only to find – on closer examination – that they are unworkable. In many cases, only one path from the current situation to the goal is written down as a plan. If everything goes well, it may work. If anything disrupts the path that was written down, it becomes worthless. But if planning has been done well – and the people who have done it are still available – the hard thinking that has gone into it will enable them to plot a new path from the new current situation to the goal and follow this instead.

This is why the real value lies in the planning activities and not in the final artifact called ‘plan’. This is why Eisenhower values planning so much, even though he deems plans useless.

Fine, you say, but this all sounds very generic. What does planning do for you that you don't get if you skip it?

Benefits of planning

First off, planning prevents you from doing really stupid things. This is a typical illustration of development progress for an agile team.

Figure 1: The team moves towards the goal in small steps.

Beginning at the bottom left, the team takes small steps in the rough direction of the goal at the top right. After each step, the team reflects on what they have accomplished and decides on the next step. The assumption is that the world is volatile and the goal may shift, so you need to readjust your direction regularly, which is why you probably will not move in a straight path but a kind of zigzag line. And if you subscribe to this line of thinking fully, there is no point in looking further than the next step because every effort you make to plan ahead will be wasted if the underlying assumptions change.

However, there is a problem with this picture. It looks as if the team were able to move in any direction it wanted. Even though it is operating in a volatile world which ostensibly influences the direction the team is going, the picture does not show this. It does not show any constraints at all. My experience is that some things are indeed volatile and may change at a moment's notice, but other things do not, even if you want them to. For example, the design and feature set of the web app you are building may evolve very quickly, but the legacy system you are relying on for half of the functionality will not – because it simply cannot. So the reality may look more like this.

Figure 2: Unbeknownst to the team, there are some serious constraints coming up.

The team is not completely free to choose its own way because the environment it is operating in imposes constraints, such as the (in-)flexibility of the legacy system. If you approach this situation without looking ahead and continue iterating towards your goal the same way as in the picture above, you will be in for a nasty surprise.

Figure 3: The team comes up against the constraints – aka the ‘Oh, shit!’ moment.

A little planning will allow you to spot such obstacles and plot a way around them. This does not mean you should stop taking small steps and re-examining your approach regularly. It merely means that you take into account not only the goal, which may be a long way off, but also the major obstacles.

Figure 4: Planning has brought up the constraints and the team has plotted a way around them.

As you can see in the picture, the team still adjusts its approach continuously, and the plan also changes accordingly. Even though the team has taken the same number of steps as above, it is much nearer to its goal when taking the obstacle into account.

Obstacles may also include critical resources your team depends on – and which may be unavailable or only available at certain times. Planning allows you to anticipate needs and arrange for them to be met when they materialize. What use is it to only consider what you are going to do for the next two weeks if your progress depends on another team's work, and this team has a lead time of a month? If you know beforehand you are going to need an API change from another team, you can coordinate with them to get it done in time. If you only know when the need has become immediate, you will be stuck. Mind you, planning does not guarantee that you will have what you need when you need it, but not planning guarantees that you will not have what you need when you need it. And planning pushes you to think about this dependency and may even surface some way to decouple your work from the other team's schedule so you will not be blocked even if they fail to deliver on time.

Planning enables better decisions. Why? Because good decisions take time. There is a concept in agile software development called ‘the last responsible moment’, which says you should delay commitment to a decision until the moment when not committing becomes more costly than committing. The reason is that deciding later allows you to make more informed – and thus better – decisions. For this to become true, however, learning needs to take place between now and later. Otherwise, the decision becomes just later, not better informed. And learning does not happen by itself. The team actively needs to create opportunities for learning along the way. And why should the team do this if it does not know that it will need to make the decision – if it has not planned for it? Planning in this case does not mean anticipating the result of the decision. It means anticipating the need for the decision, identifying important decision criteria, and ensuring that the team gathers enough relevant experience to make the decision well.

Planning allows you to move faster. This may sound counterintuitive if you think of planning as a large effort that happens before you get anything done. It doesn't need to be so. You can plan continuously as you go along. Your plan may turn out to be wrong, so advancing in small steps and continuously reassessing your position and the best way forward is still advisable, but the thinking you have done during planning will help you get this done very quickly.

In one extreme case, I have seen a team flounder after taking a successful step. How could they not know what to do next if everything went just as they wished? If they had thought a bit about the whole journey it would have been immediately clear what the next step should be. As it stood, they had to hold an hour long meeting to determine what they should do next.

Not planning is a bit like walking around with your eyes closed in broad daylight – or maybe with your eyes glued to your smartphone. You can do that, but you have to go slowly and feel your way forward lest you run into a wall and hurt yourself. You can go forth with much more confidence, less risk of hurting yourself – and faster – if you look ahead. You still have to slow down when approaching a corner you cannot see around, and if it is foggy you cannot see as far ahead as in bright sunshine, but it still helps.

Also, plans are rarely completely wrong or useless. Remember, the adage was coined for war – an extremely adversarial situation where the other party is actively trying to sabotage your plans. Software development is not that (at least in my experience). In general, the world is unpredictable, yes, but small adjustments to software projects are still more likely than drastic 180° turns. Some plans even work. Often, some details are off. Less often, major adjustments are necessary, and it is likely you will be able to anticipate some of them. And events which change everything are rare. Case in point: when the COVID-19 pandemic hit, it drastically changed my personal life; it affected how I worked to a large degree (I went from 100% on site to 100% remote in a week); it changed the goals of the project I was involved in at the time – not one bit. It also did not change the EU tender process we were going through, nor did it change our technology evaluation criteria. The details of our plans were adjusted, but the plans were not completely invalidated and helped – rather than hindered – progress.

When to plan

So should you plan everything all the time? Probably not. As I mentioned, I am a planner at heart, but even I admit you can overdo it. Very detailed plans, in particular, are pretty much useless, because details are very likely to change anyway. So when should you plan? I think planning is most useful in the following situations.

Feasibility check: You want to check whether a project is possible before committing time and resources to it. Example: your product owner has an idea for a major new feature, and it is not clear whether it is even possible to build. Planning prevents you from embarking on a fool's errand and helps with the definition of the first step if the project is deemed feasible.
Implementation start: You are about to embark on a new non-trivial project. Example: the above-mentioned feature has been added to your backlog as an epic, and you are ready to begin work on it. Planning allows you to spot potential problems and identify key decisions early on, and to provide for the necessary learning opportunities to make the decisions well-founded ones.
Direct deadline: A project absolutely must be finished at a certain point in time. Example: your contract with a service provider is expiring and you must have integrated their successor before this. Planning helps you to move faster and allows you to spot and circumvent obstacles which could jeopardize the deadline.
Indirect deadline: You are providing a service to a different team, and they have a direct deadline for which they need something from you. Example: a web shop needs to do a release for Black Friday, and they need a new feature in your API for it. Here, too, planning enables faster progress and helps you deal with obstacles proactively.
Dependency on scarce resources: Your team needs expertise or resources external to the team, which are hard to get on short notice. Example: you want to build an integration with a legacy system maintained by a third party and need integration support. Planning ahead enables you to anticipate this need and to coordinate with the support provider in good time.

These are scenarios where a little planning can make the difference between quick success and abysmal failure. In other situations, the effect may not be as noticeable, and an argument can be made that planning is not necessary.

Conclusion (and the single most important success factor)

To make this clear, I am not in favor of overly detailed planning and strict adherence to plans. This – shall we say – legacy practice goes hand in hand with rejecting – or even denying – change and results in software that is delivered late, if at all, and often fails to meet customer needs. However, the current predisposition against planning seems to me to be an overreaction to this kind of overplanning. The solution to bad planning is not no planning at all. It is to learn good planning, and to apply it in the right way.

To me, this means recognizing – like Eisenhower – that the value of planning lies not in the plans produced (which can become useless very quickly), but in the process itself and particularly in the learning that takes place during planning. This learning enables you to execute faster while avoiding stupid mistakes. It also enables you to manage dependencies and make better decisions by planning for learning along the way. And it enables you to be – yes! – more agile.

Given that the value of planning lies in the process, the most important factor is the people involved. If the plan is made by group A and the execution of the plan is assigned to group B, you lose all the learning and usually hand over only the plan – the least useful result of planning! To reap the benefits of planning, you therefore must involve the people who are going to be responsible for execution.

If you only remember two things from this post, please let it be these two:

Planning as an activity is valuable for agile teams, even if the plan is not.
But only if planning is done by the people who are going to do the work.

Thank you for reading, and happy planning!

Update: Originally, I thought I'd write two posts, one on the benefits of planning, and a second one on how I do it. Turns out there is more to it than will fit in a single follow-up post, so I turned the 'how' part into a series:

Continue with part 2 – iterative end-to-end-planning.

What is good software architecture?

Sebastian Hans — Mon, 27 May 2024 00:00:00 +0000

tl;dr

A good software architecture does not just solve a problem. It also

states the problem and its design goals (including any assumptions),
imposes only necessary constraints on the implementation (is not too fine-grained),
is visible and understandable, and
shows its work (decisions are traceable to design goals).

So, your software has an architecture.

This is a sequel to my first post, ‘What is software architecture?’, in which I defined a software architecture as a model, resulting from intentional, goal-oriented decisions, about how to solve a problem with software.

However, just because you have an architecture, this does not necessarily mean that it is a good one. What makes a software architecture good?

Isn't it enough that it is working?

If you implement a system based on your architecture, and it works, why bother with the question? First off, this approach requires you to build the software before you can evaluate your architecture. However, discovering architectural flaws after the implementation phase is typically expensive. It is much cheaper to eliminate flaws if they are found earlier in the software delivery lifecycle.

And then, there are the ‘-ilities’. Software architecture is often concerned with quality attribute requirements in areas such as security, maintainability, adaptability, and others (collectively referred to as ‘-ilities’), which deal with risk. Much of software architecture is risk management. If there is no risk you may miss your performance target, there is no need for the architecture to address performance. If there is no risk your system may be insecure, the architecture does not have to deal with security concerns. The crux of risks is that they may not materialize. Generally, this is a good thing. You don't want risks to materialize. But how can you tell then whether they have been addressed?

‘What's this?’ – ‘This is my charm against tigers.’ – ‘But there are no tigers around here!’ – ‘Yes! See how well it is working?’

How can you be sure that your anti-tiger charm is working (or that your architecture is maintainable) if there are no tigers in the country anyway (or if you have not yet had to perform maintenance on the finished system)? If you want your home to be secure, would you buy a front door lock that claims to prevent theft, but only as long as there are no thieves around? I wouldn't. I would want some assurance that the lock – or the charm – or the architecture – is actually capable of eliminating or at least mitigating the risks it is supposed to address.

So how do you do that? How do you provide this assurance?

What does ‘good’ mean?

As we have seen above, ‘good’ for an architecture does not just mean that the finished system is functional. And if you look at my definition again, you will notice that it says the architecture is a model anyway. The architecture is not the implementation. So, what does ‘good’ mean for this model? I propose four criteria:

It states the problem.
It imposes only necessary constraints.
It is visible and understandable.
It shows its work.

These are my ‘conditions for software architecture goodness’. In the following sections I will elaborate on them in turn and show why I think each of them is necessary.

A good software architecture states the problem.

By definition, a software architecture is supposed to describe how to solve a problem with software. To be considered a good software architecture, it needs to actually do this. While this may sound obvious, there is one small hurdle to clear: solving a problem requires the problem to be known. In practice, it can be surprisingly hard to get a clear and complete problem statement. While it may be easy to summarize the problem in a single sentence, this is rarely enough to give a complete picture of what is really needed.

‘I want a self-driving car.’ – ‘OK, we can do that. We'll put a little controller on an R/C car, and …’ – ‘Oh, and it must be able to drive my family to any address in the USA without getting involved in an accident.’ – ‘Ooof!’

The finished system must fulfill the needs of its stakeholders, and it must typically do so in a specific environment while being subject to a particular set of constraints. These can be formulated as a set of design goals which will drive the architecture. If it is not possible to get a definitive answer to a question relevant to the architecture (because the business requirements are not yet clear or depend on circumstances outside your control, e.g., future growth), you will have to make assumptions, which then become part of your design goals.

If the architecture of a software system does not state the problem it is set to solve (including the breakdown into design goals), the software implemented based on this architecture may still fulfill its stakeholders' needs, and it may still be able to operate more or less smoothly in the target environment, but it will only do so by accident. Since the whole point of the software architecture is to ensure that result, if it does not do so, it is a bad architecture, even though the system itself may be fit for purpose.

A further advantage of explicitly listing your design goals is that it becomes easier to see when things change. If, for example, the number of users to support is given as ‘up to 10,000’ and your business is so successful that the growth projection for the next year exceeds this number, it is a no-brainer to say, ‘Maybe we should review how our architecture is going to cope with this.’ If the number of users the architecture was designed for was not specified explicitly, you may only notice that you are operating outside of the system's specifications when the first problems appear in production.

Thus, a good software architecture contains a clear and complete problem statement as well as a set of design goals derived from the context and from the stakeholders' needs (including any assumptions you need to make). This is why the first chapter of the arc42 architecture documentation template contains an overview of the major business goals, functional requirements, and quality goals, as well as pointers to the complete documentation of each.

A good software architecture imposes only necessary constraints.

Once the problem is clearly stated, you need to make decisions about how to solve it. How many decisions? Paraphrasing Occam's razor, decisions are not to be multiplied without necessity.

Because every decision puts constraints on the solution space (and if it doesn't, it is unnecessary). This is a good thing because the whole point of software architecture is to constrain the solution space in such a way as to exclude bad implementations, i.e., implementations that fail to reach the design goals. But too many constraints are bad, too.

If there are two architectures that solve the same problem, but one of them puts more constraints on the solution than the other, the additional constraints are not necessary. Imposing them excludes valid implementations and thus limits the flexibility of the solution design. If (when) requirements change, but the context from which the design goals were derived does not, the smaller, coarser grained architecture will not need to change. The larger, finer grained architecture may need to be adapted, however, because the additional constraints may stand in the way of the necessary implementation changes.

Therefore, while there is no hard limit on how fine-grained a software architecture can be (at the limit, it could describe a single implementation exactly), I posit that a software architecture that imposes only constraints necessary to reach its design goals is strictly better than one that imposes additional constraints.

(And if you can think of a case where an additional constraint would make the architecture better, ‘because it would enable X’, then maybe you forgot to name X as a design goal.)

A good software architecture is visible and understandable.

A software architecture is a model of how to solve a problem with software. To actually solve the problem, the software system needs to be implemented according to the architecture, which means that the people implementing it need to know about the architecture. If there is only one developer, and it is the same person as the software architect, and the implementation is completed so quickly that there is no chance of forgetting something important, it may suffice if the architecture lives only in their head. In all other cases (more than one person working on the implementation, or one person working on it over a longer period of time), if the architecture is to be preserved over the course of time, it must be made visible and must be represented in a way that will be understandable to developers (either the architects themselves or others) in the future.

Sometimes, software architecture is described as a ‘shared understanding’ of the system structures by the relevant stakeholders (for references, see my previous post). Building a shared understanding is much easier if there is something visible to point to when doing the sharing.

Making a software architecture visible can be done in various ways. The obvious one is having separate architecture documentation. For guidelines on what to document and how to do it, I found the arc42 template useful.

The internal architecture of an application can be made visible in the code by making judicious use of your programming language's modularization features and appropriate naming of modules and other elements (packages, classes, functions, …). For example, in the Java world, if you want to decouple an application from the implementation of a certain piece of logic, you can put the interface definitions and implementation classes into separate JAR files and declare only the interface JAR as a compile-time dependency. The implementation JAR is added as a runtime dependency only, which prevents the application from accessing the implementation classes directly. An example of this is SLF4J, which provides an interface to logging, with several implementations backed by different logging libraries.

There are also ways to test conformance to an architecture. For Java-based software, one option is ArchUnit, which allows you to write unit tests that check dependencies between packages and classes. Some static code analysis tools offer similar features. Tools like these make the architecture visible by raising flags on architecture violations.

While none of these techniques can guarantee that the architecture will be followed when changes are made in the future, projects using them have much better chances of keeping their architecture alive.

A good software architecture shows its work.

The paper What is your definition of software architecture? cites Barry Boehm and his students at the USC Center for Software Engineering like this:

A software system architecture comprises

[…]

A rationale which demonstrates that the components, connections, and constraints define a system that, if implemented, would satisfy the collection of system stakeholders’ need statements.

While I think a software architecture that is missing this rationale is still a software architecture, i.e., the rationale is not the defining aspect, I agree that a good software architecture does show its work in this way.

I define a software architecture as a model, resulting from intentional, goal-oriented decisions. This means each decision is made to support one or more design goals, and this in turn means someone has hopefully thought about how the decision does this and why the option chosen does this better than other options that were also considered. Documenting these thoughts (possibly in Architecture Decision Records) makes the architecture better understandable for everyone who was not there when it was created (or cannot remember everything perfectly). And it makes it easier to work with when something changes. If a change is made to the business requirements that has an impact on the system's design goals, you can examine the rationale to determine which decisions may need to be revisited. If a change to the architecture is proposed, you can look up from which design goals the current state was derived, and determine the impact of the change on these goals.

Documenting these links is especially important for design goals that are not directly observable because they do not concern behavior but deal with risk reduction (like many of the ‘-ilities’ do). Sometimes, the rationale connecting a design goal with architectural derisking measures is the only visible artefact that shows the design goal has been addressed.

Conclusion

A software architecture that results in a working system when implemented is … not bad, but to be really good, it should also …

… state the problem it is solving and break it down into design goals.
… refrain from imposing constraints on the implementation that are not necessary to reach its design goals.
… be visible and presented in ways understandable by fellow architects and developers (which may include the future you).
… provide a rationale that links design goals and architecture decisions and explains how the latter result in an architecture that meets the former.

If your software architecture does these things, it will be easier to implement and easier to work with in the long term. And, of course, it should also work.

What is software architecture?

Sebastian Hans — Tue, 30 Apr 2024 00:00:00 +0000

tl;dr

A software architecture is a model, resulting from intentional, goal-oriented decisions, about how to solve a problem with software.

The discipline of Software Architecture comprises methods to identify, make, and communicate the necessary decisions and the resulting model.

A Software Architect is someone who applies these methods to a specific problem to reach a concrete set of goals.

Why?

Why yet another definition of software architecture? Don't we have enough definitions already? No, of course not. I am just following standard procedure here. 😉

On a more serious note, I think each new attempt to define software architecture brings a new perspective to what I see as an evolving field. Maybe some day we will have a common definition of software architecture everyone can agree on similar to other disciplines. But how will we get there if not by iteratively refining our understanding, discussing various aspects, coming up with new descriptions, and trying them on to see how they fit?

It is in this spirit that I present my attempt at a definition, not to replace others, but to add something I have been thinking about a lot, in the hope that someone may find it useful. I am happy to enter into a discussion on Mastodon.

What is software architecture?

I humbly propose the following definition of software architecture. (This definition applies to systems – which may contain non-software parts, such as hardware and people – just as well. I will be using the terms software architecture, systems architecture, and architecture more or less interchangeably.)

A software architecture is a model, resulting from intentional, goal-oriented decisions, about how to solve a problem with software.

This definition consists of several parts, each of which addresses an important aspect.

A software architecture is a model.
A software architecture is intentional.
A software architecture is goal-oriented.
A software architecture describes how to solve a problem.

I will discuss each of them briefly in the following sections, in reverse order.

Building on this definition of a software architecture, the discipline of Software Architecture comprises methods to identify, make, and communicate the necessary decisions and the resulting model, and a Software Architect is someone who applies these methods to a specific problem to reach a concrete set of goals.

A software architecture solves a problem.

Just as the architecture of a building is aimed at a specific purpose – the architect should know if the building is supposed to contain a factory or a sports arena or house a family – the architecture of a piece of software is intended to solve a specific problem. If there is no problem to solve, there is no need for an architecture (and no need for the software).

This is in contrast to architecture styles and architecture patterns, which describe generic approaches or solutions to classes of problems. These are useful because they codify architecture experience and provide guidance to architects, but to become an architecture they have to be applied to a concrete problem.

While solving a problem, there are many different aspects to consider, each of which may have an impact on the final architecture. To be able to address them effectively, they need to be broken down into design goals, which brings us to the next point.

A software architecture is goal-oriented.

Every software system must operate in a certain environment, is subject to a unique set of constraints, and must satisfy its stakeholders' needs, often expressed in the form of functional requirements and quality attribute requirements. The goal of a software system's architecture is to ensure that the implementation will be fit for purpose in the sense that it can operate in the target environment, that it does conform to those constraints, and that it can fulfill the requirements. Thus, environment, constraints, and requirements form a set of design goals that inform the architecture, which in turn means that the resulting architecture is inextricably linked to these design goals. If the goals are chosen differently, the architecture must be designed differently. If the goals change, chances are good that the architecture will also need to change. This is what I mean when I say designing a software architecture is an inherently goal-oriented process.

As a consequence, if you do not have any design goals, you do not have a basis to build an architecture on. Architecture cannot ‘happen’ by chance (also see my notes on intentionality below).

But what if there are no goals?

Sometimes, specific design goals are hard to come by, either because our stakeholders cannot be bothered to put them in concrete terms or because the product we are trying to build is still in an early phase and we only have a vague vision, but few hard requirements. This makes designing an appropriate architecture hard because the whole point of architecture is to meet design goals. In this case, you will have to work with assumptions and formulate goals from them. Only then will you be able to design an architecture that will meet those goals.

What if the goals are wrong?

If you base your design goals on assumptions, you run the risk of being wrong about them. Of course, this risk is always there, but the more assumptions you need to make, the greater the risk becomes.

If the goals are wrong, i.e., they do not represent the actual needs of the stakeholders, you will get an architecture that looks good on paper (because it ostensibly reaches its goals), but will perform badly in practice because it fails to address the real needs.

What if the goals are lost?

While you must have design goals for your architecture work, sometimes these design goals are not written down explicitly and live only in the heads of the architects. The resulting model will still be an architecture, possibly even a good one, but it will be hard to evaluate because no one who was not there during the architecture design can determine how well the architecture meets its unknown goals.

It is also impossible to tell whether there really is an architecture or what you are seeing is just structures that accreted over time.

A software architecture is intentional.

Problems can be solved in many ways, including throwing solutions at them and seeing what sticks, or going from one tiny subproblem to the next and fiddling around until it works. I have seen software systems built this way. I know it can work. I don't think this is architecture, however.

Architecture results from deliberate decisions.

To me, architecture implies a deliberate decision-making and modelling process. There are those who say, every system has an architecture. I disagree. If you have not deliberately formed a model of how to solve a problem, your solution does not have an architecture. It may have a structure (it must have, to be executed by a computer), but it does not have an architecture.

A tree trunk that has been blown over by the wind and fallen over a river may serve as a bridge, but since there is no underlying model, it does not have an architecture.

Figure 1: This tree trunk fallen across the river has no architecture.
Image credit: J. M. CC BY-SA 2.0

The same tree trunk, put there deliberately as a solution to river-crossing, does have an architecture, albeit a simple one.

Figure 2: This architecture diagram of the above tree trunk shows how it is supposed to span the river.

Architecture lies in the heads of the designers.

You could also say, a system does not ‘have’ an architecture at all. Because the architecture is not a property of the system, but of the system's designers. It is they who form the model of how to solve the problem. Given just the tree trunk mentioned above, there is no simple way to tell whether there exists an underlying architecture. The same is true for software systems, but there it is much worse due to their greater complexity.

Architecture comes before implementation.

If software architecture is intentional (and geared towards reaching certain goals), it must necessarily be done before the software is written. Writing software and then claiming, ‘This is the architecture!’, is like shooting at a blank wall and drawing the target where you hit it instead of first drawing the target and then taking a shot. The result may look similar to someone who has not witnessed the process, but the difference could not be greater.

This also means that an architecture cannot be reliably reverse-engineered. By performing software archaeology, the attempt to discover the underlying structures and design of an existing piece of software, you may indeed discover structures, but without supporting documentation, you will not be able to tell whether these structures were intentionally created to conform to an architecture or whether they have been formed by a different process. Or, in other words, looking at the wall with the hole in it, but no target painted on, how can you tell where the shooter wanted to hit? And how can you even tell whether the shooter took aim at all or wether it was just a random shot?

A software architecture is a model.

As a model, the software architecture is an abstraction, not the software itself. This has a couple of implications.

A model is not the thing itself.

As with the architecture of buildings, a software architecture consists of plans, typically including diagrams that show important components and how they relate to each other. It does not consist of the components themselves. The architecture is separate from the structures (buildings, software) implementing it and typically exists before the structures are built.

This is not an architecture.

Figure 3: A building itself is not its architecture. It is just a building.
Image credit: Leon CC BY 2.0

This is.

Figure 4: A (building) architecture is the model of a building.
Image credit: gottm2 CC BY 2.5

If a visitor to this building exclaims, ‘What a great architecture!’, this does not mean that the building itself is the architecture, but that the architecture is immediately apparent to the casual viewer because it has been followed strictly and the builders have refrained from adding elements which would obscure it. However, if they had (like Christo and Jeanne-Claude did when they wrapped the German Reichstag building (website in German)), the architecture would still be there.

A model leaves things out.

As an abstraction, the architecture must necessarily leave many things out. So, what should be part of the model and what should be left out? Since the point of the architecture is to solve a problem, it should only include things relevant to the solution of this problem. This may seem trivial at first glance, but when you consider that the problem statement itself may be unclear, you come to realize that this is not as simple as it sounds.

On the other hand, the model should address all aspects that are important for the solution of the problem. However, since designing and maintaining a software architecture is a deliberate process (as discussed above), it comes at a cost (in terms of effort and time). If we had infinite resources and time, we could make our architecture model so detailed that it would effectively describe an implementation. In practice, we are always working with limited resources and time, so we should constrain our architecture model to those aspects with the largest impact on the shape of the solution and its fit to the problem and goals we are trying to achieve.

Moreover, if we plan to continue developing our solution, we need to consider the impact of ongoing development on the software architecture. If our architecture is too detailed, it will need to be adapted very often as our solution evolves, leading to unnecessarily high costs of change. If our architeture is too coarse, it may not be able to ensure that the solution continues to meet its design goals as changes are made. These two concerns need to be carefully balanced to arrive at a model which contains just enough guard rails to keep development on track with respect to the design goals, but at the same time enables (instead of inhibiting) continuous change.

An undocumeted model is still a model.

A software architecture can exist in the head of a person and still be an architecture. In my view, it does not have to be written down to ‘count’ as an architecture. However, it can be hard to distinguish an architecture that lives only in someone's head from no architecture at all; so, in a team context, documentation is still important.

Decisions shape a model, but they are not the model.

Even though a software architecture is the result of a series of decisions, the decisions alone do not an architecture make. In other words, the architecture does not consist of a series of decisions, but is the model that emerges when all those decisions are applied on top of each other and any conflicts that may arise are resolved.

It is perfectly possible to write down (or read about) any number of decisions and not understand what the result looks like. As long as you do not have this understanding – the model that is the outcome of all these decisions – I argue that you do not have an architecture.

Incidentally, this is why I think keeping only Architecture Decision Records is not enough. You also need an evolving view of the result of those decisions.

How does this relate to other definitions of software architecture?

In this section I want to compare my definition of software architecture to several more or less well-known definitions and examine the differences and similarities.

ANSI/IEEE Std 1471-2000

Software Architecture: the fundamental organization of a system embodied in its components, their relationships to each other and to the environment and the principles guiding its design and evolution.

This is a classic definition of software architecture. Many similar definitions refer to components and their relationships, and this one includes a reference to more general ‘principles’.

In my definition, the term ‘model’ subsumes these three elements, and I certainly expect the model to include them all. Instead of further detailing the content of the model, I chose to focus on its intentionality and the purpose of the architecture. This part is completely missing from the ANSI definition. While the word ‘principles’ provides a clue that an architecture must be designed intentionally (since principles, unlike structures, do not simply appear unintentionally), the definition refers to a ‘system’ without saying anything about its purpose. According to this definition, a software architecture does not need to have any goals. However (as I wrote above), to me, if you do not have any goals, architecture becomes meaningless.

Also, I find it important to note that a software architecture is the result of a decision process. If a software architecture is a model arrived at after N decisions, this leaves the possibility open to add another decision, resulting in a different (evolved) architecture, whereas the wording ‘guiding its design and evolution’ in the ANSI definition seems to imply that while the system may evolve (within the frame of its architecture), the architecture itself remains static.

ISO/IEC/IEEE 42010:2011

Architecture: (system) fundamental concepts or properties of a system in its environment embodied in its elements, relationships, and in the principles of its design and evolution.

This one is similar to the ANSI definition, and the changes in the wording do not change my assessment.

Wikipedia

From the article on Software architecture on Wikipedia (accessed on April 19th, 2024):

Software architecture is the set of structures needed to reason about a software system and the discipline of creating such structures and systems. Each structure comprises software elements, relations among them, and properties of both elements and relations.

This definition is completely focused on the structural part of software architecture and ignores intentionality and goal-orientation completely. It allows for purely descriptive (e.g., reverse-engineered) ‘architectures’. To reuse my non-software examples from above, this definition would classify descriptions of both the tree trunk fallen over the river and the hole shot randomly into a wall as architectures if they help you ‘reason about’ the system, irrespective of how they came to be.

To my mind, a set of structures, even if it is helpful, is just a set of structures. To count as an architecture, it needs to be intentionally created to meet a set of design goals.

Ralph Johnson (quoted via Martin Fowler)

In his essay Who Needs an Architect?, Martin Fowler quotes Ralph Johnson like this:

In most successful software projects, the expert developers working on that project have a shared understanding of the system design. This shared understanding is called ‘architecture.’ This understanding includes how the system is divided into components and how the components interact through interfaces. These components are usually composed of smaller components, but the architecture only includes the components and interfaces that are understood by all the developers.

This definition is similar to mine in that it places the architecture in the heads of the designers. If the architecture is the shared understanding of the system design of the developers, it depends more on the people than on the system itself. However, it differs in how it views intentionality. My definition requires the architecture to result from intentional decisions, whereas a shared understanding can be developed after the fact, for example, by a team which has inherited an undocumented system and is building a shared understanding of its structure.

While I agree that this shared understanding is important, in my view it is not sufficient to constitute an architecture. However, it can be transformed into one. If the team discovers a certain structure in the software it has inherited and then intentionally decides to adopt this structure as the basis for further development in order to reach some goal (they are bound to have some goals; why would they bother to deal with this system otherwise?), then it will become part of the architecture.

If the team discovers another structure in the same piece of software and decides to ignore it because it does not help them, it will not become part of the architecture, even though the structure exists (and may continue to exist) and they have developed a shared understanding of it.

Further on in the quote, it says:

architecture is the decisions that you wish you could get right early in a project

Here we have the decisions, which covers the intentionality, and the word ‘right’ implies some sort of goal; if there was no goal, there would be no way to tell if a decision was right. What is missing is that the decisions need to come together in a coherent model to be actually useful.

The quote ends with:

Architecture is about the important stuff. Whatever that is.

While this was probably not meant to constitute a definition of architecture all by itself (it is way too broad for that; is it important to keep your developers fed?) it begs an interesting question: what is important? In my view, this depends on your design goals, which in turn depend on the problem your architecture is supposed to solve. This is why my definition includes both the problem to solve and goal-oriented decisions.

Software Engineering Institute, Carnegie Mellon University

From Software Architecture (accessed on April 19th, 2024):

The software architecture of a system represents the design decisions related to overall system structure and behavior.

This definition seems to imply that a software architecture emerges from a series of decisions, the only requirement being that those decisions relate to the system structure and behavior. While I agree in principle, I think this definition is too lax. As a thought experiment, imagine a series of random ‘design decisions related to overall system structure and behavior’. These decisions will certainly result in something, but would you call it a software architecture? I would not. I think that in order to get to a software architecture, the design decisions need to be oriented towards a common set of goals. Only then will you get a coherent model that can be called an architecture.

Also, the definition seems to equate software architecture with the series of decisions, a kind of event sourcing approach to software architecture. I don't know about you, but I would not be able to actually work with an architecture description consisting only of decision records. I would need to go through them and form a model of the resulting architecture before I could actually do anything with it. So, while it may be possible to reconstruct the architecture given just a series of decision records, this is just the point. The decisions are not the architecture. The resulting model is. In my definition, this distinction is made explicit.

Barry Boehm et al. (1995, via What is your definition of software architecture?)

Barry Boehm and his students at the USC Center for Software Engineering write that:

A software system architecture comprises

A collection of software and system components, connections, and constraints.

A collection of system stakeholders’ need statements.

A rationale which demonstrates that the components, connections, and constraints define a system that, if implemented, would satisfy the collection of system stakeholders’ need statements.

I really like that this definition includes the needs of the system's stakeholders and the connection between the system's structural properties and these needs explicitly. In contrast, my definition does not mention the stakeholders explicitly, but it requires the architecture to solve a problem and aligns architectural work (in the form of decisions) to (design) goals.

If the problem an architecture solves is a problem the system's stakeholders actually want to have solved and the design goals align with the stakeholders' needs, I would regard the architecture as good. However, if an architecture solves a problem no one has or its design goals do not align with anyone's needs, I would still regard it as an architecture, albeit a useless one. Strictly speaking, the above definition would not really count the latter example as an architecture because it requires it to address actual stakeholder needs.

On the other hand, even more strictly speaking, it only requires it to address ‘need statements’, which is not the same. The stated needs may not be the real needs, i.e., the statements may be wrong. An architecture based on wrong need statements would still fulfil the definition, so the latter example might actually fulfil the definition after all if the problem (which no one needs solved) and design goals (which do not align with anyone's needs) are written down as stakeholders' need statements, even if they are wrong. Which, for practical purposes, means there is not much of a difference between the USC group's ‘stakeholders' need statements’ and my ‘problem’ and ‘goals’.

This definition also requires there to be a model (not just decisions) and addresses goal-orientation by requiring a rationale that connects the model with the needs/goals. The only part of my definition the USC group's definition does not address at all is the deliberate decision process. As far as this definition is concerned, the architecture could appear out of thin air as long as the rationale can be constructed afterwards.

GitHub

From What is software architecture? (accessed on April 19th, 2024):

Software architecture refers to a set of fundamental structures that developers use as an overarching visual guide when planning for and building out software solutions. These frameworks also ensure that a project meets its technical and business needs by planning for important functionality such as scalability, reusability, and security. These plans are then broken down into individual components and code during the design phase.

This definition includes some elements that do not appear in any of the other definitions.

It seems to require a visual representation of the structures. While visual representations can certainly aid understanding, I would not claim non-visual structural representations do not constitute software architecture.
It mentions some important quality attributes, but does so by way of example and not as a mandatory list, so the selection does not affect the overall meaning.
It draws a boundary between software architecture and software design. However, it is unclear to me whether this is supposed to be part of the definition or not. I wrote a bit about the granularity of software architecture above. In my opinion, where software architecture ends and lower-level design activities begin depends on the forces pushing the boundary up (avoiding architecture overhead) and down (ensuring design goals are met).

What about the other elements? The definition limits the scope of software architecture to ‘fundamental structures’, which seems too narrow to me. My idea of the architecture model is more general and includes concepts and principles as mentioned in some other definitions.

The GitHub definition explicitly calls out the use of software architecture for ‘planning and building software solutions’, which aligns with my idea of an intentional and prescriptive (rather than purely descriptive) approach.

It also mentions ‘technical and business needs’, which hints at the problem-solving and goal-orientation aspects, but leaves out the decision-making process.

Conclusion

As the comparison of my definition of software architecture with several other definitions shows, most of them cover one or more of the important aspects of software architecture I discussed in my article, but none of them covers the whole set in quite the same way.

The characterization as a model as opposed to a set of decisions is important because the interplay between the decisions can be non-trivial and you have to know (and understand) the result in order to work with it.

Intentionality is important because an architecture is something you create deliberately.

Designing a software architecture is a goal-orientated process because the whole point of an architecture is to ensure the software meets certain design goals.

An architecture is a means to an end, and the end is to solve a problem. This should always be kept in mind, lest the architecture become an end in itself.

This is why, to me, a software architecture is a model, resulting from intentional, goal-oriented decisions, about how to solve a problem with software.

I hope someone finds these thoughts useful.