Endless possibilities (a socio-technical API pattern) June 26, 2025

Thinking about socio-technical API patterns some more, another one occurred to me. I'm calling it Endless Possibilities.

This one plays out over a long(ish) time frame. It starts out as a fairly simple API designed by some producer to satisfy the needs of a single consumer, or perhaps a small group of consumers. Let's assume the API is well-designed, maybe even co-designed between producer and consumer teams. It covers a small number of use cases and is well-tested and well-documented.

Then life happens. A new consumer is onboarded. Their needs align well with the established API, except for one small variation. The producer team addresses this by adding an optional property to one of the requests, with a backwards compatible default behaviour.

When another consumer is onboarded, their special needs are addressed in a similar manner. Maybe it is not a new property this time, but an additional enum value on an existing property. This new option conflicts with the first optional property, but this is not a problem because each consumer will only be using one of them. The API will return a helpful error message if a consumer tries to use both of them together.

Later, one of the existing consumers wants to implement a feature that requires a slightly different workflow, and the producer team needs to decide how to accommodate this: add another specialized option to the existing API methods? Special-case the API for the consumer? Implement new API methods instead? Or make the API more generic so it can handle both the old and the new workflow, distinguishing them by some other criteria? The team decides to take the latter route because this will enable not only the new workflow, but open up some additional possibilities the team feels will be interesting for some consumers. They add some documentation on how to choose between the old and the new workflow.

The lead developer of the provider team leaves the company (for unrelated reasons). Unfortunately, no one else on the team possesses quite the same level of domain expertise and historical knowledge of the technical intricacies. When the next feature is requested by a consumer, the provider team comes up with a solution that involves adding feature flags to 3 different API methods – to be used in a certain combination. One of the flags conflicts with an option added earlier – but only in certain edge cases. No one on the team knows with any amount of certainty if these are plausible scenarios. It may even be unknown whether the older option is still in use by some consumers. The team updates the documentation to indicate which scenarios are supported, but it remains partly ambiguous.

As time goes on, the API grows even more, gathering further options. Exhaustive testing of all combinations of workflows and optional behaviour becomes intractable. Natural churn in the provider team leads to knowledge loss. New features and changes are implemented in ways that almost, but not quite, fit the original intention. For consumers, the API begins to feel bloated and the documentation does not offer strong guidance on how to choose between all the options. Onboarding requires close collaboration with the provider team because the documentation alone is no longer sufficient to understand supported usage patterns and expected behaviour.

And there you have it: an API of Endless Possibilities. It has so many optional features and ways of using it that not even the provider team can answer questions about its behaviour outright. Instead of knowing, they have to do research in the code and maybe perform exploratory testing on their own API. It has lost coherence and gained sharp edges. Some combinations of features are well-tested and explicitly supported, others are untested, some are explicitly forbidden, some do work but have non-intuitive consequences. If bugs crop up, they are difficult to diagnose because it is unclear what the intended behaviour even is.

The evolution from a clean, well-designed API to an API of Endless Possibilities is a slow one. There is no clear inflection point, no moment in time you could point to and say, “This is where it went wrong.” Because of this, this pattern is difficult to avoid, but you can watch for warning signs such as:

• The API has multiple options that interact in non-obvious ways.
• The API has options that are incompatible for non-obvious reasons.
• Options that make sense together from a business perspective are only partially implemented (some combinations are missing).
• The API has multiple functions (methods, endpoints, …) that need to be invoked in a certain order – and the order depends on some option(s).
• When adding a new option, it is difficult to decide how it should interact with existing options.
• Certain aspects of the behaviour depend on both options sent by the consumer and configuration on the provider side.
• The team cannot answer a question about API behaviour without research.
• The team cannot answer a question about API behaviour without knowing which customer the question is about.

Here are some suggestions how the provider team can slow – or maybe even avoid – the slide into Endless Possibilities:

• Keep the API focused on a single purpose. Ensure that everyone on the team knows this purpose and has a sense of what fits this purpose and what does not. This helps to avoid feature creep.
• Resist the temptation to use options to implement features that alter the structure of the interaction between consumer and provider; provide a new API for the new interaction pattern instead. A clear separation avoids nasty edge cases that may manifest when mixing interaction patterns.
• If you do add features/options, make them orthogonal to existing features/options. This makes them easier to reason about.
• If this is not possible, at least clearly document the supported combinations and their semantics – and emit helpful error messages if a consumer tries to use an unsupported combination rather than allowing undefined behaviour (because of Hyrum's Law.
• If a consumer does have special needs that cannot be addressed in a straightforward way, it may be better to just hardcode the desired behaviour for this consumer instead of trying for a generic implementation that adds unnecessary complexity for everyone.

Of course, this is easier said than done, and it becomes more and more difficult the further an API grows and the more consumers with diverse use cases it collects. Keeping such an API from devolving into Endless Possibilities requires strong governance with a focus on long-term maintainability. If the provider team fails to establish this, it will reach a point of no return, and it becomes practically impossible to get out of this pattern in an evolutionary way. Then, the only remaining option is revolutionary, i.e., start with a new API design from scratch – and establish stronger governance this time.