The future of systems: to live in two places at the same time

Structures are important. They define a lot of our organizations’ behaviour, and how we can and cannot implement change. But a system’s structures also define how (and what) communication happens (or not). As Mel Conway noted in How committees invent:

“The need to communicate at any time depends on the system concept in effect at that time.”

As I will try to argue in this post, this can be a problem, especially when we look at how we bring change to our systems: in this case, the right kind of communication accross two distinct subsystems is essential for our change initiatives of any sort to succeed–or else, we will fail.

Let me elaborate.

Working in the system – working on the system#

While we might look at our systems from the perspective of multiple, but similar, subsystems, almost any organization or reasonably complex system¹ can actually be seen as a system comprised of two very distinct functions or subsystems²:

• One big part of the system where the “value creating” work happens: the place where we work in the system, and
• One smaller part where work on redesigning the system for the future: a place to work on the system

Illustrated (simply), our organization thus looks as follows:

In business or in government, in software development, in art or at home: Our everyday work is – usually – a kind of working in the system, within structures and processes, with objectives, doctrines and beliefs that define the boundaries of what seems possible and what not.

On the other hand, if we want to change how we work, we undertake to work on the system³, often somehow redesigning it “from the outside.”

And from a distance, all seems good. Two clear subsystems with very distinct functions, two clear mandates.

But it’s not: if we look closer, there’s no overlap between working in and on the system. In fact, they barely touch each other. They’re just two very distinct subsystems of the same overall system⁴. I think this is a problem, and one that is grossly underestimated.

To live in two subsystems at the same time#

But why is this a problem?

I think that meaningful change can only happen if at least part of the organization masters the paradox of living in two places at the same time. Or, if we take the above image again: we need overlap between “working in the system” and “working on the system”⁵:

Luckily, I’m not the only one to think so. In an older blogpost on sense-making tools, Harold Jarche quoted Daniel Dennett⁶, where Dennet notes the necessity to know the tradition before you can break it (highlights mine)⁷:

When an artistic tradition reaches the point where literally ‘anything goes’, those who want to be creative have a problem: there are no fixed rules to rebel against, no complacent expectations to shatter, nothing to subvert, no background against which to create something that is both surprising and yet meaningful. It helps to know the tradition if you want to subvert it. That’s why so few dabblers or novices succeed in coming up with anything truly creative.

Following Dennet’s observation, and looking at my personal experience so far, I would argue that every successful attempt at disruptive / substantial change implies that one simultaneously has deep (personal!) insights into the system and needs to be able to live outside the system (or at least at its edge) in order to redesign (change) it.

In other words:

In the long run, we can only bring meaningful substantial change to a system if we’re at the same time part of it and working from its edges to redesign it.

It may sound intuitive and logical, but it isn’t. It matters more than we might think and many failed projects, reorgs or change initiatives are a testament to this.

The surprising amount of detail of reality#

It probably becomes more clear when we look at another extreme of “working on the system”: management consulting. Looking at our illustration, consulting is the epitome of “detached” change. When outside consultants are hired, they will inevitably operate from beyond the edges of the system, with only very little overlap, and almost certainly no overlap with the people working in the system.

While there is undoubtedly lots of useful, helpful and appropriate consulting happening in the world⁸, there is also lots of anecdotal evidence how external consultants completely messed-up change initiatives.

In fact, even consultancies like McKinsey admit that being too detached from “working in the system” can create trouble:

Relying on a small team of smart folks to design the details is even more hazardous. When the new organization launches, it will be the employees who determine whether it will deliver value by working (or not working) in new ways and with a different boss (or a different boss’s boss’s boss).

The reason for these failures: The amount of detail that matters when dealing with reality, and where these originate from.

In “Reality has a surprising amount of detail”, John Salvatier writes:

The more difficult your mission, the more details there will be that are critical to understand for success. […] You might also hope that the important details will be obvious when you run into them, but not so. Such details aren’t automatically visible, even when you’re directly running up against them.

The details that matter when working on the system are almost exclusively details from within the system. And you can only know them with sufficient accuracy if your knowledge about the system is deep. As Dennet put it⁶:

If you want to subvert the system, it helps if you know the tradition.

Dependency#

Now that you’ve heard my argument so far, you might argue that it should be sufficient to create just enough functional overlap between the two subsystems to bring these detailed insights to the table. Just a bunch of staff who work on both sides of the schism. And to some extent, you might be right.

But in the long run, you will need much more than “just” some function overlap. What you are striving for is an approach where your redesign is constantly informed by all the relevant but intricate details of the current system–its communication structures probably first, but also the details of operations, details of the infrastructure and all the quirks that matter more than you can imagine.

In the words of this already famous XKCD cartoon: You want to particularly know about “this project some random guy in Nebraska has been thanklessly maintaining since 2003.”

Architectural heuristics as a practical help#

What we therefore need is a heuristic that will help our redesign projects to surface any such crucial detail without involving everybody who’s working in the system to be equally part of our working on the system initiative.

One such heuristic is Christopher Alexander’s principle of structure-preserving transformations (something I’ve already written about in my post on the future of government.

In the words of Nikos A. Salingaros:

This idea of design–as “transformation” using an adaptive process–is very much at the heart of […] Christopher Alexander’s work. Through that adaptive process, we generate a form that achieves our “preferred” state. But at each step of this transformation, Alexander says, we are dealing with a whole system–not an assembly of bits. (Design for a Living Planet, Chapter 20) ⁹

Altogether, there are five “architectural design heuristics” that we can consider⁹¹⁰:

• Step-wise: Perform one adaptive step at a time.
• Reversible: Test design decisions using models; “trial and error”; if it doesn’t work, un-do it.
• Structure-preserving: Each step builds upon what’s already there.
• Design from weakness: Each step improves coherence.
• New from existing: Emergent structure combines what is already there into new form.

Put together, they form a kind of dance that we can teach in our organisations–and if those working in the system learn to dance with the system, then the details needed to work on the systems will emerge naturally–and beautifully.

A problem to solve before we start redesigning our systems#

Every meaningful organisation redesign (or change of a system in the broader sense) needs both: deep and well-founded insights and experience from working in the system, and the levers at the edge to effectively change/redesign it. Ideally, both are tightly intertwined.

Yet, many, if not most, traditional organizations are not designed this way, and this is particularly true for bureaucracies–even if they sport an innovation lab or department. There are many accounts of what happens if you can’t manage to juggle of working in the system while at the same time working on it from the edges. Just two examples:

• Simone Bahn in HBR which cites “Lack of Alignment with the Business” as one major reason why Innovation Lab initiatives fail,
• IMD on change initiatives that don’t take off because senior management abdicates its responsibility to drive the process, instead of “[remaining] fully engaged throughout the transformation process, even as they continue to run the business” (highlight mine)

It takes a deliberate attempt at working towards a state where one sustains the ambiguity of doing both at the same time. What we need to learn then, is how to work in the realisation of an existing architecture–where we fully experience its shortcomings (and advantages)–and how to bring this detailed knowledge to the redesign and change we want to bring to the system (architecture) in meaningful ways.

The concepts embedded in Alexander’s thinking, the idea of heuristics that help us discover the relevant details buried deep inside our systems are my attempt to look at the problem and to think about approaches to help people working on the system to also live (or work) in the system.

Conclusion#

How we manage to effectively live in the two subsystems of our system at the same time is thus the problem to solve–even before we should think of how to redesign the system.

To sum up my argument:

• Many complex human systems can be seen as comprising of two distinct subsystems with the following functions: Working in the system and working on the system
• In order to be effective in our work on the system, we must learn to live in both subsystems at the same time
• The reason for this is the surprising amount of detail of reality which requires us to be rooted deeply in the details of the system if our redesigning of it can be effective
• Eventually, our redesign attempts need to become aware of all critical dependencies which are often buried deep in our systems and organizations¹¹
• Architecture, and more precisely, Alexander’s principle of structure-preserving transformation and other heuristics can provide us with some initial “tools” to tackle this problem, as function overlap (in itself) is probably not enough to achieve
• Therefore, before we start redesigning our systems, before answering the (undoubtedly important) question on how to redesign a system, we inevitably must solve the problem of working in and on the system at the same time

The solutions to the conundrum presented here are at the same time simple and complex. Simple because most people intuitively understand its importance if they start thinking about – complex because there’s no one single answer and because the systems we want to change are complex themselves.

Epilogue#

I’ve been ruminating quite some time about this and writing the article took much longer than anticipated. Many–if not all–of the things I wrote down have most probably been seen and written by people far better in bringing it to paper than me, so forgive me if I repeated the obvious. For me it was (is) important, as two relevant things happened while writing this:

• I’ve had a number of discussions or interactions with people about to engage in change initiatives (IT, business, organizational development – you name them) or summing up past experience. In almost all these discussions, “working in the system” and “working on the system” were things that obviously were treated as a disconnected thing.
• Danny Bürkli started to work on a talk on ’new forms of organization in public administration (as a response to a changing environment)’ and shared a lot of his thoughts on Twitter.

Loved all the generous responses to the question below. Thank you.

Here's the current structure of the argument on "new forms of organization in public administration (as a response to a changing environment)" (subject to - *a whole lot of* - change)

[Thread]

⬇️ https://t.co/hnQz0HlbGy

— Danny Buerkli (@dannybuerkli) August 23, 2020

The material Danny has compiled (both his own and from comments) is impressive. The whole thread contains 20 or more references to books and thinking about this. And all of them are fantastic. Yet, when going through them, something felt missing. Part of it can be explained by Simon Parkers’ argument which I picked up in a post of my own: we still mostly debate agile and design thinking and tools and instruments when debating the future of government. But as Simon wrote: “[P]ublic service isn’t really about services, but about highly complex problems.”

I am convinced that our thinking about the future of government (or any reasonably complex system) should not only include new forms of organization, new tools and instruments, more Agile or SAFe or ITIL or Kanban. It must rather go back and acknowledge the amount of detail that matters in solving the complex problems we face in these systems – and the necessity to get this knowledge from working in the system while working on the system.