Complexity & Organisation Development:

An Introduction

(Note: the bibliography for this article is stored as a separate page)

Richard Seel

{To download a Word version, click here.}

A man was walking home one dark and foggy night. As he groped his way through the murk he nearly tripped over someone crawling around by a lamp post. "What are you doing?" asked the traveller. "I’m looking for my keys." Replied the other. "Are you sure you lost them here?" asked the first man. "I’m not sure at all," came the reply, "but if I haven’t lost them near this lamp I don’t stand a chance of finding them."

Most people know this story. For me it is metaphor of science. ‘Normal science’ in Kuhn’s (1962) terms, consists of looking as assiduously as possible in the lit area, perhaps exploring those edges where the gloom is not quite impenetrable. From time to time someone manages to switch on a new light—a paradigm shift, in Kuhn’s terms—and a new area of exploration is opened up.

Science is about the art of the possible; it does not deny that the keys may lie in the darkness, it simply does not consider that its job consists of feeling around blindly.

Until recently, the light by which science was working was only able to illuminate simple, linear, systems. The advent of the computer changed things. It is now possible to look at complex systems in two ways: computers can solve previously-impossible nonlinear equations and they can simulate of complex systems by means of models known variously as cellular automata, genetic algorithms, neural nets and so on.

So what is a complex system? The field is still very new and there is no agreement about terms and terminology but the following quotes start to give a flavour:

…a system that is complex, in the sense that a great many independent agents are interacting with each other in a great many ways. (Waldrop 1993:11)

…to understand the behavior of a complex system we must understand not only the behaviour of the parts but how they act together to form the whole. (Bar-Yam, 1997:1)

…you generally find that the basic components and the basic laws are quite simple; the complexity arises because you have a great many of these simple components interacting simultaneously. The complexity is actually in the organization—the myriad possible ways that the components of the system can interact. (Stephen Wolfram, quoted in Waldrop 1993:86)

Complex adaptive systems consist of a number of components, or agents, that interact with each other according to sets of rules that require them to examine and respond to each other’s behaviour in order to improve their behaviour and thus the behaviour of the system they comprise. (Stacey: 1996:10)

…the complex whole may exhibit properties that are not readily explained by understanding its parts. The complex whole, in a completely nonmystical sense, can often exhibit collective properties, "emergent" features that are lawful in their own right. (Kauffman 1996:vii-viii)

The task of formulating theory for CAS [complex adaptive system] is more than usually difficult because the behaviour of a whole CAS is more than a simple sum of the behaviours of its parts; CAS abound in nonlinearities… (Holland 1995:5)

…complexity is not located at a specific, identifiable site in a system. Because complexity results from the interaction between the components of a system, complexity is manifested at the level of the system itself. There is neither something at a level below (a source), nor at a level above (a meta-description), capable of capturing the essence of complexity. (Cilliers 1998:2-3).

Complex system characteristics

In the early days of complex systems theory the emphasis was on large networks of simple agents with simple interactions. More recently there has been a realisation that smaller networks of complex agents can show the same kinds of behaviour and can be equally complex.

Complex systems have a number of properties, some of which are listed below:

Emergence

What distinguishes a complex system from a merely complicated one is that some behaviours and patterns emerge in complex systems as a result of the patterns of relationship between the elements. Emergence is perhaps the key property of complex systems and a lot of work is being done to try to understand more about its nature and the conditions which will help it to occur.

Relationships are short-range

Typically, the relationships between elements in a complex system are short-range, that is information is normally received from near neighbours. The richness of the connections means that communications will pass across the system but will probably be modified on the way.

Relationships are non-linear

There are rarely simple cause and effect relationships between elements. A small stimulus may cause a large effect, or no effect at all.

Relationships contain feedback loops

Both negative (damping) and positive (amplifying) feedback are key ingredients of complex systems. The effects of an agent’s actions are fed back to the agent and this, in turn, affects the way the agent behaves in the future. This set of constantly adapting nonlinear relationships lies at the heart of what makes a complex system special.

Complex systems are open

Complex systems are open systems—that is, energy and information are constantly being imported and exported across system boundaries. Because of this, complex systems are usually far from equilibrium: even though there is constant change there is also the appearance of stability.

The parts cannot contain the whole

There is a sense in which elements in a complex system cannot ‘know’ what is happening in the system as a whole. If they could, all the complexity would have to be present in that element. Yet since the complexity is created by the relationships between elements that is simply impossible. A corollary of this is that no element in the system could hope to control the system.

Complex systems have a history

The history of a complex system is important and cannot be ignored. Even a small change in circumstances can lead to large deviations in the future.

Complex systems are nested

Another key aspect of complex adaptive systems is that the components of the system—usually referred to as agents—as themselves complex adaptive systems. So an economy is made up of organisations which are made up of people which are made up of brains, which are made up of cells—all of which are complex adaptive systems.

Boundaries are difficult to determine

It is usually difficult to determine the boundaries of a complex system. The decision is usually based on the observer’s needs and prejudices rather than any intrinsic property of the system itself.

For instance, the boundary of an individual human being may appear easy to determine but a little more thought will show some of the ambiguities. For instance, are clothes inside or outside the boundary? If someone stares at you across a room or crowded train, especially in a lustful or aggressive way, have they invaded your boundary? When do waste products, such as hair or nail clippings, cease to be part of the body (certainly, those who practise magic feel that they remain within the boundaries)?

We often hear of groups having ‘strong’ or ‘weak’ boundaries but without any clear sense of the meaning. I believe that it is possible to gain some clarity by considering connectivity. I hypothesise that an individual agent can only have a certain number of connections to other agents (with human agents this number will change according to the state of the individual and also the state of the environment). We can then think of the strength of a group’s boundaries as the proportion of connections which are made within the group—the greater the proportion, the stronger the group boundaries. If all connections are made within the group it forms a closed system

Are organisations complex adaptive systems?

At this point it is worth asking whether complexity theory has anything useful to say about organisations. Is an organisation a complex system? Gareth Morgan offers complexity as just one of his metaphors for organisation. There are many who would argue that complexity is not just a metaphor for organisations, it is an adequate and accurate description of organisations. Morgan himself appears not to discriminate between the different kinds of metaphor he offers. Consider the following:

I believe that these three likenesses are ontologically distinct. When we speak of an organisation as a machine we are employing an obvious analogy—no-one could confuse the two.

When we speak of an organisation as a culture I will argue later that this is an inappropriate figure of speech. Culture is a property of (human) organisations; albeit an interesting and pervasive property. Culture emerges as a result of the interactions between the agents in a human system and it is simply not appropriate to compare an entity and its epiphenomenon in a metaphorical manner.

Finally, to speak of an organisation as a complex system is to adopt a theoretical stance. It is to assert that an organisation is more or less appropriately described in terms of the insights being developed by complexity theorists. It is my belief that these approaches give us the best insight we currently have into the nature of organisational behaviour. However, it must be recognised that complexity theory is at present still very tentative and undeveloped, especially in the field of human organisations.

Human beings, as agents in a complex system, are significantly more complex than those usually considered by the theory. They have volition, they are members of many different complex systems simultaneously, and so on. All of this sets them apart from the systems explored by the computer simulations So my current position is to continue to explore the implications of complexity for organisation consultancy while remaining open to other models, approaches and metaphors which may supplement or challenge these perspectives.

Implications of complexity theory for organisations

There are a number of implications which complexity theory may potentially have for organisations. I can only mention a few of them here.

Inability to predict

One of the features of complex systems is that they have what is known as sensitivity to initial conditions. This means that a vanishingly small difference in the initial conditions (whenever you choose to start) can make a staggeringly large difference as time goes on.

The classical formulation of this comes from meteorology (Edward Lorenz, a meteorologist, was one of the first (1963) to investigate the properties of complex systems such as weather systems). It states that even such a small perturbation as a butterfly flapping its wings could—because of the nonlinear nature of the system—lead to a tornado some months or years later. Of course, the chances are that it won’t; the real issue is that it is theoretically impossible to predict whether or not it will.

This has obvious major implications for strategy, most of which are consistently ignored by organisations. Not that we should advocate that organisations cease to plan; as a piece of ritual the annual planning round has a significant part to play in organisational life. Also, the fact that fully accurate prediction is impossible is actually a commonplace—everyone knows that the only certainty is that the plan will be wrong.

What is important is that the focus should be on "Planning as Learning" (de Geus 1988). Approaches such as environmental scanning and scenario planning (see, e.g. van der Heijden, 1996) should be used to help the organisation raise awareness and increase connectivity with the outside world (to reduce the strength of its boundary).

Inability to control

Perhaps the most crucial, but to clients the most controversial, perspective from complexity theory is that it is impossible to control what happens to the system. So much management literature focuses on the role of the ‘leader’—and I don’t want to underplay his or her importance as an enabler of change—that most leaders (and managers aspiring to become leaders) believe that they can ‘make things happen’. Mechanical metaphors still dominate most management thinking; organic approaches may have a higher feel-good factor but they cut little ice with those charged with satisfying shareholders.

Yet a fundamental result of systems thinking in general and complexity theory in particular, is that no one element can have enough complexity to be able to comprehend the system as a whole. If it can, the system is not complex.

Edge of chaos

A key concept in much writing about complexity and organisations is the edge of chaos. It has been popularised by Stuart Kauffman (1995) of the Santa Fe Institute, the leading centre for the study of complex adaptive systems. The term was actually coined by Chris Langton, another worker at Santa Fe who was working with a kind of computer simulation called a cellular automaton. Langton discovered that as he changed the value of a particular variable his simulation suddenly exhibited ordered behaviour and then became disordered again. The region where changes occurred, he called the edge of chaos.

This concept may help to deal with a key question in organisation development: "how can we know if an organisation is ready to change?" The answer is that we cannot know (though intuition may often be a reliable guide) but there are some key variables which have a significant effect on readiness and ability to change.

If there is too much stability in the system change is unlikely; if there is too much randomness the system will not be able to form any coherent patterns. Kaufmann and other researchers (see e.g. Kauffman 1995, Holland 1995, Bak 1997), working with computer simulations, suggest that there are three variables which are significant in moving systems to the edge of chaos: connectivity, diversity and information flow.

Basically, stable systems can move towards the edge of chaos if their agents become better connected; if there is more diversity (either in the agents themselves or in the nature of the relationships between them); and if the amount of information transferred is increased. Conversely, an unstable system, one with too much randomness, needs to reduce some or all of these variables.

Ralph Stacey, drawing on Kleinian perspectives, argues (1996:177ff) that in human systems two other variables are also significant: level of contained anxiety and the power differentials in the system. If the anxiety in an organisation is too contained there will be no possibility of change or creativity; if there is too much anxiety around there will be a tendency for ‘headless chicken’ behaviour or else for the building of spurious and unhelpful defences.

Similarly, if there is too much control, in the form of high power differentials between different parts of the organisation, creativity and readiness for change are likely to be stifled. Contrariwise, if the control mechanisms are too weak the system can dissolve into chaotic or random behaviour.

These five influences can be summarised as follows:

My own more recent thinking draws on Stacey's approach and takes it further.

Interventions such as system-wide inquiries (see below for appreciative inquiry or Marshall & McLean 1985 or Reason 1994 for different forms of collaborative inquiry) can help to build connectivity and move an organisation towards the edge of chaos. Arguably, it is the existence of the connections rather than the content of the messages which is more important.

Initiatives such as that of Patricia Shaw (1997) and Bill Critchley (1998) who describe how they worked in the shadow system of a local authority also serve to increase connectivity and diversity. They saw their role as enabling people to meet and share concerns and also to provoke existing groups to examine and question their assumptions and the structures of meaning they were creating.

The introduction of diversity is more problematic. If it is too different it may be rejected; if it is too similar it may make no difference. The challenge, especially for the consultant, is to be similar enough to be listened to, and different enough to be heard. In my own work with the British Post Office I use the phrase ‘support and subvert’ to summarise one aspect of my own approach to introducing diversity.

Self-organisation & emergence

Perhaps the most interesting aspect of complex systems is their ability to self-organise; for ordered patterns to emerge simply as a result of the relationships and interactions of the constituent agents, without any external control or design.

When a complex system is at the edge of chaos it is in a state where change may occur easily and spontaneously. Stuart Kauffman refers to this as order for free. It doesn’t arise through conscious design but is something immanent within the system—a property of the relationships between the elements rather than the elements themselves. (It is worth noting that ‘order’ in this context is not the same as stability or equilibrium; rather, it refers to behaviours which can often be quite dynamic and involve unpredictable though patterned changes.)

When an organisation is poised at the edge of chaos even a small stimulus may cause major change to ripple through, like some kind of domino effect. The work of Per Bak (1997) is interesting here. He suggests that change in such a system will take the form of a power law. That is, most of the time a small stimulus will cause a small effect, that some of the time a small stimulus will cause a medium sized effect, and occasionally a small stimulus will cause a huge effect. Unfortunately, there is no way to predict the size of the effect at any given time.

Emergent order:

Ability to influence

Another way of looking at emergence is to think about the dynamics of a complex system. If all states were equally likely, then emergence would not occur. Instead, it appears that a relatively few configurations are ‘privileged’ in some way. These configurations are sometimes known as attractors. There is a lot of misunderstanding of this term but it can be useful in helping to make sense of complex behaviour. Capra (1997:127ff) and Casti (1994:25ff) both offer reasonably accessible introductions.

So we could say that a complex system will self-organise onto an attractor. It is not possible to dictate the nature of the attractor because a complex system is intrinsically unpredictable and uncontrollable. The questions is, can we influence the nature of the attractor which the system ‘chooses’?

I believe that there is some evidence to suggest that this may be the case, at least with systems involving human beings. One example would be the placebo effect. If we assume that the immune system is a self-organising system which can be assisted by medicinal drugs then it would seem that a placebo can help it self-organise onto the same attractor as a drug—as long as the human host believes that the drug has been administered.

Similarly, labelling theory offers examples of behaviours which emerge as a result of beliefs; again the choice of attractor is influenced (but not determined) by the mind set. It is well-known that children who are labelled as ‘slow’ or ‘stupid’ at school tend to conform to that label, underachieving compared to those with similar innate ability who are positively labelled.

Interventions such as Appreciative Inquiry—often known as AI—(Cooperrider 1990, Whitney & Schau 1998) try to influence the way change can emerge in organisations. A central tenet of AI is that, ‘organisations change in the direction in which they inquire’. If this is true then the simple fact of greater connectivity is not sufficient; the content also counts.

Small set of simple rules

Cellular automata, such as Life or Cellab, show that quite simple rules, applied again and again, can lead to complex behaviour. Similar approaches have been used to model behaviours in the natural world. One of the pioneers was Craig Reynolds (1987) who modelled flocking behaviour using a small set of rules:

Separation:

steer to avoid crowding local flockmates.

Alignment:

steer towards the average heading of local flockmates.

Cohesion:

steer to move toward the average position of local flockmates.

These three simple rules can change a random assembly of agents into a cohesive group, looking just like a flock of birds or shoal of fish. The US Marines are using this principle to increase flexibility when command lines may get broken (keep moving, use surprise, take high ground wherever possible, for instance).

Another example is the Phoenix Fire Department where the judgement that fire-fighters must exercise in any situation is now governed by five words: "Prevent harm. Survive. Be nice." It is amazing how much complex behaviour these simple rules permit, as well as how much adaptation.

Fitness landscapes

Because the environment of a CAS is made up of other CASs, all competing for resources, the dynamic between them is constantly changing in a nonlinear fashion. In fact, both competition and co-operation are at work simultaneously, leading not just to evolution but to co-evolution.

Consider why an organisation changes (or wants to change, or needs to change). It is usually in response to a change in its environment. In other words, as an adaptive response to environmental change. But why does the environment change? Usually because the organisations which make up the ‘business ecosystem’ have changed. In other words, as an adaptive response to organisational change.

This complex ‘chicken-egg-chicken’ form of co-evolution is absolutely key for understanding complex systems and organisational change. Companies are neither masters nor slaves of their destinies. New competitive and collaborative strategies are now being explored in response to these insights (Moore 1996, Nalebuff & Brandenburger 1996).

Another way of looking at this wider environment is to consider the notion of ‘fitness’. At any given time some organisations are more successful than others; they are ‘fitter’ than others. The fitness of a system changes over time because of the constantly changing environment, which is being remade from moment to moment as an emergent result of the interactions between the systems. This means that a configuration which has a fitness f₁ at time t₁ (relative to the other systems in the environment) is most unlikely to have the same fitness at time t₂.

Stuart Kauffman (1996:149ff; c.f. McMaster 1997:157ff) has developed the notion of the fitness landscape, originally suggested by Sewell Wright in the 1930s (Lewin 1993:57). It offers a model of what might be happening in this complex dynamic. Kauffman suggests that a snapshot of an environment at a given time t₁ could be thought of as a landscape. If all points in the environment are equally fit, the landscape will be flat like Norfolk or Illinois. If there are differences in fitness between systems, with some very fit and others very unfit, the landscape will resemble the Himalayas. If one system is very much fitter than all the others we get what is sometimes known as a Fujiyama landscape—one enormous peak in an otherwise relatively flat landscape.

The system at the top of a peak enjoys an advantage over its fellows. Thus the system at 10 {} is currently the fittest of all the systems in its environment:

However, because of the co-created nature of the landscape, today’s mountains may easily turn into tomorrow’s molehills. A system, by standing still, may find itself no longer on a peak:

The system at 10 is no longer as fit as it could be, since the new local peaks are at 7 and 13 and it is a long way behind any system which could reach 19. However, it is not entirely stuck and could manage to improve its fitness by following what Kauffman calls an ‘adaptive walk’ (1995:166ff).

This is a strategy in which a system ‘moves’ one step on the landscape and tests its fitness. If the new location is fitter than the old, it takes another step. If the new location is less fit than the old it returns to its previous location. Thus by going ‘right’ by three steps our system could adaptively walk to the peak at 13.

However, the adaptive walk could never get our system to the best peak at 19 because it would have to walk ‘downhill’ from 13 to 17, significantly decreasing its own fitness. In practice it is likely to get stuck on what Kauffman calls a ‘local peak’ and may find itself left behind as the landscape deforms.

Actually, things are even more complicated because as a system moves across a fitness landscape it changes the nature of the landscape by virtue of its interactions with other systems. Kauffman says that it’s a bit like try to walk across a sheet of rubber. Each step deforms the sheet in a different way. Co-evolution has two messages for us: that we can never control our environment and also that we need never be passive spectators as the landscapes change. What we do both affects and is affected by others.

Patching

Because the only way to get to a distant fitness peak will involve getting less fit before getting better, organisations are often reluctant to undertake such a journey. Even those chief execs who intuitively know what has to be done seldom have models which will help them articulate and communicate their vision. It’s also pretty de-motivating to have to say to people that you want them do make the organisation worse! Shareholders might not be too impressed either.

So they try to optimise the whole system which often leads to it getting stuck on a low peak because no-one wants things to get worse (go down a valley). However, Kauffman discovered a technique, which he called patching, which seems to offer a way forward.

Patching breaks a system into connected chunks which then try to self-optimise. So an organisation might be broken into work groups, business units, profit centres, etc. Each is then given the freedom and encouragement to do as well as it can—to improve its own fitness. The side effect of success for any given patch may be to cause neighbouring patches to be worse off (go down a valley) and this may lead to the organisation becoming worse off for a time. But if the process is allowed to continue it will lead to eventual improvement as the system climbs a new peak, one that could not have been reached by a simple adaptive walk.

Kauffman also found that for any given system which he modelled that there is an optimum number of patches to help the system move to a new fitness peak. Unfortunately, there is currently no known way to predict that number even for a simple computer simulation, let alone a human organisation.

I have found some insights from complexity theory useful in thinking about organisational culture. My "Culture & Complexity" gives more on this work.

Conclusion

There is much more. Complexity theory is an immature field, still developing. It offers great challenge to the organisation theorist and some tantalising possibilities and models for the organisational practitioner.

For some it is too flaky, too counter to common sense; for others it is an inexhaustible source of stimulus and excitement.

Bibliography

This article is indexed here.

Send mail to richard@new-paradigm.co.uk   with questions or comments about this web site.
Last modified: 12th January 2008