Self-organization and emergence as natural dynamics of organization

I’ve just been directed to a post entitled Self-organization vs Emergence on Jurgen Appelo’s Noop.nl blog. This was the third of a trilogy of posts on self-organization that I enjoyed reading – albeit belatedly. However, there are a couple of points with which I would take issue, as set out below.  My particular take on the topic is based on my informal coalitions view of organizational dynamics, which looks at organizations as dynamic networks of self-organizing conversations.

 Self-organization is not about self-managing teams

To begin with, I think that there is often an unhelpful confusion between so-called "self-managing teams" and self-organization as a natural dynamic of people interacting together (which is how I read one of the comments on the post). Many writers and consultants prescribe self-managed teams as a ‘complexity-based’ response to command-and-control regimes. But I would see self-organization and emergence as being just as descriptive of organizational dynamics in heavily hierarchical set-ups as in the practical enactment of any other design.

With this in mind, I was puzzled by Jurgen Appelo’s presentation of “human-created systems” and “self-organizing systems” as a dichotomy. From my perspective, all ('organizational') outcomes emerge from the widespread interplay of the myriad local conversations and interactions that make up day-to-day organizational life. In other words, they are co-created by all of the human beings who participate in those conversations. These outcomes include the formal, ‘legitimate’ trappings of organization - its intentionally designed structures, systems, procedures etc - as well as all of the hidden, messy and informal (sometimes called “shadow-side”) dynamics that co-exist with these.  As such, even when self-managing teams or other management ‘structures’ are ostensibly put in place through formal processes, the decision to do so and the realization of these in practice are still emergent outcomes of the complex social process of everyday interaction.

Seeing organizations as complex social processes not systems

The above interactions extend well beyond the formally acknowledged 'boundaries' of the organization. So another implication of this is that there is no need to think in terms of organizations as systems, with discrete levels of existence and 'parts' wholly dependent upon an (imaginary) 'whole'.  All that exists beyond any present, local, sense-making-cum-action-taking conversations are more conversations. The products of past sense-making (such as formal structures, systems and procedures) only have any practical meaning to the extent, and in the ways, that these are made sense of and acted upon in current interactions. And the same argument applies to the tendency for present sense-making to be influenced by generalized (‘cultural’) patterns of thinking and acting that have emerged and become embedded through earlier exchanges. So, in seeing organizations as dynamic networks of self-organizing conversations (i.e. complex social processeses), I tend to talk of both the formal and informal ‘artefacts’ as ‘imprints’ of past conversations.

Interactions as enabling constraints

On a final point, this process and the outcomes that emerge from it are not within the gift of managers – or anyone else to control. Like everyone else, managers are participants in the ongoing process of conversational interaction.  Power relationships may well be biased in their favour – often significantly so. And this might enable them to exercise greater influence on outcomes than many other people are able to do. But, they cannot control the ongoing process of conversational interaction, which is self-organizing and its outcomes emergent. So here I would agree with Appelo’s comment in the earlier post, Self-organization vs Anarchy, that this process has no ‘ruler’.

However, irrespective of this, individual ‘actors’ are not free to do anything that they wish. They are both enabled and constrained by their ongoing interactions with everyone else. And, within these interactions, by the ways in which they collectively make sense of and act upon the ‘imprints of past conversations’ - at this time, in these circumstances, and with these people.

Comments

[John Tropea]

Great stuff Chris...this post echoes a lot of your past thinking...

What's your thoughts on this part (just ignore the systems part if thinking in the context of organisations):

"...there may be self-organizing systems without emergent properties..."

Thanks

[Dave_Ideate]

Hi Chris,

I like your post and the emphasis on emergence from interactions/conversations.

You might be interested in my InfoQ article - Social Collaboration + Lean Integration = Agile http://www.infoq.com/articles/social-lean-agile

In short, I think the challenge of our time is real-time interaction processing to support feedback, decision support and adaptation.

Best,
Dave

[Chris Rodgers]

Thanks for your comment, John.

The person to whom Appelo refers, in making the comment that there may be self-organizing systems without emergent properties, is Peter Corning. His work has been primarily in the areas of the natural sciences (as a biologist) and evolutionary studies, rather than in organizational dynamics.

As I have argued elsewhere in this blog (here for example: http://informalcoalitions.typepad.com/informal_coalitions/2009/08/on-selforganization-and-emergence-1.html) organizations are not living systems. As a result, drawing examples from physics, chemistry and biology to discuss self-organization and emergence doesn't work for me in relation to the dynamics of organizations.

So I'm afraid that, in an organizational context at least, I can see no situation in which a self-organizing 'system' has no emergent properties. As you're aware, I talk of organizations not as systems but (interchangeably) as either "dynamic networks of self-organizing conversations" or as "complex social processes" (of people interacting together). As such, self-organization is the fundamental dynamic of organization, out of which outcomes perpetually emerge. Some of these outcomes (eventually) enter the formal arenas of the organization in the form of structures, systems and procedures, etc. Others remain 'in the shadows', powerfully influencing ongoing interactions but without being formally acknowledged or, in some cases, without conscious awareness.

I hope this helps,

Chris

[Chris Rodgers]

Hi Dave,

Thank you for your comment and reference to your own article. I'm glad that my remarks on the centrality of "2-way conversation" to organizational dynamics and the outcomes that emerge resonate with your own thinking, from a 'Lean' and 'Agile' perspective. It's good to recognize that it's not all about applying the 'right' tools and techniques!

Cheers, Chris

[Paul Z Jackson]

Thanks Chris, a very clear contribution that may help to keep discussions about complex systems (and emergence and self-organisation) as simple as possible.

[John Tropea]

Perfect Chris...I wonder then in terms of "systems" whether "...there may be self-organizing systems without emergent properties..."

...just out of curiosity

Maybe Twitter would be a good place to crowdsource such a question.

[Chris Rodgers]

Hi John,

I struggle to see how there could be a so-called "self-organizing system" which did not have emergent ('system-wide') properties. I guess it might - again - be a question of definition.

As I've suggested earlier, I would have to exclude organizational dynamics from any consideration of this because (a) these are unavoidably self-organizing and generate emergent outcomes; and (b) organizations are not systems.

In one of his papers, Corning talks about a pile of automobile parts which have been thrown together in a heap, without this resulting in any emergent outcome (a car). At a stretch, I guess, these might be thought of as having 'self-organized' into an amorphous pile (although someone did the throwing!). Only when these components are deliberately assembled into a car "in a precise (purposeful) way" does something useful 'emerge', which is more than the sum of the individual parts. I don't know whether this is what Corning intended to convey or if it's what Appelo means in relation to your question.

In any event, I would strongly take issue with Corning's statement in the same paper that "Living systems and human organizations are largely shaped by “instructions” (functional information) and by cybernetic control processes. They are not, for the most part, self-ordered; they are predominately organized by processes that are “purposeful” (teleonomic) in nature and that rely on “control information.” As regards organizations at least, I would suggest that this reflects a very limited understanding of what's actually going on.

As you suggest, Twitter might provide some examples of self-organizing systems without emergent properties. But I'm not sure what that knowledge might satisy - other than curiosity.

Chris

[John Tropea]

@chris yep, I thought as much, it's hardly worth mentioning...I was prepared to be surprised, but thought it was a weird statement.

I applaud snowden for repeatedly making the difference between systems thinking and complexity
http://www.cognitive-edge.com/blogs/dave/2011/11/babies_should_not_be_thrown_ou.php

http://johntropea.tumblr.com/post/241183740/in-the-idealistic-approach-the-leaders-of-an
“In the idealistic approach, the leaders of an organization set out an ideal future state that they wish to achieve, identify the gap between the ideal and their perception of the present, and seek to close it...Naturalistic approaches, by contrast, seek to understand a sufficiency of the present in order to act to stimulate evolution of the system. Once such stimulation is made, monitoring of emergent patterns becomes a critical activity so that desired patterns can be supported and undesired patterns disrupted. The organization thus evolves to a future that was unknowable in advance, but is more contextually appropriate when discovered."
- Dave Snoweden

hey @dave great article, I really liked it...I have not read much on agile and lean so thanks for this

Dave Snowden has got together with some lean agile guys
http://www.cognitive-edge.com/blogs/dave/2012/01/calmalpha_1.php
http://www.cognitive-edge.com/blogs/dave/2012/01/calmalpha_2.php

[Chris Rodgers]

Thanks, Paul.

I'm glad that the post makes sense to you.

I think that there is often a tendency to seek too complicated an explanation of the complex dynamics of organization. It's the dynamics that are complex, not the explanation.

Cheers, Chris

[Chris Rodgers]

John,

Thanks for the Dave Snowden link. Interesting and informative as always.

I agree with his view that there is a mismatch between the 'gap-closing' approach to organizational interventions adopted by mainstream 'systems thinkers' and the complex dynamics of organizations. It's interesting, though, that he describes organizations as complex adaptive systems - which are the externally programmed products of laboratory experiments - rather than using these to draw insights into the unprogrammed dynamics of the political, ideological, and socially complex world of 'real-life' organizations.

In the extract you quote, I found some of his language interesting, too. For example, he talks of "monitoring" patterns - as opposed, say, to paying attention to the repetitive (and novel) themes that are emerging. The former suggests more of an ability to 'stand outside' the interactions (as the programmer does in relation to the design and operation of complex adaptive systems, for example). And this doesn't fit with the recognition that managers (and consultants) are active participants in the ongoing interactional process; not external, objective observers of other people's actions. There is also, for me, too much of a sense that the desired ("more contextually appropriate") outcomes will be achieved as intended, provided that the prescribed approach is followed. From my perspective, managers and others can act with intent but can have no certainty as to what will emerge - however well informed their actions might be.

In the linked post again, Snowden suggests that Ralph Stacey is wholly dismissive of systems thinking ("positively vitriolic" is the phrase he uses in a comment). In fact, Stacey only takes issue with those who see organizations as systems. He has no problem at all with those (like Seddon) who seek to use 'systems thinking' to improve the formal management, operating and information systems through which work is meant to be accomplished.

It is true that Stacey is particularly dismissive of the use of tools and techniques. He sees these as too often diverting attention away from the richness of what's happening in the moment of people's interactions and replacing this with a generalized, supposedly 'best practice' prescription.

Rightly in my view Stacey identifies prescriptive tools and techniques (of the ‘how to change the world in eight easy steps’ kind) as inadequate and misleading abstractions from the complex social dynamics of everyday interaction. At the same time, I believe that helping people to (re-)frame their everyday experiences, issues and events in particular ways is a central aspect of a leader’s task. And it seems perfectly legitimate to me for managers to develop, adopt or adapt sense-making frameworks to stimulate, provoke and facilitate this process. So this is one of the areas in which I depart somewhat from the stance taken by Stacey and his colleagues at the Complexity and Management Centre.

Chris

[John Tropea]

@dave here's a brilliant presentation in getting some more complexity in the lean perspective

http://www.slideshare.net/jurgenappelo/complexity-versus-lean

[Dave Snowden]

John T reminded me that my original response had been lost. As I don't have a copy this is a re-creation! We may need to agree to disagree on Ralph. I don't have the book to hand, but I distinctly remember one three page rant (in an otherwise good book) against systems dynamics in which he included Nonaka (correct), Boisot (incorrect and several others. Most CAS people see organisations as systems, the question is what type. In that respect I'm not sure Seddon really fits within any clear theoretical framework. He knows what to condemn and he appears to have the ability to persuade managers to do things better (all of which is to be commended) but all I have read or seen is theory light and largely dependent on his insight/personality.

I think everyone with any sense is against the prescriptive tools etc. I agree with you that Stacy takes this too far, and the essence of Cynefin is to say that we need ontological awareness to use different tools in different contexts.

On pattern recognition - I see nothing wrong with saying that in a complex system you need to monitor patterns, even if you are a part of that pattern you are managing the evolutionary potential of the present. The more complex the system the more you attempt to stand above and aside - which is where our software work lies. For example enabling whole of workforce engagement in situation assessment in real time. Objectivising adductive leads, including the density and diversity of human sensor networks, all increase decision making far more than developing individual capabilities as a direct (note that word) intervention.

[Chris Rodgers]

Dave,

Many thanks for your comments.

As you say, we may need to agree to disagree on Ralph Stacey. To begin with, I don’t think that I’ve ever heard or read Stacey rant about anything. It’s true that he argues strongly against many of the established tenets of management orthodoxy and that he is passionate in his belief in what he calls the “complex responsive process” nature of organization. But I think it is stretching it somewhat to call his well-argued challenges a “rant”. I’m not aware of the specific passage to which you refer. However, judging by the names you cite, it’s likely that he was discussing ‘knowledge management’. If so, my guess would be that he would have argued against the ideas that (a) knowledge can be ‘captured’ or managed at all in the ways that this term is usually understood; and (b) that it can be transferred from one person to another in a sender-receiver sort of way. Instead, he would see knowledge emerging in the moment of people’s interactions; and the interactional process itself as a complex responsive, rather than linear transmission, process. But that’s only my speculation.

I mentioned Stacey’s ‘support’ of John Seddon to illustrate the point that it is not the word “system” itself with which he takes issue. His challenge is to those who describe organizations as systems (whether “Complex Adaptive” or any other form), because this runs counter to his complex responsive process view of organizational dynamics (as it does with my informal coalitions/complex social process view). The reality of Seddon’s practice - however he might describe it and whatever the strength or otherwise of its theoretical basis - focuses on improving specific management and information systems. He does this with some success – fuelled, not unreasonably, by his disdain for the use of targets as a performance improvement tool. At the same time, he does not concern himself with the notion of organizations as systems or with the dynamics that flow from that perspective.

Your comment that “the essence of Cynfin is to say that we need ontological awareness to use different tools in different contexts” is, I’m sure, one of the reasons why Stacey would have some difficulty with your position. He would, of course, be able to speak much more articulately on this than I can. The inference I take from your statement is the presumption that managers and others can decide rationally which of the four domains of the framework they happen to be operating in at a particular time and can then use the prescribed approach for that domain to achieve the ends that they desire. Like Stacey, I can’t see this being the case, in the midst of the complex reality of everyday organizational life.

Having just written that statement, it also occurs to me that, paradoxically, the very process of sensing the context, deciding into which category it fits, and then responding accordingly aligns with the suggested response to the “simple” domain in your framework. That is, regardless of the nature of the issue being addressed, it seems that the act of intervention in using the framework itself is presumed to be simple and unproblematic. No doubt you’ll put me right on this.

The notion of managers “attempting to stand above and aside” what’s going on in order to “manage the evolutionary potential of the present” is similarly difficult from my perspective. This is why I raised the original query about your use of the word “monitor”. For me, managers are unavoidably participants in what’s going on – even when they are not present. So participation is their primary ‘state of being’ – whether active or otherwise. They are, as Stacey (after Bourdieu) would say, “immersed in the game”.
They might well strive to adopt a more reflective and reflexive approach to their own and others’ participation in the moment of their interaction – paying attention to the movement of their joint sense-making with others; identifying and exploring dominant themes; challenging clichéd responses (in the sense of actions as well as words); and so on. This is what I would see as the essence of participation from a complex social process perspective. But none of this takes place in any way ‘outside’ those interactions.

And finally, as regards “managing the evolutionary potential of the present”, I would see everyone in the organization as seeking to do this, not just managers. By this I mean that everyone seeks to bring about outcomes that reflect, in whole or in part, their own interests, ideologies and identity. Power relations might ordinarily privilege managers’ participation in this process. But outcomes nevertheless emerge from the widespread interplay of everyone’s interactions, not just those (indeed not primarily those) in which managers are directly involved.

Cheers, Chris

[Dave Snowden]

Ralph is normally charm personified, but his writing is at times polemical (mine is too by the way) and the passage I mentioned is poor (hence my use of rant). As it happens your description of KM is one that would have been supported by Boisot who Stacy lumps in with Nonaka. My comment on that one therefore stands.

On Cynefin, if you read some of my articles you would find that properly used it is an emergent property of conversations not the result of a "rational" process in the sense you use the words. The fifth domain (you make a common mistake by saying it has four) is key to this and covers ambiguity and inauthenticity. However it critically argues that humans can create order while ants/birds cannot. The framework is defined by contextual narrative that allows matching responses to context. I think you repeat the categorisation error here, but as I say is common when people just look at the model with reading the supporting material and methods so no problem there. I keep trying to think of ways to improve the visualisation to avoid that.

Of course managers are immersed in the game (and I drew heavily on Bordieu by the way), but that does not mean that they cannot increase the level of detachment. You (rather like Stacy) seem to posit an either/or here which I think is a mistake. The fact that you cannot fully detach yourself, does not mean that you cannot observe parallel safe-to-fail experiments. Our approach there is to allow investment in any coherent experiment then see what results, so the manager does not make a choice for an approach, but decides to allow coherent but contradictory approaches to improve understanding through interaction with the system. I think this is about process rather than trying to change the individual qualities of people by the way but that is another subject.

Fully agree that all parts of the organisation should be involved in managing the evolutionary potential of the present. The fact that i say managers should do it does not imply that they are the only ones. I think I used that phrase in my original comment which was lost - do you still have that?

[Chris Rodgers]

Thanks again for your comment, Dave.

Given the interesting subject matter, the following response is more post-length than comment-length. But, in the words of Mark Twain, I didn’t have time to write a short one!

To begin with, I empathize with your attempts to ‘get the drawing right’. As regards the Cynefin framework, the picture (when drawn with five domains, ‘roughly sketched’ boundaries, and the ‘fold’ between “Simple” and “Chaotic”) seems to me to be entirely consistent with your verbal explanation. But, beyond the drawing, I do still have some queries/issues re its application. By way of reference, I’ve used your co-authored HBR paper of 2007 as an authoritative summary of the framework and its intended use.

I am aware that you advocate a conversational approach to construct the framework in a particular setting. As I understand it, this allows the boundaries between the five domains to emerge through dialogue, based on people’s perceptions of their own circumstances and history, etc. However, in using the resulting framework to guide their approach to their current situation, people still have to decide in which of the domains they consider themselves to be operating at a particular time. From my perspective, I would question the presumed ability of people to do this in the complex social process that is everyday organization. It was this aspect of the Cynefin process in particular that I felt reflected rational assumptions about managers’ capacity to act.

And that was also why I pointed out what I saw as a disconnect between the framework, which acknowledges the complex reality of organizational life, and a process for using it (once constructed) which is more aligned to the guidance you provide for the “Simple” domain – i.e. sense the nature of the context, decide which category this fits into, and respond accordingly. I know that you challenge the use of the term category here. But, despite the socially constructed nature of the framework (when used as you intend it to be used), such construction only seems to relate to the relative positioning of the boundaries. The overall structure of the framework is not up for debate. That is to say, the five domains/categories and their relative orientation to each other are a given.

By the way, I see nothing wrong with this in relation to the potential utility of the framework. My issue is with the implied ability of people to use this to ‘navigate’ their way through the everyday complexities of organizational life. If, on the other hand, Cynefin is used as a way of stimulating and facilitating meaningful, sense-making-cum-action-taking conversations around specific events, issues or challenges (as in the examples in your HBR paper), that is a different matter. But these are the ‘set piece’ events of organization; analogous to the corners, free-kicks and penalties in football (or should I say line-outs, scrums and penalty kicks?). I’m concerned as much with the dynamics of ‘open play’, which accounts for most of the activity. After all, it’s out of the open-play interactions that the above set-piece events - as well as point-scoring, victories and defeats - emerge.

I am aware that there are five domains, not four. However, you say in your HBR paper, “The very nature of the fifth context—disorder—makes it particularly difficult to recognize when one is in it. Here, multiple perspectives jostle for prominence, factional leaders argue with one another, and cacophony rules. The way out of this realm is to break down the situation into constituent parts and assign each to one of the other four realms. Leaders can then make decisions and intervene in contextually appropriate ways.” So this fifth domain appears merely to be a ‘holding camp’ between one or more of the other four. The last two sentences in this extract also add to my unease about the presumed ability of people to deal with this “cacophony” in the analytical way prescribed. In the paper, you also seem to equate political dynamics with this category of “disorder”, seeing it (a) as wholly negative, and (b) as resolvable in the relatively straightforward way described. But I see organizations (that is, people in interaction) as unavoidably political, both in the ‘macro’ sense and also in terms of the politics inherent in everyday organizational interaction.

This points to a more fundamental concern that I would have with the framework, if it were to be positioned as a description of organizational dynamics ‘in the round’, rather than as a tool to facilitate joint problem solving around specific issues. This is because it explicitly states that there are organizational situations which are not complex (that is, either “simple” or “complicated”). I would argue that all situations that involve people are complex, regardless of how ‘simple’ the context might appear to be. The basic unit of organization is the conversation, and this has all of the attributes of complexity that you mention in the article and others that you don’t. For example, conversations exhibit the dynamics of self-organization, co-creation and emergence. Interestingly, column 3 in the table on page 7 of your HBR paper lays bare this inherent complexity, by identifying a number of distinctly human characteristics (i.e. complexities) that can creep into what appear to be even the simplest of situations – and these only relate to things that leaders might do wrong. Those involved in the multitude of conversations and interactions that make up everyday organizational life similarly display all of the frailties, idiosyncrasies and diverse motivations that are the ‘stuff’ of being human. And this, too, means that nothing is simple – or even merely complicated – when it comes to human interaction. It is always complex. People act in a multitude of different ways, at different times and in different circumstances. They behave thoughtlessly, wisely, irrationally, illogically, emotionally, creatively, politically, carelessly, over zealously, caringly, mistakenly, self-centredly, self-sacrificially, etc. So, as soon as you add people to the mix, simple and complicated processes, systems and procedures become complex. And this means that outcomes cannot be predicted or controlled with certainty in any circumstances. Whilst familiar patterns of response might well be the norm, the potential for novelty to emerge and unexpected outcomes to arise always exists.

Finally, I do agree with you that managers (and others for that matter), whilst unavoidably ‘on the pitch, playing’ can, at the same time, seek to take a detached view of what’s going on within and around them. This is what I meant by “…adopt[ing] a more reflective and reflexive approach to their own and others’ participation in the moment of their interaction – paying attention to the movement of their joint sense-making with others; identifying and exploring dominant themes; challenging clichéd responses (in the sense of actions as well as words); and so on.” I see this as a paradoxical relationship between participation and detachment, not an either-or choice. My challenge related to the language that you used, of managers “attempting to stand above and aside” what’s going on. This sounds more like them ‘sitting in the stands’, so to speak, watching other people’s actions rather than actively participating with them in the messy reality of day-to-day life. Here again, this is less problematic if you are thinking about it in relation to a specific issue, rather than as a description of what’s going on in the day-to-day reality of organizational interaction. But, even then, the manager’s actions (including their silence and inaction) will affect they ways in which others’ perceive, interpret and evaluate what’s going on and decide how they will act.

[Dave Snowden]

Thanks for putting the effort into a reply and I know others are watching this and appreciating the exchange.

Each to their own, but I would be more cautionary than you in taking one article as an authoritative summary of a framework. The HBR article is about the application to Leadership and not about the framework itself or its wider application. I always find it more useful to judge a wider body of work, so just as I would use one article by Stacey to define this work I don't think you should use one article to encompass Cynefin - not that anything in that article is wrong, but its not a complete statement.
I think that people have to define boundaries between different types of things. We do this all the time and it allows us to behave divergently. To say that we make such selections does not mean we are in the Simple domain, that would only apply if the selection method assumed a linear relationship between cause and effect and assumed there was a knowable correct answer. Cynefin domains are defined by multiple mediated narratives within the organisation. They carry with them increasing ambiguity as they move anti-clockwise from Simple. In day to day discourse people use commonly understood examples to create meaning in complex situations (Stacey says something similar as I remember it); so the Cynefin framework builds on that. Oh and yes the overall structure is common, but it emerges from the data. You might want to reflect on language here, without a common language communication is impossible, the same applies to frameworks.

The fifth domain is not a holding domain, that is made very very clear in several articles. Its the normal domain of understanding before people collectively differentiate context to allow divergent actions. That is why it is inauthentic. I didn't think I explained it that badly in the articles, but I will look again to see if there is any way someone could read it as a holding pen, unless of course they were just working from the HBR article without reading the other material in which case I would understand it.

Then we come to the real difference, and here you share a position with Stacy in arguing that all situations involving humans are complex. That is a bigger subject and its one where I think both you and Stacy fail to appreciate the ability of humans to use constraints to create predictability and order. That by the way is no just consciously but also through myths, taboo etc. I worry from time to time that a lot of people are taking complexity theory, but not informing it from anthropology, philosophy, cognitive science and other disciplines.

Oh, and its often more sensible for managers to sit in the stands and limit their engagement ....