You can't put an organization in a test tube - challenging evidence-based practice

In the previous post, I suggested that the idea of "evidence-based practice" is severely limited in the context of organizational dynamics. Organizations are complex social processes, not rational scientific endeavours. As such, they are not amenable to the research and testing protocols needed to provide rigorous 'evidence' of the merit of a particular practice. Or to justify claims that what is perceived to be successful practice in one context can be generalized to others.

In short, you can't put an organization in a test tube. 

The notion of evidence also presupposes the ability to link cause and effect, in an "if you do this, you'll get that" sort of way. But, as I've argued many times in this blog (see here and here, for example), the complex social dynamics of organization make it impossible to establish the links between observed outcomes and specific actions in all but a limited set of conditions.

Against this background, I've set out below what I see as some of the characteristics of so-called "evidence-based practice" (EBP), as this is applied to organizations. And I've contrasted these with others that reflect an informal coalitions understanding of organizational dynamics.

Evidence arises in the midst of practice

From an informal coalitions perspective, 'evidence' of the worthwhileness (or otherwise) of a particular practice arises from people's mindful participation in the ongoing conversational life of an organization. It is through people's everyday interactions that the meaning and value of their individual and collective practice emerges. That is to say...

... all of the 'evidence' needed to support the continuity and change of people's practice is embodied within the detail and patterning of their current interactions.

I'm calling this here, "practice-based evidence," since it arises in the midst of practice.  For ease, I've abbreviated this to PBE in the relevant statements below.

Contrasting perspectives

In the context of organizational dynamics ...

EBP: Is based on prescriptions derived from generalized representations of other people’s past practices and third parties' observations, analyses and reports of these.

PBE:  Is embodied in the specifics of people's own, present experience and practice. That is, on their perceptions, interpretations and evaluations of what’s happening and what they find themselves doing - at that time, in that situation and within those relationships. It privileges the lived experience of people, and the patterning of their being together 'in and around' the focus organization.

EBP: Uses abstract, de-contextualized observations, analyses and conclusions to support the claimed universality of ‘best practice’ prescriptions. Rarely, if ever, exposes the ‘warts and all’ details of people's actual experience.

PBE: Context-rich experience arises within - and at the same time informs - context-specific action, as people participate in ongoing, local interactions. Difference and contention are recognized as natural dynamics of everyday organizational life.

EBP: Plays to the established narrative, in which managers expect rational explanations, predictability of outcome and control of process (‘levers to pull’).

PBE: Recognizes the a-rational, political dynamics of real-world organizations. That is, the inability to predict and control 'outcomes'. And the inevitability of such characteristics as 'messiness' and informality, paradox and ambiguity, polarization and conflict, ideologically-and interest-based evaluation, and so on.

EBP: Prescribes formal and structured approaches for improving practice and performance, based on an idealized and rational model of organizational dynamics.

PBE: Facilitates ‘improvised’, 'in-the-moment' interventions, arising in the midst of local interaction. Also heightens awareness of the self-organizing patterning of taken-for-granted assumptions, which creates expectancy and tends to 'channel' ongoing sensemaking and action-taking down 'well-trodden pathways'.

EBP: Often results in seductively simple 'recipes for success' - such as 'n-step’ processes, 'best practice' guidelines, and prescriptive models and frameworks. Tends to be based on a superficial and/or sanitized understanding of past practice in other organizational contexts. Frequently confuses correlation with causation. And tends to focus on the ‘big’ things, whilst ignoring the seemingly incidental happenings and exchanges that underlie and 'stitch together' these more 'evident' events.

PBE: Avoids prescriptions and the promise of particular outcomes. Instead, sees people as acting with intention into the ongoing social process of shared sense-making-cum-action-taking. Despite such intent, offers no certainty as to the outcomes that will emerge. 'Frameworks', where explicit, are used to facilitate the ongoing sensemaking process and to provoke reflection on existing practice.

EBP: Assumes that skilled introduction of the formally agreed, 'evidence-based' interventions will reliably and consistently deliver the intended outcomes. When shortfalls occur in practice, these are put down to the failure of individuals to implement the prescribed actions correctly.

PBE: Understands that formal interventions can only ever represent idealized statements of managers' aspirations and/or intentions. These will be realized (or not) through local conversational interaction. And although managers and others might act with specific intentions in mind, what emerges will not be within their individual or collective ability to control.

EBP: Tends to provide''new managerialist' prescriptions – positive, structured and measureable - based on assumptions of technical rationality and managerial control.

PBE: The process is indeterminate, reflexive and, complex. Any appeals to technical rationality and managerial control are heavily circumscribed by awareness of the complex social dynamics of human interaction.

EBP: Is presumed to be ethical because it requires practitioners to use knowledge that has been gathered and tested empirically, in what are claimed to be the most rigorous ways possible. It is thus assumed to provide 'objective' evidence of those actions most likely to achieve the desired results and to do so in ways that meet pre-defined standards of ethical behaviour.

PBE: Sees the ethics of practice as emerging in the specifics of people's ongoing interactions - as they seek to go on together in the midst of differing interests, ideologies, identities and interpretations etc.

 

Taking experience seriously

In conclusion, using 'practice-based evidence' as the basis for ongoing action takes experience seriously. But it relies on people's local interpretation of their own, in-the-moment experience - at specific times, in specific places, and in specific circumstances - rather than on the handed-down, generalized and codified experience of others. And this implies people's routine adoption of a reflective and reflexive approach to their individual and collective practice.

Comments

[Adrian Segar]

Chris, we share a number of overlapping interests. I have written about the limitations of a "best practices" viewpoint:

http://www.conferencesthatwork.com/index.php/soapbox/2011/03/next-practices-not-best-practices/

But I'm also struck by the numerous similarities between your description of the characteristics of practice-based evidence and those of the participant-driven conferences I've been designing and facilitating over the last twenty years.

We can think of a conference as a microcosm of an organization; albeit one that exists for a short amount of time with participants who mostly don't know each other and who, therefore, need effective ways to make use of the fleeting time they are together. I work to explore and improve such ways, much as you work to explore and improve organizational effectiveness.

[Chris Rodgers]

Hi Adrian,

Apologies for a somewhat belated response!

Many thanks for your supportive comments and the link to your "Conferences that Work" website. As I've said in a later post on this site "the conversations are the work", and you clearly echo that sentiment in your own practice.

It was interesting, too, that the specific link you included was one in which you decried the use of so-called "best practice" as a route to performance improvement. This is something else with which we are in agreement. Back in November 2008, I wrote a post entitled "Where is the magic in best practice?". This similarly points to some of its many flaws. You can find the post here: http://informalcoalitions.typepad.com/informal_coalitions/2008/11/where-is-the-magic-in-best-practice.html

Thank you again for your comment.