Visible Problems and Best Practice

Following my post on Visible Problems, @jchyip commented that some organisations are good at making problems, especially important ones, visible. When I asked for some examples, he replied, "Toyota has to be the easiest one. I'd expect Lean, Agile, Kanban shops to be good at this as well".

I agree that if you have a prevailing doctrine like Lean, Agile or Kanban, then this will shine a light on some class of otherwise invisible problems. Furthermore, adherents to these doctrines (we might call then "best practices") will generally regard these problems as the most important ones.

When a doctrine first appears in an organization, it may generate a string of insights that are typically regarded with excitement by a small number of champions, and with resistance (scepticism, incomprehension) from most of their colleagues. If the doctrine eventually takes hold within the organization, the resistance will gradually dwindle, and the doctrine will become more efficient at identifying and dealing with this class of problem within this organization. However, the more the organization adapts itself to a doctrine, the more attention is drawn towards the particular class of problems in which the doctrine specializes. This is a form of cognitive closure, as described by Maturana and Varela. Which raises the obvious question - what about all the other invisible problems?

The efficient operation of a doctrine within an organization contributes positively to the intelligence of the organization under the following conditions.

• The doctrine helps to identify problems that are strongly relevant to the viability of the organization.
• The doctrine provides a shared narrative that allows efficient communication  and collaboration about these problems and their solution.

However, the overall effect of a doctrine on the intelligence of an organization can turn negative under the following conditions.

• The major problems identified by the doctrine have largely been dealt with, or have been embedded in normal operations, and the doctrine is increasingly being used for more minor or marginal problems.
• The doctrine interferes with the identification of a completely different class of problems that are equally or more relevant to the viability of the organization.
• All problems are framed according to the chosen doctrine, leaving no space for alternative narratives to emerge. 

This can get worse over time. From this point of view, maturity doesn't necessarily mean ever-increasing excellence but might mean slowly declining power.

The implications of this argument are not to abandon all doctrine (which can turn into an equally dogmatic doctrine of no-doctrine), but to encourage a degree of doctrinal pluralism that is appropriate for the scale and complexity of the organization. Instead of a single lens, an array of different lenses.

Visible Problems

@jchyip tweets "Just because a problem is visible doesn't mean that it's the most important one to deal with first."

In hierarchical organizations, the most important problem to deal with first is the one visible to your boss - or his boss. So much the worse for hierarchical organizations of course.

@flowchainsensei agrees, but says he would prefer the word "analytic", and posts a chart onto Twitter to clarify his use of the word "analytic".



Which types of organization are good at dealing with invisible problems? I searched for "invisible problems" on the Internet and found a few random examples: alcohol, presenteeism, cell-phone antennas, gambling, racism, violence against girls in school, vulnerable customers, water. One important point about any invisible problem is that problem-solving contains at least two additional steps - firstly persuading yourself that there is a problem at all, and secondly motivating others to help you solve it.

Sometimes you have to prepare a solution, but you cannot get enough resources or political support to implement your solution - until something happens to make the problem visible. Which means that whenever something dreadful happens, there are always opportunists who try to use this event as a pretext for introducing some innovation or other, which (they optimistically argue) would prevent such an event ever occurring again. Indeed, sometimes a dreadful event is so politically convenient for certain parties or interest groups that they may even be accused (by their opponents or by conspiracy theorists) of having engineered the event themselves (POSIWID).

An intelligent organization should have the capacity to identify and deal with invisible problems - making potential problems visible, and motivating people to anticipate and prevent problems before they occur. This is not the only defining characteristic of an intelligent organization, but it is one of the things that we should expect more intelligent organizations to be better at than less intelligent organizations.


I'd appreciate some good examples of such organizations to go into the book I'm writing on organizational intelligence.

Reinventing the Wheel

@jschwa1 via @hlsdk and @JohnIMM on not reinventing the wheel, recommending examples and techniques to avoid (being accused of) wasting effort and resources.

There are two issues here. Firstly the trade-off between design time and search time. In the short term, it doesn't make sense to spend half a day searching for something that you could build in a hour, even if the lifetime consequences of unnecessary duplication and complication may cost a lot more.

Secondly, there is a trade-off between using an existing wheel (which might be okay but not perfectly designed for this particular task) and designing a better wheel. I'm guessing that there are engineers at any major car manufacturer dedicated to re-inventing the wheels - otherwise we'd still be using Henry Ford's design.

Thus the management challenge here is twofold. Firstly, making sure that there is sufficient access to existing knowledge and ideas to allow engineers to build on what went before. And secondly making sure that there is enough management information and intelligence to achieve a reasonable balance between innovation and reuse.

---

Great summary from @j4ngis When you re-invent wheels - re-use knowledge about existing wheels.

Clockwork or Snakepit?

In a classic essay (later included in his book Narcissistic Process and Corporate Decay, New York University 1990), Howard Schwartz distinguishes between two views of organization - clockwork or snakepit (extract). John Darwin later introduced a third view, which he called rainforest Working the Boundaries, (Social Issues, October 2001).

---

Clockwork

• Everyone knows what the organization is all about, and is concerned solely with carrying out its mission
• People are basically happy in their work
• Level of anxiety is low
• People interact and cooperate without friction. Mutual support.
• Management problems are easily solved with proper skills and correct techniques.

Snakepit

• Everything is always falling apart. Your first concern is to make sure it doesn’t fall on you.
• Nobody really knows what is going on. But everyone wants to know, because there is danger in not knowing.
• Anxiety and stress are constant companions.
• People deal with one another with little pleasure and considerable suspicion.
• Management problems are intractable. Managers feel they’ve done well if they can make it through the day.

Rainforest

• Accept unpredictability and the likelihood of emergence
• Search for and discover patterns beneath complexity
• Accept fuzziness (and distinguish fuzzy thinking from sloppy thinking!)
• Identify and use both positive and negative feedback
• Recognise the capacity for self-organisation, and the freedom that must be given to facilitate this
• Address the need to develop the organisation’s intelligence and ability to generate knowledge
• Recognise codependent arising: the mutually interactive creation of the organisation and its environment
• Accept the need for disruptive action
• Exercise what the poet John Keats called Negative Capability: the ability to be "in uncertainties, mysteries, doubts”

---

Darwin points out that it is simplistic to regard the clockwork and the snakepit as two contrasting alternatives. These can be complementary views, alternative ways of making sense of the same organization. Darwin regards either/or logic as limited, and uses fuzzy logic to argue that the clockwork, snakepit and rainforest metaphors all offer potential strengths for individuals at different times in organisations and all are limited in certain circumstances.

Darwin's characterization of three metaphors appears to play the same kind of categorical function in organizing organizational narratives as Dave Snowdon's Cynefin framework. I'm not saying they are equivalent, or that there any simple mapping between them, but merely that they are at the same logical level.

---

See my presentation on SenseMaking.

Behavioural Economics and Organizational Intelligence

@cbcurran and @mkrigsman link behavioral economics and project failure. Michael calls this the IT failure domino effect : bias leads to poor decision-making and culminates in failed projects.

They talk about three behavioural factors that may affect project failure.

• Decision shortcuts
• Irrational value assessment
• Emotional and social impacts

Behavioural economics challenges the conventional notion of economic rationality. So where does organizational intelligence fit into this debate? Does intelligence entail rationality? Let's look at each of these factors in turn.

Decision shortcuts

Chris lists three common short-cuts.

• Using the cost of something else similar for comparison instead of knowing the actual value of something
• Using “default” option as the best option
• Using rules of thumb rather than understanding actual need

But using these short-cuts is perfectly compatible with intelligence. Indeed, in some complex situations, it may be impossible to reach a rational conclusion without taking any short-cuts, in which case the only intelligent course of action may be to jump to conclusions. (The passage of time from "Time for Understanding" to "Moment to Conclude" is not amenable to rational logic - see my presentation on Mastering Time.)

As I see it, the problem is not the short-cuts themselves, but the uncritical over-reliance on these short-cuts, and in some cases an apparent lack of awareness that a short-cut is being taken at all. Intelligent use of short-cuts calls for feedback and experimentation and learning, so that a set of effective and context-appropriate short-cuts are refined over time.

Irrational value assessment

Behavioural economics points out that people value things differently according to the way the choice is framed. For example, people may be unduly impressed by "free" offers, or may have an asymmetrical preference for avoiding loss, in situations that mathematical economists perceive as equivalent. Choices are often driven by a highly subjective notion of risk and reward.

In an intelligent organization, these effects may be moderated by an ability to articulate and share notions of risk and reward between multiple decision-makers, not relying purely on dry economic measures but understanding how these measures apply to real practical issues.

Emotional and social impacts

One form of emotional influence explored by behavioural economists is the so-called ego trap, which includes over-attachment to past decisions as well as over-confidence. When a situation becomes adverse (whether an investment or a project), it is psychologically easier to hope that the situation will sort itself out (for example, the markets turn positive, or the project gets back on track) than to reverse the original decision.

In organizational settings, there is a high transaction cost in changing your mind, and people who abandon previous commitments too quickly tend to be distrusted as fickle and unreliable. Furthermore, most projects experience difficulties at some stage, and if people didn't have some ego investment to carry them through these difficulties, no projects would ever be completed. So it is often better to work through difficulties than to cancel a project at the first sign of trouble. On the other hand, there are always going to be some decisions that made sense at the time, but are no longer viable under the changed circumstances.

Sticking to some decisions and changing others - this is incredibly hard, and is a test of character as well as intelligence. An intelligent organization doesn't have to suppress ego, but it needs to have a way of dealing appropriately and authentically with these kinds of issue, not only so that the right decisions can be reached more often, but also so that ventures into the unknown can be taken with the confidence that the downside risk can be contained.

See also: From Organizational Stupidity to IT Disaster

Can we improve organizational intelligence?

There are several reasons why you might not wish to do anything about the intelligence of your organization.

One reason is that you think your organization is intelligent enough already. When asked if you experience any of the Symptoms of Organizational Stupidity, you deny it, or say it doesn't matter. (Denial happens to be one of the symptoms of stupidity by the way, but of course that doesn't prove that all instances of denial are evidence of stupidity.)

Another reason is that you or your bosses benefit personally from suppressing the intelligence of your organization. I have never met anyone who admits to this for themselves, but I have often heard people wondering if their bosses were really as stupid as they pretended to be.

The next possible reason is that you regard intelligence as a fixed and unalterable quality. After all, you can't turn a stupid person into an intelligent person, even if brain gym and fish oil and listening to Mozart might have a fractional advantage. So why would we think that organizations are any different from people?

But yes, organizations are different from people. As I've pointed out many times, there is no automatic correlation between the intelligence of an organization and the intelligence of the people: a stupid organization can be packed with brainy people who don't talk to each other. So improving the intelligence of an organization might be achieved by getting people to collaborate better.

Hold on, that looks like cultural change. And doesn't cultural change take a long time? Not necessarily. Think of it like gardening. Some things certainly take a long time to bear fruit: I planted a small cherry tree in the garden about ten year ago, and it's only in the last couple of years that we got a decent crop. But if you cut the grass and trim the hedges, you can see the benefits immediately. Gardening involves a lot of mini-projects, some short-term and some longer-term, and the outcomes are generally pretty uncertain, but a lot of people think it's still worth doing.

What gardening teaches us is that some things call for persistence and patience - you can't just go out on a sunny spring afternoon and fix things that have been neglected for years and expect them to stay fixed until next spring without further attention. It also teaches us the limitations of command and control - you can't simply command the plants to grow as if you were an Old Testament prophet. There are many aspects of management that share these characteristics of gardening. So "Urgh, CultureChange" doesn't seem much of an excuse for inaction.

Finally, we might note that organizations are investing in all sorts of initiatives and technologies that I can only make sense of by interpreting these as attempts to improve various aspects of organizational intelligence - better business intelligence, better decision-making, better knowledge sharing, and so on. Now I'd certainly caution against the fantasy that a stupid organization can magically become more intelligent by buying a load of expensive software, nor would I argue that other people spending money and management time on this stuff is a valid reason for you to spend anything. However, there seems to be some willingness to invest in this class of organizational benefit, possibly even in your own organization.

So perhaps I should ask a different question - not "Do you want to do anything about the intelligence of your organization?" but "Wouldn't the organizational intelligence lens help you achieve greater synergies from what you are already doing?"

Wilensky on Organizational Intelligence 3

Post partly based on Chapter 8 of Organizational Intelligence by Harold Wilensky, published in 1967, in which he summarizes his conclusions on organizational intelligence.

---

Intelligence failures are built into complex organizations

"On the one hand, the most readily accomplished revamping of structure turns out to be mere organizational tinkering."

Wilensky points out that reorganization often merely gives official recognition to practices that had already evolved informally - for example, an interdepartmental committee that institutionalizes information sharing and collaboration, or a public affirmation of fixed departmental positions. But that doesn't mean organizational innovation can always be dismissed as tinkering, merely that there is often a time-lag between the development of an innovation in the real organization and its being captured in the formal organization.

"On the other hand, even when the reorganization of formal structure is pushed to its limit, the basic sources of distortion remain in some degree."

Wilensky expresses scepticism about any singleminded approach to administrative reforms that facilitate the flow of accurate information, for the following reasons.

• the proper mastery of the task calls for specialization
• the need to motivate and control personnel necessitates hierarchy
• coordination demands centralization
• the exigencies of decision seem to require direct answers, if not short-run predictions of the future
• internal security and outside competition necessitate security - to the extent that other organizational interests must be protected

I agree with his conclusion that a singleminded approach is inappropriate, but I don't agree with all the reasons he cites. For example, although the book contains many criticisms of hierarchy and its constraints, he still expected organizations to be largely hierarchical in structure, and he could not have foreseen the radical alternatives to hierarchy that have emerged since this book was published.

"Many sources of intelligence failures are natural to the state of the organization's development and are therefore substantially beyond its control."

If we focus (as Wilensky does) on the formal organization, the fact that some individuals may bypass the regular machinery and construct informal intelligence mechanisms on their own initiative can be seen as an organizational intelligence failure. However, if we focus on the informal (de facto) organization, the ability of the organization to accommodate this kind of initiative is a positive indicator.

But reliance on heroes taking personal initiatives is not enough for real organizational intelligence. Wilensky's book contains many examples of organizations that have been led astray by false certainties, narrow thinking, and insufficiently rigorous and critical debate. His final paragraph (which it is not hard to project onto more recent events) is as follows.

"To read the history of modern intelligence failures is to get the nagging feeling that men at the top are often out of touch, that good intelligence is difficult to come by and enormously difficult to listen to; that big decisions are very delicate but not necessarily deliberative; that sustained good judgement is rare. Bemoaning the decline of meaningful action, T.S. Eliot once spoke of a world that ends 'not with a bang but a whimper'. What we have to fear is that the bang will come, preceded by the contemporary equivalent of the whimper - a faint rustle of paper as some self-convinced chief of state, reviewing a secret memo full of comfortable rationalizations just repeated at the final conference, fails to muster the necessary intelligence and wit and miscalculates the power and intent of his adversaries."

VPEC-T and pluralism

#vpect is a deceptively simple systems thinking lens developed by a couple of friends of mine (Carl Bate and Nigel Green), and described in their book Lost in Translation. The letters stand for Values, Policies, Events, Content and Trust. For a good brief description, see VPEC-T the 5D lens by @RoyGrubb.

---

VPEC-T is based on a profoundly radical philosophy of plurality. Instead of a single centralized value system (as found in top-down command-and-control organizations), we expect to find a range of different (overlapping, conflicting) value systems. Instead of a single coherent set of policies, we expect to find complex interaction between different kinds of policies (commercial, security, safety, corporate responsibility, and so on). Instead of a simple set of routine events, the post-modern organization is faced with a dynamic set of emerging events. Instead of a rigid set of database records, systems content is rich and evolving. And finally, the whole human activity system is underpinned by a complex set of trust relationships between people and organizations.

If you’ve spent any time working and thinking in business and systems, these statements will probably seem like common sense. However, many twentieth century systems methodologies were based on simplistic assumptions – for example, that there should be a single ordered set of goals and objectives, completely and consistently driving all system activity; that there should be a single ordered set of policies, rationally decided by management according to best-practice strategic principles, and so on.

And in our working lives and elsewhere we are surrounded by systems that have been designed according to these simplifying assumptions – systems that are inflexible, that serve a narrow set of stakeholders and to hell with everyone else. Not just software systems (for example, badly designed websites) but all sorts of human activity systems. Think of the hours of passenger time that are wasted because airline and airport procedures are designed for the convenience of their own operations rather than for the passengers. Think of the huge inefficiency of healthcare and other public services. Think of organizations whose attention is almost exclusively focused on internal squabbling rather than building external relationships, and which only survive because their competitors are no better. Think of counter-productive targets and expensive white elephants and all the other trappings of bureaucracy. These are the products of a certain way of thinking about business organizations and systems, which VPEC-T regards as obsolete and obscene.

So VPEC-T starts from the opposite assumption and expects trouble – there will be tension, there will be contradictions, and so on. Some people might think this is a pessimistic position. After all, wouldn’t it be wonderful if the world was as consistent and orderly and predictable as the 20th century methodologies assumed?

Actually I don’t think so – I think that kind of mechanical uniformity takes us towards the utopian nightmare described by countless science fiction writers. The plurality and richness which I find in VPEC-T is not only more realistic but also more human.

VPEC-T is not just a tool for making sense of complex human activity systems, but also a tool for communicating and telling stories about these systems. This is important for three reasons

• Firstly, because any single view of a complex system is necessarily unreliable and incomplete. We need multiple lenses and multiple voices to give us greater coverage of the issues, and greater confidence in the analysis.

• Secondly, because the philosophy of plurality makes it necessary for us to work collaboratively with other people, not only with different perspectives but with different value systems. So we need a language for collaboration.

• And thirdly, because of the reflexive nature of trust, which requires transparency of the models and assumptions on which the system is based. Ultimately, VPEC-T cannot be a private matter for a closed elite group of system designers without compromising trust principles.

---

See also POSIWID should be plural

VPEC-T for design

#vpect is a deceptively simple systems thinking lens developed by a couple of friends of mine (Carl Bate and Nigel Green), and described in their book Lost in Translation. The letters stand for Values, Policies, Events, Content and Trust. For a good brief description, see VPEC-T the 5D lens by @RoyGrubb.

---

Besides using the VPEC-T lens in a number of practical situations, I've also run a couple of Next Practice workshops featuring the VPEC-T lens. My own experience leans towards using VPEC-T as a diagnostic lens - for understanding problems, analysing requirements and identifying risks, and I have found that people accept this pretty readily. I have previously posted some examples of this kind of diagnosis on this blog (see VPEC-T category), and I've recently been using VPEC-T as a lens to help me build an Organizational Intelligence assessment and diagnostic tool.

In a systems thinking context, diagnostic tools are useful to appreciate the positive aspects of an existing or proposed system (why it works at all) as well as the negative aspects (why it doesn't work as well as it could/should). Therefore not just analysing problems but also checking that a proposed solution represents a real and effective improvement on the status quo. But a diagnostic tool alone cannot produce solutions, it merely helps you evaluate any given solution. In other words, it tells you what's going on, it doesn't tell you what to do, but it might tell you what not to do.

Which raises the question whether VPEC-T also works as a problem-solving and design lens. From my own experience as well as the examples I've looked at, I have always got the impression that the heavy lifting in design terms is done by the P-E-C elements (Policies, Events, Content), especially in an IS context, with V and T helping to frame the organizational change and adoption engineering.

This impression was confirmed by a tweet today from Nigel, who said he had been "doing a lot of P-E-C work recently but now the gears were shifting to the V and the T big time" (@taotwit). "While P-E-C can help shape design patterns the V and the T are required to get buy-in to them" (@taotwit). Chris Bird agreed: "I think you have to have your PEC thinking in place so you can illustrate VT during change project" (@seabird20).

VT is certainly important for adoption engineering, but we certainly shouldn't regard design and adoption as two separate questions. Does it make sense to talk about "solving" the V and T requirements as well as the PEC requirements? (For example, in designing a communication system joining two professional communities with distinct value systems and a history of mistrust, what are the sociotechnical design patterns that can support healthy and productive collaboration across the trust divide.)

VPEC-T practitioners may set out with the intention of paying attention to all five dimensions in a balanced way throughout a systems intervention, but this is not easy. As Nigel says, "Believing that VT are critical to maintain throughout a change programme is vital (@taotwit) but getting folk to keep it high on their agenda when they get focused on PEC in say, a design cycle, is a big challenge! (@taotwit)" And conversely, "sometimes the PEC messages can get swamped by VT debates" (@taotwit).

As a diagnostic lens, VPEC-T has been used to help understand a broad range of organizational and social questions. However, all of the examples I've seen of its use as a problem-solving and design lens have been within the IS domain. So I shall be interested to explore the use of VPEC-T as a problem-solving lens in other domains.