Open Democracy in the Age of AI

An interesting talk by Professor Hélène @Landemore at @TORCHOxford yesterday, exploring the possibility that some forms of artificial intelligence might assist democracy. I haven't yet read her latest book, which is on Open Democracy.

There are various organizations around the world that promote various notions of Open Democracy, including openDemocracy in the UK, and the Coalition for Open Democracy in New Hampshire, USA. As far as I can see, her book is not specifically aligned with the agenda of these organizations.

Political scientists often like to think of democracy in terms of decision-making. For example, the Stanford Encyclopedia of Philosophy defines democracy as a method of group decision making characterized by a kind of equality among the participants at an essential stage of the collective decision making, and goes on to discuss various forms of this including direct participation in collective deliberation, as well as indirect participation via elected representatives.

At times in her talk yesterday, Professor Landemore's exploration of AI sounded as if democracy might operate as a massive multiplayer online game (MMOG). She talked about the opportunities for using AI to improve public consultation, saying my sense is that there is a real potential for AI to basically offer us a better picture of who we are and where we stand on issues. 

When people talk about decision-making in relation to artificial intelligence, they generally conform to a technocratic notion of decision-making that was articulated by Herbert Simon, and remains dominant within the AI world. When people talk about the impressive achievements of machine learning, such as medical diagnosis, this also fits this technocratic paradigm.

However, the limitations of this notion of decision-making become apparent when we compare it with Sir Geoffrey Vickers' notion of judgement in human systems, which contains two important elements that are missing from the Simon model - sensemaking (which Vickers called appreciation) and ethical/moral judgement. The importance of the moral element was stressed by Professor Andrew Briggs in his reply to Professor Landemore.

Although a computer can't make moral judgements, it might perhaps be able to infer our collective moral stance on various issues from our statements and behaviours. That of course still leaves a question of political agency - if a computer thinks I am in favour of some action, does that make me accountable for the consequences of that action?

In my own work on collective intelligence, I have always regarded decision-making and policy-making as important but not the whole story. Intelligence also includes observation (knowing what to look for), sensemaking and interpretation, and most importantly learning from experience.

Similarly, I should regard democracy as broader than decision-making alone, needing to include the question of governance. How can the People observe and make sense of what is going on, how can the People intervene when things are not going in accordance with collective values and aspirations, and how can Society make progressive improvements over time. Thus openDemocracy talks about accountability. There are also questions of reverse surveillance - how to watch those who watch over us. And maybe openness is not just about open participation but also about open-mindedness. Jane Mansbridge talks about being open to transformation.

There may be a role for AI in supporting some of these questions - but I don't know if I'd trust it to.

---

Ethics in AI Live Event: Open Democracy in the Age of AI (TORCH Oxford, 13 November 2020) via YouTube

Nathan Heller, Politics without Politicians (New Yorker, 19 February 2020)

Hélène Landemore, Open Democracy: Reinventing Popular Rule for the 21st Century (Princeton University Press 2020)

Jane Mansbridge et al, The Place of Self-Interest and the Role of Power in Deliberative Democracy (The Journal of Political Philosophy:Volume 18, Number 1, 2010) pp. 64–100

Richard Veryard, Building Organizational Intelligence (LeanPub 2012)

Geoffrey Vickers, The Art of Judgment: A Study in Policy-Making (Sage 1965), Human Systems are Different (Paul Chapman 1983)

Stanford Encyclopedia of Philosophy: Democracy

Working with stupid

@jasoncrawford makes some excellent points in his post How to work with “stupid” people.

• Calling someone stupid is often based on the fact that the person has made (what seems to be) a bad decision.

• Just because you disagree with a decision doesn't make it bad. Just because a decision doesn't produce the desired outcome doesn't make it bad. And apparently bad decisions may not be so bad when you frame them differently, appreciating the context and constraints.

• Bad decisions may be caused by bad information or bad technique. Just because a person is intelligent doesn't prevent them making bad decisions when misinformed, or when skipping some of the validation and consultation steps.

• And bad decisions may also be caused by timidity or emotional pressure. Just because a person is intelligent doesn't stop them being intimidated by other people.

• Regarding someone as stupid (even if they are) is not a helpful starting point for understanding the causes of a difference of opinion, let alone resolving it.

---

Just as it is risky to draw conclusions about a person's stupidity from a single decision, so it is also risky to draw conclusions about a person's intelligence from a decision that had a lucky outcome. Labelling people as intelligent can be almost as damaging as labelling them as stupid - it distorts future decision-making and in some circumstances has been shown to discourage hard work. (See my post Explaining Enron.)

Of course we do often come to conclusions about the intelligence and character of our colleagues and partners, and this may be based not on a single lucky or unlucky decision but on a repeated pattern of decision and action. But what that really tells us is how intelligently that person is able to operate within a particular organizational context. Does the person have the information and tools and appropriate management support, and the right kind of motivation and pressure, to think about the right issues in a sufficiently rigorous manner? One of the symptoms of a stupid organization is that nobody appears able to make intelligent decisions. And one of the strongest arguments for improving the intelligence of an organization is to release the latent intelligence of its people, which must surely be of benefit to them as individuals as well as to the organization as a whole.

---

So what about working with stupid organizations? People inside organizations may perceive many of the symptoms of organizational stupidity, may despair of a culture that inhibits open discussion and learning, and may seek to remedy this in various ways. But what about relationships between organizations with different styles of decision-making? The goal is surely to build collaborations where the intelligence of the whole is greater than the sum of its parts. Which entails a clear understanding of the cognitive strengths and weaknesses of each organization, and the kinds of collaboration that may be effective.

The challenge for consultants, of course, is that it is the stupid organizations that may need our help most.

Ethics and Intelligence

@flowchainsensei (Bob Marshall) argues that All Executives are Unethical (pdf).

More precisely, he argues that it is unethical to believe things without proper evidence. (He is particularly interested in beliefs about product and software development, but the argument applies more generally.)

As far as I can see, there are three steps in this argument.

1. People are ethically responsible for their beliefs. (According to Bob, this was the basis for a controversial paper presented to the Metaphysical Society by William Kingdon Clifford in 1876.)

2. An unfounded belief is unethical.

3. A person who holds unfounded beliefs is unethical.


Let's look at step 1 first. This appears to entail an ethical obligation to subject one's beliefs to some kind of "due diligence". However, most of our beliefs are based, not on evidence that we have personally collected and analysed, but at least partly on evidence that has been filtered through other sources. We may have reasons to trust certain sources more than others, but if it is unethical to believe things without proper evidence, it would also surely be unethical to trust things without proper evidence. We may accept an ethical obligation to subject our beliefs to "due diligence", but this is normally a collective obligation rather than an individual obligation.

Step 2 asserts that any failure to ground beliefs in proper evidence is an ethical failure. People are rightly held accountable for failing to act in certain circumstances (for example failing to save someone from drowning), but ethical censure generally assumes both awareness (knowing that someone needed rescue) and capability (being able to swim). So the problem with Step 2 is that the more complex the beliefs are, the greater the intellectual power (intelligence) that is required to appreciate and thoroughly investigate these beliefs. If the management team isn't individually or collectively intelligent enough to understand what proper evidence would look like, then believing things without proper evidence is a consequence of insufficient intelligence.

Does being stupid count as an ethical failure? (Being deliberately or avoidably stupid might, but most instances of stupidity are not deliberate.) Appointing people and teams who don't have enough intelligence might be unethical, but only if the appointment was deliberate or avoidable, and so on along the responsibility chain until we can find someone who should have known better.

Step 3 assumes that we can categorize people as ethical or unethical based on incidence of ethical or unethical behaviour. Once we have a hard-and-fast concept of sin, then we can define a sinner as a person who has committed (and not yet purged) at least one sin. The trouble with this is that if we are all sinners, the category of "sinner" ceases to have much value except for the purposes of hellfire rhetoric. Labelling all executives as unethical (and why stop at executives, by the way) becomes merely a rhetorical gesture.

---

So where does this leave the virtues of diligence, responsibility and probity? Firstly, I hold that these are collective virtues - executives display moral character in a particular organizational setting, and we may not know how their ethics would stand up in a different setting.

Secondly, I think character and intelligence are distinct virtues. We should not automatically suppose that intelligent people are more ethical than less intelligent people, and therefore we should not define "ethical" to mean something that only very intelligent or highly educated people can comply with.

Thirdly, there is a widespread belief (especially among consultants) in the value of knowledge (although I don't know exactly what would count as proper evidence for this belief - if executives are unethical, I dread to think where this leaves consultants). If we define knowledge as justified true belief, then knowledge is degraded to the extent that it is unjustified or untrue, or for that matter disbelieved. If it is unethical to believe something without proper evidence, it may sometimes also be unethical to disbelieve something. Sometimes excessive scepticism shades into cynicism and negativity, and maybe this can be just as unethical as unjustified optimism.

 

---

Related posts: Intelligence and Governance (February 2013), Ethics and Uncertainty (March 2019)

Where is the intelligence?

When a system behaves intelligently, we may want to discover where the intelligence is located. In smart technical systems, we may imagine that there is some clever lump of software somewhere doing the smart bits. In intelligent organizations, we may imagine that there must be some clever people pulling the strings. So we may wish to take the system apart to discover how the system works.

But I prefer to see intelligence as a holistic property of the whole system, emerging from the interaction between all the parts of the system, rather than something that can be specifically located in certain components or subsystems.

Let's consider any team sport - baseball, cricket, or any kind of football. The top teams are successful not just because they have the most talented (and expensive) players, but because they respond quickly and fluently to the tactics of the opposing side, which involves all of the elements of organizational intelligence.

The team may have a captain, but in many sports this is merely an honorary position for the oldest or wealthiest member of the team and there may be no visible difference in behaviour between the captain and the other players. The collective intelligence of a good team is not just the eleven men on the field but also the man on the touchline in the suit; not just the 90 minutes on the field but the whole training regime that enables them to perform at that level. When our team plays another team, where is the "memory" located of the other team's recent performance? It may not be the players themselves who watch and analyse the TV footage; it may be the coaches who feed this knowledge into the pre-match training.

So does that mean the knowledge is somehow "owned" by the coaches (on behalf of the team as a whole) and then "transferred" to the players before the match? But that is a gross simplification - it makes it seem as if the players were merely blank slates for the coaching staff to write upon. All we can accurately say about this kind of knowledge is that it is derived from observation and interpretation of TV footage, contained somewhere in the whole-system, and somehow influences the behaviour of the whole-system. It's not about transferring knowledge from one subsystem to another, but about communication and collaboration that results in the whole-system using the knowledge effectively.

Describing intelligence in this way allows us to focus on a series of critical questions - how does the team (as embodied whole-system) use knowledge intelligently, what kinds of signal does it pay attention to, how does it learn from past experience, how are new tactics and counter-tactics developed, and so on.

Those primary questions raise some secondary questions about the kinds of organizational structure and team culture in which this kind of intelligence may flourish, as well as the tools, platforms and other mechanisms that may be helpful, but we must always be wary of simplistic means-ends thinking - implement this regime and install these practices, and these outcomes will magically ensue.

for VPEC-T, see comments below

Early Warning Signals

As @bmichelson reports, rockstar Dave Lee Roth used to demand a bowl of M&Ms in his dressing room, with all the brown ones removed. Why? Because he wanted to test the venue's attention to detail. If they couldn't even get the sweets right, how could he trust them to correctly install the complex wiring and lighting required for the gig? If the test failed, what that triggered wasn't a childish tantrum but righteous anger and a thorough test of everything else before the band would go on the stage. [Business Advice from Van Halen, Fast Company March 2010 via Elemental Links] As @bmichelson points out, this can be regarded as a form of instrumentation.

I have seen a similar trick described somewhere else, possibly in one of Mark McCormack's books. When discussing a major event with a large hotel, casually ask for orange juice for all participants, freshly squeezed on the premises that morning. If hotel management appears to regard this requirement as trivial, then this is a warning that it may not take the other requirements seriously either.



Obviously these tricks only work when they are secret. As soon as people realise that these tricks are being used as tests or predictors of performance or quality, then they will alter (distort) their behaviour accordingly. (I'm imagining a scene from a Spinal Tap remake in which a rival rock band sends its roadies to tip off the venue and disrupt proceedings, so that the M&Ms are perfect, but the wiring electrocutes the guitarist and wreaks the concert.)

So if you are going to use these kinds of trick as a shortcut to detect what's going on, you need four things.

1. to design a test that is a good indicator of incompetence, laziness or inattention
2. to behave congruently as if the test really mattered
3. to act appropriately when the test fails (e.g. test the wiring, look for an alternative venue)
4. to watch out for signs that the test has been rumbled

And if you suspect that these kinds of tricks are being used against you, to test your competence and attention to detail, then you are probably already paying enough attention to detail to be able to do an all-round competent job. Maybe next time they'll trust you a bit more, okay?

Puzzles and Mysteries

For my second post inspired by Malcolm Gladwell's latest book "What the Dog Saw, and Other Adventures", I want to turn to the chapter "Open Secrets" (originally published in the New Yorker, Jan 2007).

In his analysis of different kinds of intelligence, Gladwell picks up the distinction between puzzles and mysteries originally proposed by Gregory Treverton. (See Curtis Frye's review of his 2003 book Reshaping National Intelligence for an Age of Information. See also Risks and Riddles, published in the Smithsonian Magazine, June 2007. And see my post on Making Intelligence Relevant.)

A puzzle is characterized by having a definite answer, if we can only find it. A puzzle is difficult only because we don't have enough information. For example, Gladwell and Treverton classify the question "Where is Osama Bin Laden?" as a puzzle. If we knew exactly where Bin Laden was, then this would cease to be a puzzle at all. So the purpose of intelligence here is to get relevant pieces of information that help narrow down the field of search.

A mystery is characterized by ambiguity and uncertainty. For example, the question "What is Osama Bin Laden up to?". There is perhaps no shortage of information here, there may even be too much information, the difficulty is in interpreting the information correctly.

Gladwell uses the distinction to discuss the Enron case. He argues against the popular view that Enron management concealed their dealings, and points out that the information was freely available to those that took the trouble to wade through the documents. In contrast to the Watergate case, which was only revealed because an insider (the famous "Deep Throat") leaked information to Woodward and Bernstein, journalists investigating the Enron case simply downloaded the information they needed from the Enron website. Indeed, several years before Enron fell, a bunch of MBA students had carried out a pretty accurate analysis, based merely on the published accounts. In Gladwell's opinion, Enron therefore counts as a mystery rather than a puzzle.

This is consistent with an assertion made by Harold Wilensky in his 1967 book Organizational Intelligence (which Gladwell has cited elsewhere), that a sophisticated reporter working with open sources may achieve more than an agent working with top-secret information. Wilensky highlights the distorting effects a doctrine of secrecy can have on intelligence: one example in Wilensky's book concerns the possible consequences of an American invasion of Cuba, where reporters read the situation more accurately than the CIA experts.

Dishonest people can create puzzles simply by withholding important information. Enron executives went to prison because their conduct was judged dishonest. Gladwell agrees that Enron was reckless and incompetent, but defends the company and its executives against the charge of concealment. The state of Enron's finances was too complicated even for its own executives to understand; we might imagine that some of the executives consciously took advantage of this complexity, but we could equally imagine that this was a situation that Enron blundered into without any deliberate strategy. Hence the mystery,

Characterizing Enron as a mystery also goes some way to explaining why the auditors were useless in detecting the fraud - if it was indeed a fraud. The audit process is designed to spot errors and omissions in the financial accounts, which might indicate dodgy dealings somewhere in the organization. The audit process is not designed to spot excessive complexity or risk, and auditors do not generally practice the kind of ratio analysis that the MBA students used. (Auditing is therefore one of those "best practices" whose flaws are exposed by complexity.)

Another lens through which the Enron accounts could reasonably have been viewed was the taxation lens. The fact that Enron wasn't paying much corporation tax (in several years it paid no income tax at all) might have been seen as an important clue to its lack of real profitability. However, those who wanted to believe in Enron's profitability could easily convince themselves that the low level of tax payments represented clever tax avoidance - in other words, interpreting it as evidence of the smartness of the accountants and/or the stupidity of the tax authorities. (Thus the accountancy lens was used to discredit alternative lenses that might have revealed alternative truths.)

Another way of thinking about the difference between puzzles and mysteries is that puzzles are about people (deliberate conspiracies) while mysteries are about systems. As Gladwell tells the story, Enron wasn't about a handful of bad people misleading everyone else, it was about a system that led everyone astray. Working out a mystery is not a question of collecting more information, but about finding a frame or lens for systematic analysis, to make sense of the information we already have.

Does it make sense to divide intelligence problems into puzzles and mysteries, as Treverton and Gladwell do? I'm not convinced there is a simple either/or, but I'm not sure that's what Treverton and Gladwell are claiming anyway. I  think what is important here is not to identify which problems count as puzzles and which ones count as mysteries, but to acknowledge that at least some problems do count as mysteries in Treverton's terminology, and therefore we need an intelligence capability that helps us to make sense of too much information, and not rely solely on an intelligence capability that merely gathers more information in the hope of resolving something. Examples of this need can be found both in national security and in business.

In terms of organizational intelligence, this means achieving a good balance between two capabilities - the information gathering capability (Perception) and the analysis capability (Sense-Making) - and linking effectively into the remaining capabilities (Decision, Action, Learning). Sometimes merely collecting more information doesn't help solve the problem, especially if we don't have the capacity to interpret the information we already have, or if the new information merely provides an excuse for further procrastination.

(This is similar to the point some of us were discussing recently on Twitter: whether it is always good to produce more ideas, or whether it is sometimes possible to have too many ideas, especially if you don't have the capacity to use them effectively.)

Was Enron intelligent? Enron certainly did some clever and innovative things, but the spectacular failure of Enron suggests that there were some flaws in the thinking. Here are two suggestions. Firstly, a collective failure within Enron to appreciate the scale of its exposure to risk indicates a weakness in sense-making - an insufficiently robust way of seeing beyond the complexity of the accounts and understanding the true financial state of the company. And secondly, a collective refusal to learn from the voices of doubt coming from external critics. When a company is convinced by its own cleverness, this conviction can become a barrier to learning, and therefore a limitation of true intelligence.

---

Other examples of the puzzle/mystery dichotomy

• Bob Rosner (ABC News Feb 2007) suggests that puzzles are left-brain and mysteries are right-brain
• Larry Ferlazzo suggests that puzzles are data-driven and mysteries are data-informed
• Security blogger Gunnar Peterson suggests that Authorization is a Puzzle, Authentication is a Mystery
• The Moonrider blog looks at accidents versus collisions in Motorcycle Safety
  
• Brett Miller quotes Gary Kasparov on Life versus Chess: Uncertainty is far more challenging
• In the 1950s, Frederik Buytendijk described men as puzzles and women as mysteries (via Annemarie Mol).
  

Related posts: Connecting the Dots, Explaining Enron, Making Intelligence Relevant

A reasonable percentage (3)

One piece of intelligence was accurate.

A man described as Abu Musab al-Zarqawi's "spiritual adviser" inadvertently led US forces to the spot where the militant leader was finally located and killed, the US military says.

Major General William Caldwell said the operation to track down the most wanted man in Iraq was carried out over many weeks, before he was killed after two US air force F-16s bombed a house in a village north of Baghdad.

"The strike last night did not occur in a 24-hour period. It truly was a very long, painstaking deliberate exploitation of intelligence, information gathering, human sources, electronic, signal intelligence that was done over a period of time - many, many weeks," Gen Caldwell said on Thursday.

[BBC News]

---

One piece of intelligence was flawed.

Anti-terror police raided a house at Forest Gate last week after saying they received "specific intelligence" that a chemical device might be found there.

Scotland Yard later said they had "no choice" but to act while the prime minister said it was essential officers took action if they received "reasonable" intelligence suggesting a terror attack.

Tony Blair said he backed the police and security services 101% and he refused to be drawn on suggestions that the armed operation had been a failure.

[BBC News]

---

It's a reasonable percentage. (Previous posts: April 9th, April 18th.)

But that's part of the problem with intelligence - it delivers probability rather than certainty. Perhaps the outcomes are the right way around this time - the presumed-guilty man was killed, and the presumed-innocent man merely injured. (So we shouldn't complain, should we? Imagine the complaints if it had been the other way around!)

But over the long run, are there too many errors? (Difficult to tell, as we only know of some of the better publicized successes and failures.) Should we be uneasy about the errors of intelligence, and the consequences of acting upon erroneous intelligence? There are fundamental questions here about the relationship between knowledge (or ignorance) and action (or inaction).

Intelligence or Fear?

Bruce Schneier has drawn some security lessons from a recent scare on a flight from Sidney to Los Angeles. Shortly after take-off, the air crew found an airsickness bag in the toilet, interpreted the word "BOB" scribbled on the bag as a bomb threat, and decided to turn the plane back. Schneier's account was published in the Sidney Morning Herald and reprinted in his CryptoGram newsletter (August 2004).
We can analyse this incident in terms of organizational intelligence, as follows.

perception monitoring	A member of the crew found the bag and noticed the word "BOB"	Good observation
communication	This finding was shared with the pilot and other crew	Enables collective response
appreciation sensemaking	The word "BOB" was interpreted as "Bomb on Board"	Schneier points out that there are several more likely interpretations.
reasoning action	The captain decided the risk was serious enough to turn the plane around and land back in Sydney.	Schneier writes: "Even a moment's reflection is enough to realize that this is an extreme overreaction to a nonexistent threat."
learning	The Australian transport minister blamed the person who wrote on the bag, calling him "irresponsible at the least and horrendously selfish and stupid at the worst".	Schneier writes: "Irresponsible for what? For writing his name? For perpetuating common flight-attendant slang? It wasn't the writer who did anything wrong; it was those who reacted to the writing."
	Prime Minister John Howard praised the crew for their quick reactions, diligence, and observation skills.	Schneier writes: "I'm sorry, but I see no evidence of any of that. All I see are people who have been thrust into an important security role reacting from fear, because they have not been properly trained in how to sensibly evaluate security situations: the risks, the countermeasures, and the trade-offs."
knowledge memory	Now every wacko in the world knows that all he needs to do to ground an international flight is to write "BOB" on an airsickness bag.	Schneier writes: "Somehow, I don't think that's the outcome any of us wanted."

Further comments and examples in Schneier's blog (October 7, 2004).

Demise of the Super Star

Military thinkers are moving away from reliance on individual expertise, and developing strategies that depend on collective expertise and organizational intelligence.

In the organizational equivalent of natural selection, leaders in industry and senior commanders in the military have long been selected for their superior intuition. ... Experts use intuition rather than rational decisionmaking to make sense of situations. These experts discern patterns and relate these patterns to their prior experience and/or knowledge to determine the nature of the situation and the appropriate response. ...

It is becoming increasingly clear that the complexity of the situations faced and the responses needed have outpaced not only decision theoretic approaches, but have also outpaced the ability of even the best of experts (super stars) to deal with the complexities involved.

• First, the sources of complexity are accelerating. These sources of complexity include the variety of events and entities that are connected, the density of the interactions, and the speed of interactions that make it difficult to relate a cause to an effect and almost impossible to predict cascading effects.
• Second, it takes a long time for individuals to become experts and senior decisionmakers in industry and the military, spending decades to arrive in positions of leadership. This means that the bulk of their experience is well aged, increasingly out-of-date, and of questionable relevance.

Intuition is not only unlikely to help (in the face of growing complexity and volatility), but may often be misleading. Rather than rely on individual genius, Information Age processes tap collective knowledge and collaboration.

Alberts & Hayes, Power to the Edge, pp 88-89

Related posts: Power to the Edge, Explaining Enron

Bearing Limit and WMD

From the Butler Report on WMD:

"... more weight was placed on the intelligence than it could bear ..."

In our risk management practice, we have for many years been using the term "bearing limit" to refer to the capacity of a unit to contain risk or uncertainty. (Among other things, the term is a punning reference to the downfall of Barings Bank, brought down by the activities of Nic Leeson, a so-called rogue trader who exceeded his bearing limit.)

The example of WMD intelligence now extends this notion of bearing limit to cover knowledge and intelligence.

---

The term "risk-bearing limit" is used in a Sept 2001 paper Reflections on New Financial System in Japan (pdf)

A notion of bearing limit is widely found in mechanical engineering, although it can also be found in horticultural systems. (Fruit Thinning)

Systems of intelligence

29th April, 1999. John Searle appears on the radio to discuss his recent New York Review article.

Old questions: "Are machines more intelligent than people? Can computers have self-consciousness and freewill?"

Some people might ask: How intelligent are people anyway? Can people have self-consciousness and freewill? (Common sense notions of consciousness and freewill are undermined by both psychoanalysis and hypnotism.)

I'd prefer to ask a different set of questions. What kinds of system can have such (emergent) properties as intelligence, self-consciousness and freewill?

People and computers alike display intelligent behaviour in some contexts, and not in others. People and computers alike depend on a complex support network. A person's ability to solve certain puzzles depends on various cultural factors. A computer's ability to beat a Grandmaster at chess depends on a team of chess experts and skilled programmers.

People and computers seem increasingly disembodied, fragmented.

A person is not just a mass of organic material, but also a mass of characteristic ideas, thoughts and feelings, expressed in words or acted out, distributed across diaries and letters, or captured in the memories and interpretations of other people. My name is held on countless databases, with various fragments of information about me, and embossed on several pieces of plastic card. No doubt much of this information is incorrect, incomplete or out-of-date.

What appears to be a self-contained computer may be merely a façade, providing access to a distributed network of other machines and systems.

Children of competitive middle-class parents are increasingly being subjected to additional training, including IQ coaching, in order to get high scores on various tests - perhaps to win places and scholarships to elite schools, or perhaps simply because it is thought to be a worthy activity in its own right. But what is being tested here, what do the tests really reveal? - the brain-power of the child, the skill of the coach, or the enthusiasm and resources of the parents?

What appears to be a self-contained child may be merely a test-scoring system. But if we reduce our children to test-scoring systems, if we reduce our schools to test-scoring-system improvement systems, where's the intelligence in that?