Ethics committee raises alarm

Dr @BenGoldacre was the keynote speaker at an IT conference I attended recently. In the context of the growing interest in technology ethics, especially AI ethics, I asked him what IT could learn from medical ethics. He responded by criticising the role of the ethics committee, and mentioned a recent case in which an ethics committee had blocked an initiative that could have collected useful data concerning the effectiveness of statins. This is an example of what Goldacre calls the ethical paradox. As he wrote in 2008,

"You can do something as part of a treatment program, entirely on a whim, and nobody will interfere, as long as it’s not potty (and even then you’ll probably be alright). But the moment you do the exact same thing as part of a research program, trying to see if it actually works or not, adding to the sum total of human knowledge, and helping to save the lives of people you’ll never meet, suddenly a whole bunch of people want to stuck their beaks in."

Within IT, there is considerable controversy about the role of the ethics committee, especially after Google appointed and then disbanded its Ethics Board. In a recent article for Slate, @internetdaniel complains about company ethics boards offering "advice" rather than meaningful oversight, and calls this ethics theatre. @ruchowdh prefers to call it ethics washing.

So I was particularly interested to find a practical example of an ethics committee in action in this morning's Guardian. While the outcome of this case is not yet clear, there seem to be some positive indicators in @sloumarsh's report.

Firstly, the topic (predictive policing) is clearly an important and difficult one. It is not just about applying a simplistic set of ethics principles, but balancing a conflicting set of interests and concerns. (As @oscwilliams reports, this topic has already got the attention of the Information Commissioner's Office.)

Secondly, the discussion is in the open, and the organization is making the right noises. “This is an important area of work, that is why it is right that it is properly scrutinised and those details are made public.” (This contrasts with some of the bad examples of medical ethics cited by Goldacre.)

Thirdly, the ethics committee is (informally) supported by a respected external body (Liberty), which adds weight to its concerns, and has helped bring the case to public attention. (Credit @Hannah_Couchman)

Fourthly, although the ethics committee mandate only applies to a single police force (West Midlands), its findings are likely to be relevant to other police forces across the UK. For those forces that do not have a properly established governance process of their own, the default path may be to follow the West Midlands example.


So it is possible (although not guaranteed) that this particular case may produce a reasonable outcome, with a valuable contribution from the ethics committee and its external supporters. But it is worrying if this is what it takes for governance to work, because this happy combination of positive indicators will not be present in most other cases.

---

Ben Goldacre, Where’s your ethics committee now, science boy? (Bad Science Blog,23 February 2008), When Ethics Committees Kill (Bad Science Blog, 26 March 2011), Taking transparency beyond results: ethics committees must work in the open (Bad Science Blog, 23 September 2016)

Sarah Marsh, Ethics committee raises alarm over 'predictive policing' tool (The Guardian, 20 April 2019)

Daniel Susser, Ethics Alone Can’t Fix Big Tech (Slate, 17 April 2019)

Jane Wakefield, Google's ethics board shut down (BBC News, 5 April 2019)

Oscar Williams, Some of the UK’s biggest police forces are using algorithms to predict crime (New Statesman, 4 February 2019)

National Decision Model and Lessons Learned

The appointment of Cressida Dick as the first female commissioner of the Metropolitan Police has been criticized in some quarters because of her involvement in the fatal shooting of Jean Charles de Menezes in 2005.

Dick was the "gold commander" who instructed armed officers to "stop" de Menezes. At the time, however, armed officers were following a new set of police guidelines known as Operation Kratos. In the context of these guidelines, Dick's orders were interpreted as shoot-to-kill. At the Old Bailey in 2007, Dick denied that this had been her intention.

As Mary Dejevsky argues, the de Menezes case provides a lasting reminder of what can go wrong

"whether because the overall atmosphere has not been properly appraised, because the orders given were not precise enough, or simply because insufficient account has been taken of the human factor".

The National Decision Model, which was introduced a few years after this incident, provides a framework that should (at least in theory) prevent this kind of miscommunication. See my post on the National Decision Model (May 2014). Perhaps this is one of the areas where "lessons have been learned". Or perhaps not.

Iain Gould is a solicitor. One of his clients was involved in an incident in 2013 that resulted in his being tasered. The Independent Police Complaints Commission (IPCC) attributed this escalation, in part, to a failure to follow the National Decision Model.

"I would question whether PC B gave enough emphasis to the first element of the National Decision Model, which is to communicate. ... More effort should have been made, in line with the National Decision Model, to engage Mr S in dialogue."

The IPCC commissioned a report in 2015, which contains some analysis of the National Decision Model, and some recommendations for its improved use. There are two versions of the report:

• Police Use of Force (IPCC version)
• Police Use of Force (TNS-BMRB version)
• both accessed via IPCC webpage on Police Use of Force (retrieved 27 February 2017). 

---

The Guardian view on the Met police: changing, but too slowly (23 February 2017)

Duncan Campbell et al, Leaks raise sharp questions about police tactics (Guardian, 17 August 2005)

Mary Dejevsky, Can Cressida Dick win over the public? Yes, if she’s learned from her mistakes (Guardian, 23 February 2017)

Iain Gould, Is Police Taser Policy Working? (11 May 2016)

Martin Hoscik, Sadiq Khan says ‘My heart goes out to the de Menezes family’ but insists Cressida Dick is the right choice to protect London (MayorWatch, 25 February 25, 2017)

Maxwell Mclean, The Coroner in England and Wales; Coronial Decision-­Making and Local Variation in Case Outcomes (Doctoral Thesis, University of Huddersfield, 2015)

Wail Qasim, Lessons Learned (LRB Blog, 27 February 2017)

---

Related blogpost

• National Decision Model (May 2014)

National Decision Model

@antlerboy asks does anyone know theoretical underpinning of the (rather good) police decision-making model?

The National Decision Model (NDM) is a risk assessment framework, or decision making process, that is used by police forces across the UK. It replaces the former Conflict Management Model. Some sources refer to it as the National Decision-Making Model.

Looking around the Internet, I have found two kinds of description of the model - top-down and bottom-up.

The top-down (abstract) description was published by the Association of Chief Police Officers (ACPO) sometime in 2012, and has been replicated in various items of police training material including a page on the College of Policing website. It is fairly abstract, and provides five different stages that officers can follow when making any type of decision - not just conflict management.

Some early responses from the police force regarded the NDM as an idealized model, only weakly connected to the practical reality of decision-making on the ground. See for example The NDM and decision making – what’s the reality? (Inspector Juliet Bravo, April 2012).

In contrast, the bottom-up (context-specific) description emerges when serving police officers discuss using the NDM. According to Mr Google, this discussion tends to focus on one decision in particular - to Taser or not to Taser.

"For me the Taser is a very important link in the National Decision Making Model . It bridges that gap between the baton and the normal firearm that has an almost certain risk of death when used." (The Peel Blog, July 2012). See also Use of Force - Decision Making (Police Geek, July 2012).

ACPO itself adopts this context-specific perspective in its Questions and Answers on Taser (February 2012, updated July 2013), where it is stated that Taser may be deployed and used as one of a number of tactical options only after application of the National Decision Model (NDM).

Of course, the fact that Taser-related decisions have a high Google ranking doesn't imply that these decisions represent the most active use of the National Decision Model. The most we can infer from this is that these are the decisions that police and others are most interested in discussing. (See footnote)

In a conflict situation, a police officer often has to decide how much force to use. The officer needs to have a range of tools at his disposal and the ability to select the appropriate tool - in policing, this is known as a use-of-force continuum. More generally, it is an application of the principle of Requisite Variety.

In a particular conflict situation, the police can only use the tools they have at their disposal. The decision to use a Taser can only be taken if the police have the Taser and the training to use it properly. In which case the operational decision must follow the NDM.

More strategic decisions operate on a longer timescale - whether to equip police with certain equipment and training, what rules of engagement to apply, and so on. A truly abstract decision-making model would provide guidance for these strategic decisions as well as the operational decisions.

And that's exactly what the top-down description of NDM asserts. "It can be applied to spontaneous incidents or planned operations by an individual or team of people, and to both operational and non-operational situations."

Senior police officers have described the use of the NDM for non-conflict situations. For example, Adrian Lee (Chief Constable of Northants) gave a presentation on the Implications of NDM for Roads Policing (January 2012).

The NDM has also been adapted for use in other organizations. For example, Paul Macfarlane (ex Strathclyde Police) has used the NDM to produced a model aimed at Business Continuity and Risk Management. which he calls Defensive Decision-Making.

---

How does the NDM relate to other decision-making models? According to Adrian Lee's presentation, the NDM is based on three earlier models:

• The Conflict Management Model (CMM). For a discussion from 2011, see Police Oracle.
• The SARA model (Scan, Analyze, Respond, Assess) - which appears to be similar to the OODA loop.
• Something called the PLANE model. (I tried Googling this, and I just got lots of Lego kits. If anyone has a link, please send.)

There is considerable discussion in the USA about the relevance of the OODA loop to policing, and this again focuses on conflict management situations (the "Active Shooter"). There are two important differences between combat (the canonical use of OODA) and conflict management. Firstly, the preferred outcome is not to kill the offender but to disarm him (either physically or psychologically). This means that you sometimes need to give the offender time to calm down, orienting himself into making the right decision.

And the cop needs to stay calm. George “Doc” Thompson, who taught US police a de-escalation technique known as Verbal Judo, once said "We know that the most deadly weapon we carry is not the .45 or the 9mm, it is in fact the cop’s tongue ... A single sentence fired off at the wrong person at the wrong time can get you fired, it can get you sued, it can get you killed."

So it's not just about having a faster OODA loop than the other guy (although clearly some American cops think this is important). And secondly, there is a lot of talk about situation awareness and anticipation. For example, Dr. Mike Asken, who is a State Police psychologist, has developed a model called AAADA (Anticipating, Alerting, Assessing, Deciding and Acting). There is also a Cognitive OODA model I need to look into.

However, I interpret @antlerboy's request for theoretical underpinning as not just a historical question (what theories of decision-making were the creators of NDM consciously following) but a methodological question (what theories of decision-making would be relevant to NDM and any other decision models). But this post is already long enough, and the sun is shining outside, so I shall return to this topic another day.

 

---

Footnote

I referred above to the decisions that police and others are most interested in discussing. 

Argyris and Schön introduced the distinction between Espoused Theory and Theory-In-Use. Perhaps we need a third category to refer to what people imagine to be the central or canonical examples of the theory. We might call it Theory-in-View or Theory-in-Gaze.

This is not quite the same as Prototype Theory, which determines membership of a given category in terms of the similarity to the canonical example.

Update 2022: Reading Annemarie Mol has introduced me to Lolle Nauta's concept of exemplary situation. So perhaps that's the term I should use?

 

---

Sources and Links

Michael J. Asken, From OODA to AAADA - A cycle for surviving violent police encounters (Dec 2010)

Erik P. Blasch et al, User Information Fusion Decision-Making Analysis with the C-OODA Model (Jan 2011)

Tom Dart, ‘Verbal judo’: the police tactic that teaches cops to talk before they shoot (Guardian 21 July 2016)

Adrian Lee, Implications of NDM for Roads Policing (January 2012).

Steve Papenfuhs, The OODA loop, reaction time, and decision making (PoliceOne, 23 February 2012)

National Decision Model (ACPO, 2012?)

National Decision Model (College of Policing, 2013)

SARA model (Center for Problem-Oriented Policing)


Related Posts: National Decision Model and Lessons Learned (Feb 2017), Decision-Making Models (March 2017)

Four Types of Trust

This post is partially based on the book Trust and Mistrust by Aidan and John, who are occasional contributors to this blog.

---

Trust is a property of a system or relationship based on expectations of reasonable and fair behaviour. Trust and mistrust are commonly regarded as complementary opposites.


Trust

• Trust is commonly regarded as a positive good - the lubricant of business relationships. Like oil, it is slippery and difficult to grasp. 

• We define trust as a property of a system or relationship based on expectations of reasonable and fair behaviour. In some situations, trust is regarded as simply the absence of mistrust. 

• We generally trust people and situations, unless we have some specific reason to distrust them. However, this may sometimes be insufficient basis for positive trust.

Mistrust

• There is a clear difference between a simple absence of trust, and positive mistrust. Mistrust is supposedly a consequence of past actions. In fact, it is often a consequence of a complex set of beliefs, perceptions, associations and interpretations. 

• Mistrust is usually more difficult to deal with than simple absence of trust. How do I deal with other people's mistrust of me, whether this is fairly deserved or not? Do I tackle false beliefs head-on, or do I try to dissociate myself from the events that triggered the mistrust, or do I simply switch my identity and reappear under a new guise?

Practical Questions

• How can we assess the level of trust in a given situation?
• How can we increase the level of trust? 
• How should we deal with people, companies and situations we mistrust? 
• How can we respond to the declared or inferred mistrust of others? 

Four types of trust (and mistrust)

Centralized

• Authority (Top Down) Trust - Trust is guaranteed by reference to some authority, and is typically configured hierarchically. This mode of trust is found in a range of centralized identity management (authorization/authentication) including Passport, Verisign and Identrus. 

• Commodity (Contract) Trust - Trust is supplied as a product or service. It carries a price and expected service levels.  

Decentralized

• Network (Web) Trust - Trust is conferred by the collective action of a distributed network. The internet is trusted to do certain things because there is no single point of failure. Competitive markets are trusted to set fair prices.

• Relationship (Authentic) Trust - Trust is inherent in an authentic and committed engagement with business partners. 

Police force example

A police force is an organization that generally requires high levels of trust. All four types of trust are relevant to a police force.

• Authority (Top Down) Trust - For many citizens, the police force both symbolizes and realises a form of social authority. This authority is reinforced by police procedure, and by wearing the uniform. Members of certain communities or subcultures sometimes have a hostile attitude to this authority. Defence lawyers may attack police procedure in order to undermine trust in police evidence. 

• Commodity (Contract) Trust - Trust in the quality of service provided by the police. Service targets - crime rates, crime clear-up rates, speed of response. These are primary political concerns.  

• Network (Web) Trust - The police do not operate in isolation but in collaboration with other bodies such as social services, health, education and the judiciary, as well as community groups. Trust in the police force depends on complex institutional and often semi-formal arrangements with these bodies, and may be affected by problems elsewhere in this network. 

• Relationship (Authentic) Trust - Trust created by personal relationships between individual police officers and the community.

---

Aidan Ward and John Smith, Trust and Mistrust (Wiley 2003)


Originally posted at http://www.users.globalnet.co.uk/~rxv/trust/index.htm