In fiction, servants cough discreetly to make people aware of their presence. (I'm thinking of P.G. Wodehouse, but there must be other examples.)
Technological devices sometimes call our attention to themselves for various reasons. John Ehrenfeld calls this presencing. The device goes from available (ready-to-hand) to conspicuous (visible).
In many cases this is seen as a malfunction, when the device fails to provide the expected commodity (obstinate) and thereby interrupts our intended action (obstructive).
However, in some cases the presencing is part of the design - the device nudging us into some kind of conscious engagement (or even what Borgmann calls focal practice).
Ehrenfeld's example is the two-button toilet flush, which allows the user to select more or less water. He sees this as "lending an ethical context to the task at hand" (p155) - thus the user is not only choosing the quantity of water but also being mindful of the environmental impact of this choice. Even if this mindfulness may diminish with familiarity, "the ethical nature of the task has become completely intertwined with the more practical aspects of the process". In other words, the environmentally friendly path has become routine (normalized).
Of course, people who are really mindful of the environmental or financial impact of wasting water may sometimes choose not to flush at all (following the slogan “If it’s yellow, let it mellow; if it’s brown, flush it down”) or perhaps to wee behind a tree in the garden rather than use the toilet. It is quite possible that the two button flush might nudge a few more people to think this way.
So sometimes a little gentle obstinacy on the part of our technological devices may be a good thing.
Albert Borgmann, Technology and the Character of Contemporary Life (Chicago, 1984)
John Ehrenfeld, Sustainability by Design (Yale, 2008)
Showing posts with label nudge. Show all posts
Showing posts with label nudge. Show all posts
Monday, September 23, 2019
Sunday, May 19, 2019
The Nudge as a Speech Act
As I said in my previous post, I don't think we can start to think about the ethics of technology nudges without recognizing the complexity of real-world nudges. So in this post, I shall look at how nudges are communicated in the real world, before considering what their artificial analogues might look like.
Once upon a time, nudges were physical rather than verbal - a push on the shoulder perhaps, or a dig in the ribs with an elbow. The meaning was elliptical and depended almost entirely on context. "Nudge nudge, wink wink", as Monty Python used to say.
Even technologically mediated nudges can sometimes be physical, or what we should probably call haptic. For example, the fitness band that vibrates when it thinks you have been sitting for too long.
But many of the acts we now think of as nudges are delivered verbally, as some kind of speech act. But which kind?
The most obvious kind of nudge is a direct suggestion, which may take the form of a weak command. ("Try and eat a little now.") But nudges can also take other illocutionary forms, including questions ("Don't you think the sun is very hot here?") and statements / predictions ("You will find that new nose of yours very useful to spank people with.").
(Readers familiar with Kipling may recognize my examples as the nudges given by the Bi-Coloured-Python-Rock-Snake to the Elephant's Child.)
The force of a suggestion may depend on context and tone of voice. (A more systematic analysis of what philosophers call illocutionary force can be found in the Stanford Encyclopedia of Philosophy, based on Searle and Vanderveken 1985.)
@tonyjoyce raises a good point about tone of voice in electronic messages. Traditionally robots don't do tone of voice, and when a human being talks in a boring monotone we may describe their speech as robotic. But I can't see any reason why robots couldn't be programmed with more varied speech patterns, including tonality, if their designers saw the value of this.
Meanwhile, we already get some differentation from electronic communications. For example I should expect an electronic announcement to "LEAVE THE BUILDING IMMEDIATELY" to have a tone of voice that conveys urgency, and we might think it is inappropriate or even unethical to use the same tone of voice for selling candy. We might put this together with other attention-seeking devices, such as flashing red text. The people who design clickbait clearly understand illocutionary force (even if they aren't familiar with the term).
A speech act can also gain force by being associated with action. If I promise to donate money to a given charity, this may nudge other people to do the same; but if they see me actually putting the money in the tin, the nudge might be much stronger. But then the nudge might be just as strong if I just put the money in the tin without saying anything, as long as everyone sees me do it. The important point is that some communication takes place, whether verbal or non-verbal, and this returns us to something closer to the original concept of nudge.
From an ethical point of view, there are particular concerns about unobtrusive or subliminal nudges. Yeung has introduced the concept of the Hypernudge, which combines three qualities: nimble, unobtrusive and highly potent. I share her concerns about this combination, but I think it is helpful to deal with these three qualities separately, before looking at the additional problems that may arise when they are combined.
Proponents of the nudge sometimes try to distinguish between unobtrusive (acceptable) and subliminal (unacceptable), but this distinction may be hard to sustain, and many people quote Luc Bovens' observation that nudges "typically work better in the dark". See also Baldwin.
I'm sure there's more to say on this topic, so I may update this post later. Relevant comments always welcome.
Robert Baldwin, From regulation to behaviour change: giving nudge the third degree (The Modern Law Review 77/6, 2014) pp 831-857
Luc Bovens, The Ethics of Nudge. In Mats J. Hansson and Till Grüne-Yanoff (eds.), Preference Change: Approaches from Philosophy, Economics and Psychology. (Berlin: Springer, 2008) pp. 207-20
John Danaher, Algocracy as Hypernudging: A New Way to Understand the Threat of Algocracy (Institute for Ethics and Emerging Technologies, 17 January 2017)
J. Searle and D. Vanderveken, Foundations of Illocutionary Logic (Cambridge: Cambridge University Press, 1985)
Karen Yeung, ‘Hypernudge’: Big Data as a Mode of Regulation by Design (Information, Communication and Society (2016) 1,19; TLI Think! Paper 28/2016)
Stanford Encyclopedia of Philosophy: Speech Acts
Related posts: On the Ethics of Technologically Mediated Nudge (May 2019), Nudge Technology (July 2019)
Updated 28 May 2019. Many thanks to @tonyjoyce
Once upon a time, nudges were physical rather than verbal - a push on the shoulder perhaps, or a dig in the ribs with an elbow. The meaning was elliptical and depended almost entirely on context. "Nudge nudge, wink wink", as Monty Python used to say.
Even technologically mediated nudges can sometimes be physical, or what we should probably call haptic. For example, the fitness band that vibrates when it thinks you have been sitting for too long.
But many of the acts we now think of as nudges are delivered verbally, as some kind of speech act. But which kind?
The most obvious kind of nudge is a direct suggestion, which may take the form of a weak command. ("Try and eat a little now.") But nudges can also take other illocutionary forms, including questions ("Don't you think the sun is very hot here?") and statements / predictions ("You will find that new nose of yours very useful to spank people with.").
(Readers familiar with Kipling may recognize my examples as the nudges given by the Bi-Coloured-Python-Rock-Snake to the Elephant's Child.)
The force of a suggestion may depend on context and tone of voice. (A more systematic analysis of what philosophers call illocutionary force can be found in the Stanford Encyclopedia of Philosophy, based on Searle and Vanderveken 1985.)
@tonyjoyce raises a good point about tone of voice in electronic messages. Traditionally robots don't do tone of voice, and when a human being talks in a boring monotone we may describe their speech as robotic. But I can't see any reason why robots couldn't be programmed with more varied speech patterns, including tonality, if their designers saw the value of this.
Meanwhile, we already get some differentation from electronic communications. For example I should expect an electronic announcement to "LEAVE THE BUILDING IMMEDIATELY" to have a tone of voice that conveys urgency, and we might think it is inappropriate or even unethical to use the same tone of voice for selling candy. We might put this together with other attention-seeking devices, such as flashing red text. The people who design clickbait clearly understand illocutionary force (even if they aren't familiar with the term).
A speech act can also gain force by being associated with action. If I promise to donate money to a given charity, this may nudge other people to do the same; but if they see me actually putting the money in the tin, the nudge might be much stronger. But then the nudge might be just as strong if I just put the money in the tin without saying anything, as long as everyone sees me do it. The important point is that some communication takes place, whether verbal or non-verbal, and this returns us to something closer to the original concept of nudge.
From an ethical point of view, there are particular concerns about unobtrusive or subliminal nudges. Yeung has introduced the concept of the Hypernudge, which combines three qualities: nimble, unobtrusive and highly potent. I share her concerns about this combination, but I think it is helpful to deal with these three qualities separately, before looking at the additional problems that may arise when they are combined.
Proponents of the nudge sometimes try to distinguish between unobtrusive (acceptable) and subliminal (unacceptable), but this distinction may be hard to sustain, and many people quote Luc Bovens' observation that nudges "typically work better in the dark". See also Baldwin.
I'm sure there's more to say on this topic, so I may update this post later. Relevant comments always welcome.
Robert Baldwin, From regulation to behaviour change: giving nudge the third degree (The Modern Law Review 77/6, 2014) pp 831-857
Luc Bovens, The Ethics of Nudge. In Mats J. Hansson and Till Grüne-Yanoff (eds.), Preference Change: Approaches from Philosophy, Economics and Psychology. (Berlin: Springer, 2008) pp. 207-20
John Danaher, Algocracy as Hypernudging: A New Way to Understand the Threat of Algocracy (Institute for Ethics and Emerging Technologies, 17 January 2017)
J. Searle and D. Vanderveken, Foundations of Illocutionary Logic (Cambridge: Cambridge University Press, 1985)
Karen Yeung, ‘Hypernudge’: Big Data as a Mode of Regulation by Design (Information, Communication and Society (2016) 1,19; TLI Think! Paper 28/2016)
Stanford Encyclopedia of Philosophy: Speech Acts
Related posts: On the Ethics of Technologically Mediated Nudge (May 2019), Nudge Technology (July 2019)
Updated 28 May 2019. Many thanks to @tonyjoyce
Friday, May 17, 2019
On the Ethics of Technologically Mediated Nudge
Before we can discuss the ethics of technologically mediated nudge, we need to recognize that many of the ethical issues are the same whether the nudge is delivered by a human or a robot. So let me start by trying to identify different categories of nudge.
In its simplest form, the nudge can involve gentle persuasions and hints between one human being and another. Parents trying to influence their children (and vice versa), teachers hoping to inspire their pupils, various forms of encouragement and consensus building and leadership. In fiction, such interventions often have evil intent and harmful consequences, but in real life let's hope that these interventions are mostly well-meaning and benign.
In contrast, there are more large-scale forms of nudge, where a team of social engineers (such as the notorious
I shall call these two types unmediated and mediated respectively.
Mediated nudges may be delivered in various ways. For example, someone in Central Government may design a nudge to encourage job-seekers to find work. Meanwhile, YouTube can nudge us to watch a TED talk about nudging. Some nudges can be distributed via the Internet, or even the Internet of Things. In general, this involves both people and technology - in other words, a sociotechnical system.
To assess the outcome of the nudge, we can look at the personal effect on the nudgee or at the wider socio-economic impact, either short-term or longer-term. In terms of outcome, it may not make much difference whether the nudge is delivered by a human being or by a machine, given that human beings delivering the nudge might be given a standard script or procedure to follow, except in so far as the nudgee may feel differently about it, and may therefore respond differently. It is an empirical question whether a given person would respond more positively to a given nudge from a human bureaucrat or from a smartphone app, and the ethical difference between the two will be largely driven by this.
The second distinction involves the beneficiary of the nudge. Some nudges are designed to benefit the nudgee (Cass Sunstein calls these
A slightly more complicated case from an ethical perspective is where the intended outcome of the nudge is to get the nudgee to behave more ethically or responsibly towards someone else.
Sunstein sees the
Meanwhile, there is a significant category of self-nudging. There are now countless apps and other devices that will nudge you according to a set of rules or parameters that you provide yourself, implementing the kind of self-binding or precommitment that Jon Elster described in Ulysses and the Sirens (1979). Examples include the Tomato system for time management, fitness trackers that will count your steps and vibrate when you have been sitting for too long, money management apps that allocate your spare change to your chosen charity. Several years ago, Microsoft developed an experimental Smart Bra that would detect changes in the skin to predict when a women was about to reach for the cookie jar, and give her a friendly warning. Even if there is no problem with the nudge itself (because you have consented/chosen to be nudged) there may be some ethical issues with the surveillance and machine learning systems that enable the nudge. Especially when the nudging device is kindly made available to you by your employer or insurance company.
And even if the immediate outcome of the nudge is benefical to the nudgee, in some situations there may be concerns that the nudgee becomes over-dependent on being nudged, and thereby loses some element of self-control or delayed gratification.
The final distinction I want to introduce here concerns the direction of the nudge. The most straightforward nudges are those that push an individual in the desired direction. Suggestions to eat more healthy food, suggestions to direct spare cash to charity or savings. But some forms of therapy are based on paradoxical interventions, where the individual is pushed in the opposite direction, and they react by moving in the direction you want them to go. For example, if you want someone to give up some activity that is harming them, you might suggest they carry out this activity more systematically or energetically. This is sometimes known as reverse psychology or prescribing the symptom. For example, faced with a girl who was biting her nails, the therapist Milton Erickson advised her how she could get more enjoyment from biting her nails. Astonished by this advice, which was of course in direct opposition to all the persuasion and coercion she had received from other people up to that point, she found she was now able to give up biting her nails altogether.
(Richard Bordenave attributes paradoxical intervention to Paul Watzlawick, who worked with Gregory Bateson. It can also be found in some versions of Neuro-Linguistic Programming (NLP), which was strongly influenced by both Bateson and Erickson.)
Of course, this technique can also be practised in an ethically unacceptable direction as well. Imagine a gambling company whose official message to gamblers is that they should invest their money in a sensible savings account instead of gambling it away. This might seem like an ethically noble gesture, until we discover that the actual effect on people with a serious gambling problem is that this causes them to gamble even more. (In the same way that smoking warnings can cause some people to smoke more. Possibly cigarette companies are aware of this.)
Update: new study on warning messages to gamblers indicates a possible (but not statistically significant) counterproductive effect. See link below.
Reverse psychology may also explain why nudge programmes may have the opposite effect to intended one. Shortly before the 2016 Brexit Referendum, an English journalist writing for RT (formerly known as Russia Today) noted a proliferation of nudges trying to persuade people to vote remain, which he labelled propaganda. While the result was undoubtedly affected by covert nudges in all directions, it is also easy to believe that the pro-establishment style of the Remain nudges could have been counterproductive.
Paradoxical interventions make perfect sense in terms of systems theory, which teaches us that the links from cause to effect are often complex and non-linear. Sometimes an accumulation of positive nudges can tip a system into chaos or catastrophe, as Donella Meadows notes in her classic essay on Leverage Points.
The Leverage Point framework may also be useful in comparing the effects of nudging at different points in a system. Robert Steele notes the use of a nudge based on restructuring information flows; in contrast, a nudge that was designed to alter the nudgee's preferences or goals or political opinions could be much more dangerously powerful, as @zeynep has demonstrated in relation to YouTube.
One of the things that complicates the ethics of Nudge is that the alternative to nudging may either be greater forms of coercion or worse outcomes for the individual. In his article on the Ethics of Nudging, Cass Sunstein argues that all human interaction and activity takes place inside some kind of Choice Architecture, thus some form of nudging is probably inevitable, whether deliberate or inadvertent. He also argues that nudges may be required on ethical grounds to the extent that they promote our core human values. (This might imply that it is sometimes irresponsible to miss an opportunity to provide a helpful nudge.) So the ethical question is not whether to nudge or not, but how to design nudges in such a way as to maximize these core human values, which he identifies as welfare, autonomy and human dignity.
While we can argue with some of the detail of Sunstein's position, I think his two main conclusions make reasonable sense. Firstly, that we are always surrounded by what Sunstein calls Choice Architectures, so we can't get away from the nudge. And secondly, that many nudges are not just preferable to whatever the alternative might be but may also be valuable in their own right.
So what happens when we introduce advanced technology into the mix? For example, what if we have a robot that is programmed to nudge people, perhaps using some kind of artificial intelligence or machine learning to adapt the nudge to each individual in a specific context at a specific point in time?
Within technology ethics, transparency is a major topic. If the robot is programmed to include a predictive model of human psychology that enables it to anticipate the human response in certain situations, this model should be open to scrutiny. Although such models can easily be wrong or misguided, especially if the training data set reflects an existing bias, with reasonable levels of transparency (at least for the appropriate stakeholders) it will usually be easier to detect and correct these errors than to fix human misconceptions and prejudices.
In science fiction, robots have sufficient intelligence and understanding of human psychology to invent appropriate nudges for a given situation. If we start to see more of this in real life, we could start to think of these as unmediated robotic nudges, instead of the robot merely being the delivery mechanism for a mediated nudge. But does this introduce any additional ethical issues, or merely amplify the importance of the ethical issues we are already looking at?
Some people think that the ethical rules should be more stringent for robotic nudges than for other kinds of nudges. For example, I've heard people talking about parental consent before permitting children to be nudged by a robot. But other people might think it was safer for a child to be nudged (for whatever purpose) by a robot than by an adult human. And if you think it is a good thing for a child to work hard at school, eat her broccoli, and be kind to those less fortunate than herself, and if robotic persuasion turns out to be the most effective and child-friendly way of achieving these goals, do we really want heavier regulation on robotic child-minders than human ones?
Finally, it's worth noting that as nudges exploit bounded rationality, any entity that displays bounded rationality is capable of being nudged. As well as humans, this includes animals, algorithmic machines, as well as larger social systems (including markets and elections).
Richard Bordenave, Comment les paradoxes permettent de réinventer les nudges (Harvard Business Review France, 30 January 2019). Adapted English version: When paradoxes inspire Nudges (6 April 2019)
Rob Davies, Warning message on gambling ads does little to stop betting – study (The Guardian, 4 August 2019)
Jon Elster, Ulysses and the Sirens (1979)
Sam Gerrans, Propaganda techniques nudging UK to remain in Europe (RT, 22 May 2016)
Jochim Hansen, Susanne Winzeler and Sascha Topolinski, When the Death Makes You Smoke: A Terror Management Perspective on the Effectiveness of Cigarette On-Pack Warnings (Journal of Exp,erimental Social Psychology 46(1):226-228, January 2010) HT @ABMarkman
Donella Meadows, Leverage Points: Places to Intervene in a System (Whole Earth Review, Winter 1997)
Robert Steele, Implementing an integrated and transformative agenda at the regional and national levels (AtKisson, 2014)
Cass Sunstein, The Ethics of Nudging (Yale J. on Reg, 32, 2015)
Iain Thomson, Microsoft researchers build 'smart bra' to stop women's stress eating (The Register, 6 Dec 2013)
Zeynep Tufekci, YouTube, the Great Radicalizer (New York Times, 10 March 2018)
Wikipedia: Behavioural Insights Team ("Nudge Unit"), Bounded Rationality, Reverse Psychology,
Stanford Encyclopedia of Philosophy: The Ethics of Manipulation
Related posts: Good Ideas from Flaky Sources (December 2009), Have you got big data in your underwear? (December 2014), Ethical Communication in a Digital Age (November 2018), The Nudge as a Speech Act (May 2019), Nudge Technology (July 2019)
Updated 4 August 2019
In its simplest form, the nudge can involve gentle persuasions and hints between one human being and another. Parents trying to influence their children (and vice versa), teachers hoping to inspire their pupils, various forms of encouragement and consensus building and leadership. In fiction, such interventions often have evil intent and harmful consequences, but in real life let's hope that these interventions are mostly well-meaning and benign.
In contrast, there are more large-scale forms of nudge, where a team of social engineers (such as the notorious
Nudge Unit) design ways of influencing the behaviour of lots of people, but don't have any direct contact with the people whose behaviour is to be influenced. A new discipline has grown up, known as Behavioural Economics.
I shall call these two types unmediated and mediated respectively.
Mediated nudges may be delivered in various ways. For example, someone in Central Government may design a nudge to encourage job-seekers to find work. Meanwhile, YouTube can nudge us to watch a TED talk about nudging. Some nudges can be distributed via the Internet, or even the Internet of Things. In general, this involves both people and technology - in other words, a sociotechnical system.
To assess the outcome of the nudge, we can look at the personal effect on the nudgee or at the wider socio-economic impact, either short-term or longer-term. In terms of outcome, it may not make much difference whether the nudge is delivered by a human being or by a machine, given that human beings delivering the nudge might be given a standard script or procedure to follow, except in so far as the nudgee may feel differently about it, and may therefore respond differently. It is an empirical question whether a given person would respond more positively to a given nudge from a human bureaucrat or from a smartphone app, and the ethical difference between the two will be largely driven by this.
The second distinction involves the beneficiary of the nudge. Some nudges are designed to benefit the nudgee (Cass Sunstein calls these
paternalistic), while others are designed to benefit the community as a whole (for example, correcting some market failure such as the Tragedy of the Commons). On the one hand, nudges that encourage people to exercise more; on the other hand, nudges that remind people to take their litter home. And of course there are also nudges whose intended beneficiary is the person or organization doing the nudging. We might think here of dark patterns, shades of manipulation, various ways for commercial organizations to get the individual to spend more time or money. Clearly there are some ethical issues here.
A slightly more complicated case from an ethical perspective is where the intended outcome of the nudge is to get the nudgee to behave more ethically or responsibly towards someone else.
Sunstein sees the
paternalisticnudges as more controversial than nudges to address potential market failures, and states two further preferences. Firstly, he prefers nudges that educate people, that serve over time to increase rather than decrease their powers of agency. And secondly, he prefers nudges that operate at a slow deliberative tempo (
System 2) rather than at a fast intuitive tempo (
System 1), since the latter can seem more manipulative.
Meanwhile, there is a significant category of self-nudging. There are now countless apps and other devices that will nudge you according to a set of rules or parameters that you provide yourself, implementing the kind of self-binding or precommitment that Jon Elster described in Ulysses and the Sirens (1979). Examples include the Tomato system for time management, fitness trackers that will count your steps and vibrate when you have been sitting for too long, money management apps that allocate your spare change to your chosen charity. Several years ago, Microsoft developed an experimental Smart Bra that would detect changes in the skin to predict when a women was about to reach for the cookie jar, and give her a friendly warning. Even if there is no problem with the nudge itself (because you have consented/chosen to be nudged) there may be some ethical issues with the surveillance and machine learning systems that enable the nudge. Especially when the nudging device is kindly made available to you by your employer or insurance company.
And even if the immediate outcome of the nudge is benefical to the nudgee, in some situations there may be concerns that the nudgee becomes over-dependent on being nudged, and thereby loses some element of self-control or delayed gratification.
The final distinction I want to introduce here concerns the direction of the nudge. The most straightforward nudges are those that push an individual in the desired direction. Suggestions to eat more healthy food, suggestions to direct spare cash to charity or savings. But some forms of therapy are based on paradoxical interventions, where the individual is pushed in the opposite direction, and they react by moving in the direction you want them to go. For example, if you want someone to give up some activity that is harming them, you might suggest they carry out this activity more systematically or energetically. This is sometimes known as reverse psychology or prescribing the symptom. For example, faced with a girl who was biting her nails, the therapist Milton Erickson advised her how she could get more enjoyment from biting her nails. Astonished by this advice, which was of course in direct opposition to all the persuasion and coercion she had received from other people up to that point, she found she was now able to give up biting her nails altogether.
(Richard Bordenave attributes paradoxical intervention to Paul Watzlawick, who worked with Gregory Bateson. It can also be found in some versions of Neuro-Linguistic Programming (NLP), which was strongly influenced by both Bateson and Erickson.)
Of course, this technique can also be practised in an ethically unacceptable direction as well. Imagine a gambling company whose official message to gamblers is that they should invest their money in a sensible savings account instead of gambling it away. This might seem like an ethically noble gesture, until we discover that the actual effect on people with a serious gambling problem is that this causes them to gamble even more. (In the same way that smoking warnings can cause some people to smoke more. Possibly cigarette companies are aware of this.)
Update: new study on warning messages to gamblers indicates a possible (but not statistically significant) counterproductive effect. See link below.
Reverse psychology may also explain why nudge programmes may have the opposite effect to intended one. Shortly before the 2016 Brexit Referendum, an English journalist writing for RT (formerly known as Russia Today) noted a proliferation of nudges trying to persuade people to vote remain, which he labelled propaganda. While the result was undoubtedly affected by covert nudges in all directions, it is also easy to believe that the pro-establishment style of the Remain nudges could have been counterproductive.
Paradoxical interventions make perfect sense in terms of systems theory, which teaches us that the links from cause to effect are often complex and non-linear. Sometimes an accumulation of positive nudges can tip a system into chaos or catastrophe, as Donella Meadows notes in her classic essay on Leverage Points.
The Leverage Point framework may also be useful in comparing the effects of nudging at different points in a system. Robert Steele notes the use of a nudge based on restructuring information flows; in contrast, a nudge that was designed to alter the nudgee's preferences or goals or political opinions could be much more dangerously powerful, as @zeynep has demonstrated in relation to YouTube.
One of the things that complicates the ethics of Nudge is that the alternative to nudging may either be greater forms of coercion or worse outcomes for the individual. In his article on the Ethics of Nudging, Cass Sunstein argues that all human interaction and activity takes place inside some kind of Choice Architecture, thus some form of nudging is probably inevitable, whether deliberate or inadvertent. He also argues that nudges may be required on ethical grounds to the extent that they promote our core human values. (This might imply that it is sometimes irresponsible to miss an opportunity to provide a helpful nudge.) So the ethical question is not whether to nudge or not, but how to design nudges in such a way as to maximize these core human values, which he identifies as welfare, autonomy and human dignity.
While we can argue with some of the detail of Sunstein's position, I think his two main conclusions make reasonable sense. Firstly, that we are always surrounded by what Sunstein calls Choice Architectures, so we can't get away from the nudge. And secondly, that many nudges are not just preferable to whatever the alternative might be but may also be valuable in their own right.
So what happens when we introduce advanced technology into the mix? For example, what if we have a robot that is programmed to nudge people, perhaps using some kind of artificial intelligence or machine learning to adapt the nudge to each individual in a specific context at a specific point in time?
Within technology ethics, transparency is a major topic. If the robot is programmed to include a predictive model of human psychology that enables it to anticipate the human response in certain situations, this model should be open to scrutiny. Although such models can easily be wrong or misguided, especially if the training data set reflects an existing bias, with reasonable levels of transparency (at least for the appropriate stakeholders) it will usually be easier to detect and correct these errors than to fix human misconceptions and prejudices.
In science fiction, robots have sufficient intelligence and understanding of human psychology to invent appropriate nudges for a given situation. If we start to see more of this in real life, we could start to think of these as unmediated robotic nudges, instead of the robot merely being the delivery mechanism for a mediated nudge. But does this introduce any additional ethical issues, or merely amplify the importance of the ethical issues we are already looking at?
Some people think that the ethical rules should be more stringent for robotic nudges than for other kinds of nudges. For example, I've heard people talking about parental consent before permitting children to be nudged by a robot. But other people might think it was safer for a child to be nudged (for whatever purpose) by a robot than by an adult human. And if you think it is a good thing for a child to work hard at school, eat her broccoli, and be kind to those less fortunate than herself, and if robotic persuasion turns out to be the most effective and child-friendly way of achieving these goals, do we really want heavier regulation on robotic child-minders than human ones?
Finally, it's worth noting that as nudges exploit bounded rationality, any entity that displays bounded rationality is capable of being nudged. As well as humans, this includes animals, algorithmic machines, as well as larger social systems (including markets and elections).
Richard Bordenave, Comment les paradoxes permettent de réinventer les nudges (Harvard Business Review France, 30 January 2019). Adapted English version: When paradoxes inspire Nudges (6 April 2019)
Rob Davies, Warning message on gambling ads does little to stop betting – study (The Guardian, 4 August 2019)
Jon Elster, Ulysses and the Sirens (1979)
Sam Gerrans, Propaganda techniques nudging UK to remain in Europe (RT, 22 May 2016)
Jochim Hansen, Susanne Winzeler and Sascha Topolinski, When the Death Makes You Smoke: A Terror Management Perspective on the Effectiveness of Cigarette On-Pack Warnings (Journal of Exp,erimental Social Psychology 46(1):226-228, January 2010) HT @ABMarkman
Donella Meadows, Leverage Points: Places to Intervene in a System (Whole Earth Review, Winter 1997)
Robert Steele, Implementing an integrated and transformative agenda at the regional and national levels (AtKisson, 2014)
Cass Sunstein, The Ethics of Nudging (Yale J. on Reg, 32, 2015)
Iain Thomson, Microsoft researchers build 'smart bra' to stop women's stress eating (The Register, 6 Dec 2013)
Zeynep Tufekci, YouTube, the Great Radicalizer (New York Times, 10 March 2018)
Wikipedia: Behavioural Insights Team ("Nudge Unit"), Bounded Rationality, Reverse Psychology,
Stanford Encyclopedia of Philosophy: The Ethics of Manipulation
Related posts: Good Ideas from Flaky Sources (December 2009), Have you got big data in your underwear? (December 2014), Ethical Communication in a Digital Age (November 2018), The Nudge as a Speech Act (May 2019), Nudge Technology (July 2019)
Updated 4 August 2019
Wednesday, June 13, 2018
Practical Ethics
A lot of ethical judgements appear to be binary ones. Good versus bad. Acceptable versus unacceptable. Angels versus Devils.
Where questions of ethics reach the public sphere, it is common for people to take strong positions for or against. For example, there have been some high-profile cases involving seriously sick children, whether they should be provided with some experimental treatment, or even whether they should be kept alive at all. These are incredibly difficult decisions for those closely involved, but the experts are then subjected to vitriolic attack from armchair critics (often from the other side of the world) who think they know better.
Practical ethics are mostly about trade-offs, interpreting the evidence, predicting the consequences, estimating and balancing the benefits and risks. There isn't a simple formula that can be applied, each case must be carefully considered to determine where it sits on a spectrum.
The same is true of business and technology ethics. There isn't a blanket rule that says that these forms of persuasion are good and these forms are bad, there are just different degrees of nudge. We might want to regard all nudges with some suspicion, but retailers have always nudged people to purchase things. The question is whether this particular form of nudge is acceptable in this context, or whether it crosses some fuzzy line into manipulation or worse. Where does this particular project sit on the spectrum?
Technologists sometimes abdicate responsibility for such questions. Whatever the client wants, or whatever the technology enables, is okay. Responsibility means owning that judgement.
When Google published its AI ethics recently, Eric Newcomer complained that balancing the benefits and risks sounded like the utilitarianism he learned about at high school. But he also complained that Google's approach lacks impartiality and agent-neutrality. It would therefore be more accurate to describe Google's approach as consequentialism.
In the real world, even the question of agent-neutrality is complicated. Sometimes this is interpreted as a call to disregard any judgement made by a stakeholder, on the grounds that they must be biased. For example, ignoring professional opinions (doctors, teachers) because they might be trying to protect their own professional status. But taking important decisions about healthcare or education away from the professionals doesn't solve the problem of bias, it merely replaces professional bias with some other form of bias.
In Google's case, people are entitled to question how exactly Google will make these difficult judgements, and the extent to which these judgements may be subject to some conflict of interest. But if there is no other credible body that can make these judgements, perhaps the best we can ask for (at least for now) is some kind of transparency or scrutiny.
As I said above, practical ethics are mostly about consequences - which philosophers call consequentialism. But not entirely. Ethical arguments about the human subject aren't always framed in terms of observable effects, but may be framed in terms of human values. For example, the idea people should be given control over something or other, not because it makes them happier, but just because, you know, they should. Or the idea that certain things (truth, human life, etc.) are sacrosanct.
In his book The Human Use of Human Beings, first published in 1950, Norbert Wiener based his computer ethics on what he called four great principles of justice. So this is not just about balancing outcomes.
Of course, a complex issue may require more than a single dimension. It may be useful to draw spider diagrams or radar charts, to help to visualize the relevant factors. Alternatively, Cathy O'Neil recommends the Ethical or Stakeholder Matrix technique, originally invented by Professor Ben Mepham.
Jessi Hempel, Want to prove your business is fair? Audit your algorithm (Wired 9 May 2018)
Ben Mepham, Ethical Principles and the Ethical Matrix. Chapter 3 in J. Peter Clark Christopher Ritson (eds), Practical Ethics for Food Professionals: Ethics in Research, Education and the Workplace (Wiley 2013)
Eric Newcomer, What Google's AI Principles Left Out (Bloomberg 8 June 2018)
Tom Upchurch, To work for society, data scientists need a hippocratic oath with teeth (Wired, 8 April 2018)
Stanford Encyclopedia of Philosophy: Computer and Information Ethics, Consequentialism, Utilitarianism
Related posts: Conflict of Interest (March 2018), Data and Intelligence Principles From Major Players (June 2018)
Where questions of ethics reach the public sphere, it is common for people to take strong positions for or against. For example, there have been some high-profile cases involving seriously sick children, whether they should be provided with some experimental treatment, or even whether they should be kept alive at all. These are incredibly difficult decisions for those closely involved, but the experts are then subjected to vitriolic attack from armchair critics (often from the other side of the world) who think they know better.
Practical ethics are mostly about trade-offs, interpreting the evidence, predicting the consequences, estimating and balancing the benefits and risks. There isn't a simple formula that can be applied, each case must be carefully considered to determine where it sits on a spectrum.
The same is true of business and technology ethics. There isn't a blanket rule that says that these forms of persuasion are good and these forms are bad, there are just different degrees of nudge. We might want to regard all nudges with some suspicion, but retailers have always nudged people to purchase things. The question is whether this particular form of nudge is acceptable in this context, or whether it crosses some fuzzy line into manipulation or worse. Where does this particular project sit on the spectrum?
Technologists sometimes abdicate responsibility for such questions. Whatever the client wants, or whatever the technology enables, is okay. Responsibility means owning that judgement.
When Google published its AI ethics recently, Eric Newcomer complained that balancing the benefits and risks sounded like the utilitarianism he learned about at high school. But he also complained that Google's approach lacks impartiality and agent-neutrality. It would therefore be more accurate to describe Google's approach as consequentialism.
In the real world, even the question of agent-neutrality is complicated. Sometimes this is interpreted as a call to disregard any judgement made by a stakeholder, on the grounds that they must be biased. For example, ignoring professional opinions (doctors, teachers) because they might be trying to protect their own professional status. But taking important decisions about healthcare or education away from the professionals doesn't solve the problem of bias, it merely replaces professional bias with some other form of bias.
In Google's case, people are entitled to question how exactly Google will make these difficult judgements, and the extent to which these judgements may be subject to some conflict of interest. But if there is no other credible body that can make these judgements, perhaps the best we can ask for (at least for now) is some kind of transparency or scrutiny.
As I said above, practical ethics are mostly about consequences - which philosophers call consequentialism. But not entirely. Ethical arguments about the human subject aren't always framed in terms of observable effects, but may be framed in terms of human values. For example, the idea people should be given control over something or other, not because it makes them happier, but just because, you know, they should. Or the idea that certain things (truth, human life, etc.) are sacrosanct.
In his book The Human Use of Human Beings, first published in 1950, Norbert Wiener based his computer ethics on what he called four great principles of justice. So this is not just about balancing outcomes.
Freedom. Justice requires “the liberty of each human being to develop in his freedom the full measure of the human possibilities embodied in him.”
Equality. Justice requires “the equality by which what is just for A and B remains just when the positions of A and B are interchanged.”
Benevolence. Justice requires “a good will between man and man that knows no limits short of those of humanity itself.”
Minimum Infringement of Freedom. “What compulsion the very existence of the community and the state may demand must be exercised in such a way as to produce no unnecessary infringement of freedom”
Of course, a complex issue may require more than a single dimension. It may be useful to draw spider diagrams or radar charts, to help to visualize the relevant factors. Alternatively, Cathy O'Neil recommends the Ethical or Stakeholder Matrix technique, originally invented by Professor Ben Mepham.
"A construction from the world of bio-ethics, the ethical or “stakeholder” matrix is a way of determining the answer to the question, does this algorithm work? It does so by considering all the stakeholders, and all of their concerns, be them positive (accuracy, profitability) or negative (false negatives, bad data), and in particular allows the deployer to think about and gauge all types of best case and worst case scenarios before they happen. The matrix is color coded with red, yellow, or green boxes to alert people to problem areas." [Source: ORCAA]
"The Ethical Matrix is a versatile tool for analysing ethical issues. It is intended to help people make ethical decisions, particularly about new technologies. It is an aid to rational thought and democratic deliberation, not a substitute for them. ... The Ethical Matrix sets out a framework to help individuals and groups to work through these debates in relation to a particular issue. It is designed so that a broader than usual range of ethical concerns is aired, differences of perspective become openly discussed, and the weighting of each concern against the others is made explicit. The matrix is based in established ethical theory but, as far as possible, employs user-friendly language." [Source: Food Ethics Council]
Jessi Hempel, Want to prove your business is fair? Audit your algorithm (Wired 9 May 2018)
Ben Mepham, Ethical Principles and the Ethical Matrix. Chapter 3 in J. Peter Clark Christopher Ritson (eds), Practical Ethics for Food Professionals: Ethics in Research, Education and the Workplace (Wiley 2013)
Eric Newcomer, What Google's AI Principles Left Out (Bloomberg 8 June 2018)
Tom Upchurch, To work for society, data scientists need a hippocratic oath with teeth (Wired, 8 April 2018)
Stanford Encyclopedia of Philosophy: Computer and Information Ethics, Consequentialism, Utilitarianism
Related posts: Conflict of Interest (March 2018), Data and Intelligence Principles From Major Players (June 2018)
Monday, January 21, 2013
The Price of Fish
Michael Mainelli and Ian Harris have written a wide-ranging survey of economics, choice theory (game theory, psychology and ethics), systems theory, chaos theory, global warming and evolution. So what's all that got to do with the price of fish?
One of the themes running through the book is that the price of fish bears no relation to the value of fish, especially if we are concerned about long-term value and the sustainability of fish stocks.
Oscar Wilde famously defined a cynic as one who knows the price of everything and the value of nothing. This definition has also been applied to accountants and economists. Michael and Ian are leaders of the Long Finance initiative, a movement within the City of London that aims to overcome this kind of short-term financial cynicism.
Michael and Ian describe the price of fish as a wicked problem - a problem that lacks easy definition as well as easy answers. "Sustaining the supply of edible fish is a wicked problem that presents global risks." (p 301) And yet they suggest that the system might possibly sort itself out. "As fish run out and have to be sustainably fished, the historic underpricing of fish ceases." (293)
But this is no time for naive optimism, and the system will undoubtedly need some intervention. "When the price is the same as the value, there are opportunities for sustainable financing. So far, price has not equaled value for fish. This is the biggest, wicked decision-making problem of all: knowing how to set a price that equals the value." (p 295)
In other words, the problem is not just the alarming dwindling of fish stocks but the collective cynicism that not only led to this problem but also amplifies it and resists dealing with it effectively. The key word in the problem statement is the word "set" - even if a few clever people can agree what the right price of fish should be, the real challenge is to set this price into global trading and consumption systems.
While the survey is light on the sociopolitical elements of the problem, the authors complain that governments have often made things worse, by inappropriate regulations and subsidies. Thus lazy or short-term thinking on the part of government is another manifestation of cynicism.
The Wikipedia article What's that got to do with the ...? derives the phrase from the alleged tendency of economists to connect everything with everything else. The authors go much further in this respect than most economists. But one trouble with systems thinking is this: once you start it's difficult to know where to stop. (In systems thinking circles, this is known as the warning of the doorknob.) Although the authors have covered a great deal of material, it's not hard to think of other stuff they could have mentioned.
I don't think the authors are in any hurry to write a sequel, but if they did it might be about the Price of a Bee. While fish are undervalued, bees (apart from those involved in honey production) have no direct economic value at all; but when the bee population is threatened, global agriculture as a whole is in serious jeopardy. The indirect value of bees is vastly greater than the market for honey. Under the right conditions, with appropriate political and financial systems, people and communities may be able to make long-term ethical investments in sustainable fisheries, with a reasonable prospect of a long-term financial return: the Long Finance initiative is trying to stimulate the kind of system change that will create these conditions. But how on earth do we get communities to invest in the world bee population, without creating a market for bees? And would we really want that? Systems thinking tells us to be careful what we wish for. (Mary Catherine Bateson calls this The Revenge of the Good Fairy.)
Systems thinking also tells us that management (both public sector and private sector) has a tendency to over-intervene, to meddle and micromanage and ultimately make things worse. On the other hand, doing nothing doesn't feel like a good option either. (See Owen Barder on Good Global Citizenship, January 2013.) But there may be some kind of leverage or nudge that might just help a complex system to avoid catastrophe. It is always difficult to steer a path between naive optimism and pessimistic fatalism, but the battle against cynicism requires that we try.
Price of Fish website
Long Finance website
Wikipedia What's that got to do with the ...?
For the warning of the doorknob, see my post We Ought to Know the Difference (April 2013)
One of the themes running through the book is that the price of fish bears no relation to the value of fish, especially if we are concerned about long-term value and the sustainability of fish stocks.
Oscar Wilde famously defined a cynic as one who knows the price of everything and the value of nothing. This definition has also been applied to accountants and economists. Michael and Ian are leaders of the Long Finance initiative, a movement within the City of London that aims to overcome this kind of short-term financial cynicism.
Michael and Ian describe the price of fish as a wicked problem - a problem that lacks easy definition as well as easy answers. "Sustaining the supply of edible fish is a wicked problem that presents global risks." (p 301) And yet they suggest that the system might possibly sort itself out. "As fish run out and have to be sustainably fished, the historic underpricing of fish ceases." (293)
But this is no time for naive optimism, and the system will undoubtedly need some intervention. "When the price is the same as the value, there are opportunities for sustainable financing. So far, price has not equaled value for fish. This is the biggest, wicked decision-making problem of all: knowing how to set a price that equals the value." (p 295)
In other words, the problem is not just the alarming dwindling of fish stocks but the collective cynicism that not only led to this problem but also amplifies it and resists dealing with it effectively. The key word in the problem statement is the word "set" - even if a few clever people can agree what the right price of fish should be, the real challenge is to set this price into global trading and consumption systems.
While the survey is light on the sociopolitical elements of the problem, the authors complain that governments have often made things worse, by inappropriate regulations and subsidies. Thus lazy or short-term thinking on the part of government is another manifestation of cynicism.
The Wikipedia article What's that got to do with the ...? derives the phrase from the alleged tendency of economists to connect everything with everything else. The authors go much further in this respect than most economists. But one trouble with systems thinking is this: once you start it's difficult to know where to stop. (In systems thinking circles, this is known as the warning of the doorknob.) Although the authors have covered a great deal of material, it's not hard to think of other stuff they could have mentioned.
I don't think the authors are in any hurry to write a sequel, but if they did it might be about the Price of a Bee. While fish are undervalued, bees (apart from those involved in honey production) have no direct economic value at all; but when the bee population is threatened, global agriculture as a whole is in serious jeopardy. The indirect value of bees is vastly greater than the market for honey. Under the right conditions, with appropriate political and financial systems, people and communities may be able to make long-term ethical investments in sustainable fisheries, with a reasonable prospect of a long-term financial return: the Long Finance initiative is trying to stimulate the kind of system change that will create these conditions. But how on earth do we get communities to invest in the world bee population, without creating a market for bees? And would we really want that? Systems thinking tells us to be careful what we wish for. (Mary Catherine Bateson calls this The Revenge of the Good Fairy.)
Systems thinking also tells us that management (both public sector and private sector) has a tendency to over-intervene, to meddle and micromanage and ultimately make things worse. On the other hand, doing nothing doesn't feel like a good option either. (See Owen Barder on Good Global Citizenship, January 2013.) But there may be some kind of leverage or nudge that might just help a complex system to avoid catastrophe. It is always difficult to steer a path between naive optimism and pessimistic fatalism, but the battle against cynicism requires that we try.
Price of Fish website
Long Finance website
Wikipedia What's that got to do with the ...?
For the warning of the doorknob, see my post We Ought to Know the Difference (April 2013)
Matt McGrath, Dispute means mackerel is no longer catch of the day (BBC News 22 January 2013)
Timothy Taylor, Do markets work for bees? (10 July 2014)
Related post: The Price of Everything (May 2017)
10 July 2014
Saturday, October 10, 2009
Lean versus Complex
Some interesting discussion contrasting Lean with Complex Adaptive Systems (CAS).
The Danger of Complex Adaptive Systems (Alan Shalloway)
The Danger of Lean - Ignoring Social Complexity (Jurgen Appelo)
Alan has a nice example of a school of mullet being consumed by a gang of dolphins. The mullet (species) has evolved a stratagem of swimming together to protect against predators. The dolphins (species) have evolved a higher intelligence, which enables them (specific individuals) to exploit the stratagem for their own advantage.
Which is the complex adaptive system here? Alan thinks it is the mullet system, and questions the value of CAS for protecting the mullet against risk. It is true that the mullet system has a degree of complexity: the dolphin cannot predict the path of a single fish, and can only nudge the system rather than control it exactly. But the mullet system is up against a much more intelligent adaptive system - that of the dolphins, whose tactics are learned rather than inherited.
Inherited behaviour, however complex it may seem, represents an adaptation to past challenges. Sometimes the same can be true of learned behaviour - the habits of management and problem-solving that have worked in the past. But in a dynamic competitive environment, the advantage of learned behaviour is that it can be changed. We regard dolphins as more intelligent than mullet because they are able to learn new behaviours, both by themselves and from one another.
Jurgen explains the problem of Lean and the advantage of CAS in terms of Cynefin. Appealing to a simplistic distinction drawn by Dave Snowden between "systems thinking" and "social complexity", he labels Lean as "systems thinking", argues that Lean fails to handle social complexity, and dismisses (that school of) systems thinking as outdated.
I prefer to explain the problem of Lean using ideas borrowed from Bateson. Lean is an adaptation that makes predictable systems more efficient. But highly adapted systems typically lack future adaptability. The mullet strategy is vulnerable to the dolphin intelligence. Indeed, the dolphin intelligence has evolved, among other things, in order to overcome the mullet strategy. So which system is the more complex, which system is the more adaptive?
Update
Alan has withdrawn his post rather than continue the argument with Jurgen. There is an archive version of the post on the Wayback Machine, complete with an interesting discussion thread. http://web.archive.org/web/20091101113550/http://www.netobjectives.com/blogs/the-dangers-of-complex-adaptive-systems
Updated 2 December 2016
The Danger of Complex Adaptive Systems (Alan Shalloway)
The Danger of Lean - Ignoring Social Complexity (Jurgen Appelo)
Alan has a nice example of a school of mullet being consumed by a gang of dolphins. The mullet (species) has evolved a stratagem of swimming together to protect against predators. The dolphins (species) have evolved a higher intelligence, which enables them (specific individuals) to exploit the stratagem for their own advantage.
Which is the complex adaptive system here? Alan thinks it is the mullet system, and questions the value of CAS for protecting the mullet against risk. It is true that the mullet system has a degree of complexity: the dolphin cannot predict the path of a single fish, and can only nudge the system rather than control it exactly. But the mullet system is up against a much more intelligent adaptive system - that of the dolphins, whose tactics are learned rather than inherited.
Inherited behaviour, however complex it may seem, represents an adaptation to past challenges. Sometimes the same can be true of learned behaviour - the habits of management and problem-solving that have worked in the past. But in a dynamic competitive environment, the advantage of learned behaviour is that it can be changed. We regard dolphins as more intelligent than mullet because they are able to learn new behaviours, both by themselves and from one another.
Jurgen explains the problem of Lean and the advantage of CAS in terms of Cynefin. Appealing to a simplistic distinction drawn by Dave Snowden between "systems thinking" and "social complexity", he labels Lean as "systems thinking", argues that Lean fails to handle social complexity, and dismisses (that school of) systems thinking as outdated.
I prefer to explain the problem of Lean using ideas borrowed from Bateson. Lean is an adaptation that makes predictable systems more efficient. But highly adapted systems typically lack future adaptability. The mullet strategy is vulnerable to the dolphin intelligence. Indeed, the dolphin intelligence has evolved, among other things, in order to overcome the mullet strategy. So which system is the more complex, which system is the more adaptive?
Update
Alan has withdrawn his post rather than continue the argument with Jurgen. There is an archive version of the post on the Wayback Machine, complete with an interesting discussion thread. http://web.archive.org/web/20091101113550/http://www.netobjectives.com/blogs/the-dangers-of-complex-adaptive-systems
Updated 2 December 2016
Subscribe to:
Posts (Atom)