Saturday, July 4, 2020

Limitations of Machine Learning

In a recent discussion on Twitter prompted by some examples of erroneous thinking in Computing Science, I argued that you don't always need a philosophy degree to spot these errors. A thorough grounding in statistics would seem to settle some of them.

@DietrichEpp disagreed completely. If you want a machine to learn then you have to understand the difference between data and knowledge. Stats classes don’t normally cover this.

So there are at least two questions here. Firstly, how much do you really have to understand in order to build a machine. As I see it, getting a machine do something (including learning) counts as engineering rather than science. Engineering requires two kinds of knowledge - practical knowledge (how to reliably, efficiently and safely produce a given outcome) and socio-ethical knowledge (whom shall the technology serve). Engineers are generally not expected to fully understand the scientific principles that underpin all the components, tools and design heuristics that they use, but they have a professional and ethical responsibility to have some awareness of the limitations of these tools and the potential consequences of their work.

In his book on Design Thinking, Peter Rowe links the concept of design heuristic to Gadamer's concept of enabling prejudice. Engineers would not be able to function without taking some things for granted.

So the second question is - which things can/should an engineer trust. Most computer engineers will be familiar with the phrase Garbage In Garbage Out, and this surely entails a professional scepticism about the quality of any input dataset. Meanwhile, statisticians are trained to recognize a variety of potential causes of bias. (Some of these are listed in the Wikipedia entry on statistical bias.) Most of the statistics courses I looked at on Coursera included material on inference.

Looking for relevant material to support my position, I found some good comments by Ariel Guersenzvaig, reported by Derek du Preez.
Unbiased data is an oxymoron. Data is biased from the start. You have to choose categories in order to collect the data. Sometimes even if you don’t choose the categories, they are there ad hoc. Linguistics, sociologists and historians of technology can teach us that categories reveal a lot about the mind, about how people think about stuff, about society.

And arriving too late for this Twitter discussion, two more stories of dataset bias were published in the last few days. Firstly, following an investigation by Vinay Prabhu and Abeba Birhane, MIT has withdrawn a very large image dataset, which has been widely used for machine learning, and asked researchers and developers to delete it. And secondly, FiveThirtyEight has published an excellent essay by Mimi Ọnụọha on the disconnect between data collection and meaningful change, arguing that it is impossible to collect enough data to convince people of structural racism.

So there are indeed some critical questions about data and knowledge that affect the practice of machine learning, and some critical insights from artists and sociologists. As for philosophy, famous philosophers from Plato to Wittgenstein have spent 2500 years exploring a broad range of abstract ideas about the relationship between data and knowledge, so you can probably find a plausible argument to support any position you wish to adopt. So this is hardly going to provide any consistent guidance for machine learning.

Mimi Ọnụọha, When Proof Is Not Enough (FiveThirtyEight, 1 July 2020)

Vinay Uday Prabhu and Abeba Birhane, Large Image Datasets: A pyrrhic win for computervision?(Preprint, 1 July 2020)

Derek du Preez, AI and ethics - ‘Unbiased data is an oxymoron’ (Diginomica, 31 October 2019)

Katyanna Quach, MIT apologizes, permanently pulls offline huge dataset that taught AI systems to use racist, misogynistic slurs Top uni takes action after El Reg highlights concerns by academics (The Register, 1 July 2020)

Peter Rowe, Design Thinking (MIT Press 1987)

Stanford Encyclopedia of Philosophy: Gadamer and the Positivity of Prejudice

Wikipedia: Algorithmic bias, All models are wrong, Bias (statistics), Garbage in garbage out

Further points and links in the following posts: Faithful Representation (August 2008), From Sedimented Principles to Enabling Prejudices (March 2013), Whom does the technology serve? (May 2019), Algorithms and Auditability (July 2019), Algorithms and Governmentality (July 2019), Naive Epistemology (July 2020) 

Friday, July 3, 2020

Naive Epistemology

One of the things I learned from studying maths and philosophy is an appreciation of what things follow from what other things. Identifying and understanding what assumptions are implicit in a given argument, what axioms required to establish a given proof.

So when I see or hear something that I disagree with, I feel the need to trace where the disagreement comes from - is there a difference in fact or value or something else? Am I missing some critical piece of knowledge or understanding, that might lead me to change my mind? And if I want to correct someone's error, is there some piece of knowledge or understanding that I can give them, that will bring them around to my way of thinking?

(By the way, this skill would seem important for teachers. If a child struggles with simple arithmetic, exactly which step in the process has the child failed to grasp? However, teachers don't always have time to do this.)

There is also an idea of the economy of argument. What is the minimum amount of knowledge or understanding that is needed in this context, and how can I avoid complicating the argument by bringing in a lot of other material that may be fascinating but not strictly relevant. (I acknowledge that I don't always follow this principle myself.) And when I'm wrong about something, how can other people help me see this without requiring me to wade through far more material than I have time for.

There was a thread on Twitter recently, prompted by some weak thinking by a certain computer scientist. @jennaburrell noted that computer science has never been very strong on epistemology – either recognizing that it implicitly has one, that there might be any other, or interrogating its weaknesses as a way of understanding the world.

Some people suggested that the solution involves philosophy.

I completely agree with Dietrich about the value of philosophy and other humanities in general. However, I felt it was overkill for addressing the specific weaknesses identified by Professor Burrell, as her argument against this particular fallacy didn't seem to require any non-STEM knowledge or understanding.

Of course, statistics is not the whole answer; but then neither is philosophy. I mentioned statistics as an example of a STEM discipline in which students should have the opportunity to unlearn naive epistemology; but of course any proper scientific discipline should include some understanding of scientific method. Although computing often calls itself a science, it is largely an engineering discipline; if you use the word methodology with computer people, they usually think you are talking about design methods. Social scientists (I believe Professor Burrell's PhD is in sociology) tend to have a much better understanding of research methodology.

And of course, it's not just epistemology but also ethics.

One of the problems with professional philosophy is that it can be quite compartmentalized. There are philosophers who promote themselves as experts on technology ethics, but their published papers don't reference any recent literature on the philosophy of science and technology, or reveal any deep understanding of the challenges faced by scientists and engineers.

So although there is undoubtedly good reasons for broader education in both directions, I'm sceptical about expecting clever people in one discipline to acquire a small but dangerous amount of expertise in some other discipline. I'm much more interested in promoting dialogue between disciplines. In his tribute to Steve Jobs, @jonahlehrer called this Consilience.

What set all of Steve Jobs’s companies apart ... was an insistence that computer scientists must work together with artists and designers—that the best ideas emerge from the intersection of technology and the humanities.
The final word should go to @abebab

Jonah Lehrer, Steve Jobs: “Technology Alone Is Not Enough” (New Yorker, 7 October 2011)

Related posts:  From Convenience to Consilience - “Technology Alone Is Not Enough"  (October 2011), The Habitual Vice of Epistemology (June 2019), Limitations of Machine Learning (July 2020)

Monday, June 29, 2020

Bold, Restless Experimentation

In his latest speech, invoking the spirit of Franklin Delano Roosevelt, Michael Gove calls for bold, restless experimentation.

Although one of Gove's best known pronouncements was his statement during the Brexit campaign that people in this country have had enough of experts ..., Fraser Nelson suggests he never intended this to refer to all experts: he was interrupted before he could specify which experts he meant.

Many of those who share Gove's enthusiasm for disruptive innovation also share his ambivalence about expertise. Joe McKendrick quotes Valar Afshar of DisrupTV: If the problem is unsolved, it means there are no experts.

Joe also quotes Michael Sikorsky of Robots and Pencils, who links talent, speed of decision and judgement, and talks about pushing as much of the decision rights as possible right to the edge of the organization. Meanwhile, Michael Gove also talks about diversifying the talent pool - not only a diversity of views but also a diversity of skills.

In some quarters, expertise means centralized intelligence - for example, clever people in Head Office. The problems with this model were identified by Harold Wilensky in his 1967 book on Organizational Intelligence, and explored more rigorously by David Alberts and his colleagues in CCRP, especially under the Power To The Edge banner.

Expertise also implies authority and permission; so rebellion against expertise can also take the form of permissionless innovation. Adam Thierer talks about the tinkering and continuous exploration that takes place at multiple levels, while Bernard Stiegler talks about disinhibition - a relaxation of constraints leading to systematic risk-taking.
Elevating individual talent over collective expertise is a risky enterprise. Malcolm Gladwell calls this the Talent Myth, while Stiegler calls it Madness. For further discussion and links, see my post Explaining Enron.

Michael Gove, The Privilege of Public Service (Ditchley Annual Lecture, 27 June 2020)

Henry Mance, Britain has had enough of experts, says Gove (Financial Times, 3 June 2016)

Fraser Nelson, Don't ask the experts (Spectator, 14 January 2017)

Bernard Stiegler, The Age of Disruption: Technology and Madness in Computational Capitalism (Polity Press, 2019). Review by John Reader (Postdigital Science and Education, 2019).

Adam Thierer, Permissionless Innovation (Mercatus Center, 2014/2016)

Related posts: Demise of the Superstar (August 2004), Power to the Edge (December 2005), Explaining Enron (January 2010), Enemies of Intelligence (May 2010), The Ethics of Disruption (August 2019)

Tuesday, January 28, 2020

The Algorithmic Child and the Anxious Parent

#OIILondonLecture An interesting lecture by @VickiNashOII of @oiioxford at @BritishAcademy_ this evening, entitled Connected cots, talking teddies and the rise of the algorithmic child.

Since the early days of the World Wide Web, people have been concerned about the risks to children. Initially, these were seen in terms of protecting children from unsuitable content and from contact with unsuitable strangers. Children also needed to be prevented from behaving inappropriately on the Internet.

In the days when a typical middle-class household had a single fixed computer in a downstairs room, it was relatively easy for parents to monitor their children's use of the Internet. But nowadays childen in Western countries think themselves deprived if they don't have the latest smartphone, and even toddlers often have their own tablet computers. So much of the activity can be hidden in the bedroom, or even under the bedclothes after lights out.

Furthermore, connection to the Internet is not merely through computers, phones, tablets and games consoles, but also through chatbots and connected toys, as well as the Internet of Things. So there is increasing awareness of some additional threats to children, including privacy and security, and it is becoming increasingly difficult for parents to protect their children from all these threats. (Even confiscating the phones may not solve the problem: one resourceful Kentucky teenager managed to send messages from the family smartfridge.)

And as Dr Nash pointed out, it's no longer just about how children use the internet, but also how the internet uses children. Large-scale collection and use of data is not just being practised by the technology giants, but by an increasing number of consumer companies and other commercial enterprises. One of the most interesting developments here is the provision of surveillance tools to help parents monitor their children.

Parents are being told that good parenting means keeping your children safe, and keeping them safe means knowing where they are at all times, what they are doing, whom they are with, and so on. All thanks to various tracking apps that provide real-time information about your children's location and activity. And even when they are at home, asleep in their own beds, there are monitoring technologies to track their temperature or breathing, and alert the parents of any abnormal pattern.

Dr Nash argues that this expectation of constantly monitoring one's children contributes to a significant alteration in the parent-child relationship, and in our norms of parenthood. Furthermore, as children become teenagers, they will increasingly be monitoring themselves, in healthy or unhealthy ways. So how should the monitoring parents monitor the monitoring?

One of the problems with any surveillance technology is that provides a single lens for viewing what is going on. Although this may be done with good intentions, and may often be beneficial, it is also selective in what it captures. It is so easy to fall into the fallacy of thinking that what is visible is important, and what is not visible is not important.  Those aspects of a child's life and experience that can be captured by clever technology aren't necessarily those aspects that a parent should be paying most attention to.

Linda Geddes, Does sharing photos of your children on Facebook put them at risk? (The Guardian, 21 Sep 2014)

Victoria Nash, The Unpolitics of Child Protection (Oxford Internet Institute, 5 May 2013)

Victoria Nash, Connected toys: not just child’s play (Parent Info, May 2018)

Victoria Nash, Huw Davies and Allison Mishkin, Digital Safety in the Era of Connected Cots and Talking Teddies (Oxford Internet Institute, 25 June 2019)

Caitlin O'Kane, Teen goes viral for tweeting from LG smart fridge after mom confiscates all electronics (CBS News 14 August 2019)

Related posts IOT is coming to town (December 2017), Shoshana Zuboff on Surveillance Capitalism (February 2019), Towards Chatbot Ethics (May 2019)

Thursday, November 7, 2019


Until the arrival of the motor car, the street belonged to humans and horses. The motor car was regarded as an interloper, and was generally blamed for collisions with pedestrians. Cities introduced speed limits and other safety measures to protect pedestrians from the motor car.

The motor industry fought back. Their goal was to shift the blame for collisions onto the foolish or foolhardy pedestrian, who had crossed the road in the wrong place at the wrong time, or showed insufficient respect to our new four-wheeled masters. A new crime was invented, known as jaywalking, and newspapers were encouraged to describe road accidents in these terms.

In March 2018, a middle-aged woman was killed by a self-driving car. This is thought to be the first recorded death by a fully autonomous vehicle. According to the US National Safety Transportation Board (NTSB), the vehicle failed to recognise her as a pedestrian because she was not at an obvious designated crossing. In other words, she was jaywalking.

As I've observed before, ethics professors like to introduce the Trolley Problem into the ethics of self-driving cars, often carrying out opinion surveys (whom shall the vehicle kill?) because these are easily published in peer-reviewed journals. A recent study at MIT found that many people thought law-abiding pedestrians had more right to safety than jaywalkers. Therefore, if faced with this unlikely choice, the car should kill the jaywalker and spare the others. You have been warned.

Jack Denton, Is the Trolley Problem Derailing the Ethics of Self-Driving Cars? (Pacific Standard 29 November 2018)

Aidan Lewis, Jaywalking: How the car industry outlawed crossing the road (BBC News, 12 February 2014)

Peter Norton, Street Rivals: Jaywalking and the Invention of the Motor Age Street (Technology and Culture, Vol 48, April 2007)

Katyanna Quach, Remember the Uber self-driving car that killed a woman crossing the street? The AI had no clue about jaywalkers (The Register, 6 November 2019)

Joseph Stromberg, The forgotten history of how automakers invented the crime of "jaywalking" (Vox, 4 November 2015)

Related posts: Whom Does The Technology Serve? (May 2019), The Game of Wits between Technologists and Ethics Professors (June 2019)

Wednesday, October 30, 2019

What Difference Does Technology Make?

In his book on policy-making, Geoffrey Vickers talks about three related types of judgment – reality judgment (what is going on, also called appreciation or sense-making), value judgment and action judgment.

In his book on technology ethics, Hans Jonas notes "the excess of our power to act over our power to foresee and our power to evaluate and to judge" (p22). In other words, technology disrupts the balance between the three types of judgment identified by Vickers.

Jonas (p23) identifies some critical differences between technological action and earlier forms
  • novelty of its methods
  • unprecedented nature of some of its objects
  • sheer magnitude of most of its enterprises
  • indefinitely cumulative propagation of its effects
In short, this amounts to action at a distance - the effects of one's actions and decisions reach further and deeper, affecting remote areas more quickly, and lasting long into the future. Which means that accepting responsibility only for the immediate and local effects of one's actions can no longer be justified.

Jonas also notes that the speed of technologically fed developments does not leave itself the time for self-correction (p32). An essential ethical difference between natural selection, selective breeding and genetic engineering is not just that they involve different mechanisms, but that they operate on different timescales.

(Of course humans have often foolishly disrupted natural ecosystems without recourse to technologies more sophisticated than boats. For example, the introduction of rabbits into Australia or starlings into North America. But technology creates many new opportunities for large-scale disruption.)

Another disruptive effect of technology is that it affects our reality judgments. Our knowledge and understanding of what is going on (WIGO) is rarely direct, but is mediated (screened) by technology and systems. We get an increasing amount of our information about our social world through technical media: information systems and dashboards, email, telephone, television, internet, social media, and these systems in turn rely on data collected by a wide range of monitoring instruments, including IoT. These technologies screen information for us, screen information from us.

The screen here is both literal and metaphorical. It is a surface on which the data are presented, and also a filter that controls what the user sees. The screen is a two-sided device: it both reveals information and hides information.

Heidegger thought that technology tends to constrain or impoverish the human experience of reality in specific ways. Albert Borgmann argued that technological progress tends to increase the availability of a commodity or service, and at the same time pushes the actual device or mechanism into the background. Thus technology is either seen as a cluster of devices, or it isn't seen at all. Borgmann calls this the Device Paradigm.

But there is a paradox here. On the one hand, the device encourages to pay attention to the immediate affordance of the device, and ignore the systems that support the device. So we happily consume recommendations from media and technology giants, without looking too closely at the surveillance systems and vast quantities of personal data that feed into these recommendations. But on the other hand, technology (big data, IoT, wearables) gives us the power to pay attention to vast areas of life that were previously hidden.

In agriculture for example, technology allows the farmer to have an incredibly detailed map of each field, showing how the yield varies from one square metre to the next. Or to monitor every animal electronically for physical and mental welbeing.

And not only farm animals, also ourselves. As I said in my post on the Internet of Underthings, we are now encouraged to account for everything we do: footsteps, heartbeats, posture. (Until recently this kind of micro-attention to oneself was regarded as slightly obsessional, nowadays it seems to be perfectly normal.)

Technology also allows much more fine-grained action. A farmer no longer has to give the same feed to all the cows every day, but can adjust the composition of the feed for each individual cow, to maximize her general well-being as well as her milk production.

In the 1980s when Borgmann and Jonas were writing, there was a growing gap between the power to act and the power to foresee. We now have technologies that may go some way towards closing this gap. Although these technologies are far from perfect, as well as introducing other ethical issues, they should at least make it easier for the effects of new technologies to be predicted, monitored and controlled, and for feedback and learning loops to be faster and more effective. And responsible innovation should take advantage of this.

Albert Borgmann, Technology and the Character of Everyday Life (University of Chicago Press, 1984)

Hans Jonas, The Imperative of Responsibility (University of Chicago Press, 1984)

Geoffrey Vickers, The Art of Judgment: A Study in Policy-Making (Sage 1965)

Wikipedia: Rabbits in Australia, Starlings in North America

Sunday, October 20, 2019

On the Scope of Ethics

I was involved in a debate this week, concerning whether ethical principles and standards should include weapons systems, or whether military purposes should be explicitly excluded.

On both sides of the debate, there were people who strongly disapproved of weapons systems, but this disapproval led them to two opposite positions. One side felt that applying any ethical principles and standards to such systems would imply a level of ethical approval or endorsement, which they would prefer to withhold. The other side felt that weapons systems called for at least as much ethical scrutiny as anything else, if not more, and thought that exempting weapons systems implied a free pass.

It goes without saying that people disapprove of weapons systems to different degrees. Some people think they are unacceptable in all circumstances, while others see them as a regrettable necessity, while welcoming the economic activity and technological spin-offs that they produce. It's also worth noting that there are other sectors that attract strong disapproval from many people, including gambling, hydrocarbon, nuclear energy and tobacco, especially where these appear to rely on disinformation campaigns such as climate science denial.

It's also worth noting that there isn't always a clear dividing line between those products and technologies that can be used for military purposes and those that cannot. For example, although the dividing line between peaceful nuclear power and nuclear weapons may be framed as a purely technical question, this has major implications for international relations, and technical experts may be subject to significant political pressure.

While there may be disagreements about the acceptability of a given technology, and legitimate suspicion about potential use, these should be capable of being addressed as part of ethical governance. So I don't think this is a good reason for limiting the scope.

However, a better reason for limiting the scope may be to simplify the task. Given finite time and resources, it may be better to establish effective governance for a limited scope, than taking forever getting something that works properly for everything. This leads to the position that although some ethical governance may apply to weapons systems, this doesn't mean that every ethical governance exercise must address such systems. And therefore it may be reasonable to exclude such systems from a specific exercise for a specific time period, provided that this doesn't rule out the possibility of extending the scope at a later date.

Update. The US Department of Defense has published a high-level set of ethical principles for the military use of AI. Following the difference of opinion outlined above, some people will think it matters how these principles are interpreted and applied in specific cases (since like many similar sets of principles, they are highly generic), while other people will think any such discussion completely misses the point.

David Vergun, Defense Innovation Board Recommends AI Ethical Guidelines (US Dept of Defense, 1 November 2019)