AI/ML Ethics, Biases, & Responsibility

Part 2 & Alumni (2018)
1

Following Zuckerberg’s testimony today, I wanted to create this open discussion thread in the fastai forums. I feel like the issue of biases and overall ethics in the whole field of AI is such an important issue that more careful consideration and attention needs to be put into all DL/ML algorithm development training and testing process.

While I realize this is not something specifically related to content in the course (my apologies if I am off-point in creating this thread), but it is my belief that in our role as practitioners/students/teachers in this space, we should take on some responsibility and discuss this issue openly and if possible find some consensus on best practices that could be better incorporated into our work/studies. How can we take preventive measures and/or make improvements to existing tools/processes?

Would be great to hear from everyone/anyone here including @jeremy and @rachel. I know this is something that Rachel, in particular, has done an amazing job already in calling attention to it. (btw follow her twitter if you haven’t already :slight_smile: )

15 Likes
2

(Rachel’s twitter handle is @math_rachel.)

3

Thanks for bringing this up. While we’re having so much fun learning and creating NNs, I think we should be seriously considering not only the obvious benefits of our work to society, but also the potential negative impact. IMHO, the biggest issue is the potential for AI to eliminate ~50% of existing jobs in the next few decades. A universal basic income (UBI) is one possible solution: https://futurism.com/images/universal-basic-income-answer-automation/

2 Likes
4

Wow! I just realized @jeremy appears in that linked infographic after I posted :slight_smile:

2 Likes
5
1 Like
6

Thanks for starting this thread!

Here are some talks I recommend:
FAT (Fairness, Accountability, Transparency) Conf Keynote by Latanya Sweeney
21 Definitions of Fairness by Arvind Narayanan

Here are links to a few of my blog posts/talks on this topic (also, I gave a keynote on bias in AI at QCon.ai a few weeks ago and will add the video link here once it’s available):

Facebook and privacy: it’s time to change the conversation
When Data Science Destabilizes Democracy and Facilitates Genocide
Bias in Word Embeddings workshop

9 Likes
7

I feel there are parallels here with gun control (if we think about it, guns are disruptive tech too). Deep Learning algorithms don’t generate fake news, people generate fake news. The algorithm doesn’t have any intent to deceive. We could snuff all tech than can be weaponized in a batman-esque paranoia, but “when has ever a tool not sharp enough to cut ourselves on been useful”?(JW Tukey)

I think it’s unfair to hold technology to a higher moral standard than humans to the point of stunting its growth. Might I coin the term - “techsist” while I’m at it.

Feel free to inform my ignorance if that’s how this comes across.

1 Like
8

Weeellllll the problem is that there have been a number of cases where technologists have given somebody an AI product and said, “here, this is good for you, use it” and it turned out to be bad for someone but hard to prove.

Sort of like if someone said, “Here’s a blanket, enjoy it!” Except that most of blankets turn out to be infected with measles. Most people wouldn’t notice because they are vaccinated against measles, and it’s a nice warm, snuggly blanket. But for the cohort of Americans born in 1963-1967 who got the inactivated measles virus, they are susceptible to the virus*. Imagine some small fraction of that cohort got measles from the blankets, with pretty bad consequences. It would be really hard to prove that the blankets were the problem, it would only harm a minority (the 63-67 cohort), and there would be resistance from people who really liked the blankets and had been using them for years with no ill effects.

If the blanket manufacturer said, “Well, you shouldn’t have been putting the blankets near your mouth and face and hands” or even “well, people born in 1963-67 shouldn’t have bought the blankets” wouldn’t absolve them of blame (especially if they had a monopoly on blankets" or “blankets don’t kill people, people using blankets kill themselves”. The manufacturer did not warn of measles,and the normal method of use turned out to be highly dangerous for some minority.

Now, this is a made-up example, but a lot of the examples given in class today are direct analogues. If a company says, “here’s an algorithm you can use to figure out if you should grant bail or parole, have fun”, and it discriminates against African-Americans, that’s pretty clear harm to African-Americans… but it’s hard to prove that it is discriminatory, it only harms a minority, and there would be resistance from judges and parole boards who really liked not having to think about bail and parole decisions (and had been using the AI for years with no ill effects to themselves).

*While the blankets are a made-up example, the susceptibility of the 1963-67 cohort is completely true, although many in that cohort do not realize it. That susceptibility is why there were such awful measles outbreaks on university campuses in ~1986, and why essentially all universities now insist on proof of measles vaccinations before allowing you to enroll.

2 Likes
9

On another technical side, one of the things that Cathy O’Neil speaks out against in her book “Weapons of Math Destruction” is the proliferation of proxy measures whose values and intentions drift when the algorithms in question are used for many years, and for underrepresented communities. This is a class of problem that has been lumped under the name ‘Goodhart’s Law’. Recently, researchers named David Manheim and Scott Garrabrant published work where he categorized four variants of this problem. I think this is important work and we should probably all read this: Categorizing Variants of Goodhart’s Law.

3 Likes
10

Thanks for that. I think I get some of the idea of what we might be responsible for as practitioners.

Loved the hypothetical measles blanket example. I think the warning really is against authority bias that black-box tech might provide. Somewhat still a human flaw though. The reaction would be to burn all blankets (measles ridden or not) if we tried to transfer the analogy to research. There is a less of the fear-of-the-unknown factor around blankets. On the other hand- what if these blankets were curing cancer and they took some people down doing it? There was an example of IBM and Nazis developing tech for management — are we also expected to stop using microwaves, antibiotics or space-tech?

Who is to say that these judges and parole officers don’t select these algorithms only because they confirm their biases? I understand the tragedy though. Then again - racism and sexism have predated technology. If one of these algorithms were predicting against the judges to send more white people to jail, chances are that racist judges would have written them off and not used technology.

Of course, I am not sure how to prevent this or if anything is provable. But I think the machine’s point of view can be lost in all this and it might end up taking the blame for someone who said “here…go use this” and also for someone who wanted a cure-all lazy solution and believed that person’s authority. The onus is on the humans and I think Jeremy was probably talking about a fine print that comes along with the model. But how far can we control our creation even if we warn people against its perils? And that leads into a whole discussion of patents…

1 Like
11

Thanks for the great discussion and resources linked so far!

I particularly liked @ducky’s measles blanket example - an engaging and powerful way to highlight unintended consequences, errors of commission vs omission, the added responsibilities (I would hope) that should come with monopoly power, among other themes.

I agree with you that the responsibility cannot rest on the technology itself. Technology has always been an uncomplaining scapegoat and an easy way to absolve oneself - will “I was just following the algorithm” be the new “I was just doing my job”?. The responsibility is always on us as the creators, implementers, regulators, and advocates.

So what can we do?

As excited as we are about pushing the State Of The Art on specific task performance in every domain, let us also be equally excited and pour equal thought and effort (at the minimum, and ideally much more since we’re much farther behind on this) into:

  • systems that expose and evaluate how our training techniques and performance metrics are flawed and open to bias. I.e. checking that datasets are diverse enough to represent the true population. As seen in lecture, even actions as simple as presenting data in crosstab to more clearly communicate where the biases are.
  • open, diverse, and thorough deliberation and policymaking into what should be considered ethical vs unethical forms of research and deployment into the world. I.e. Facebook selectively serving up negatively vs positively sentiment-ed content to 700,000 of their users to try to manipulate their emotions as a study of “emotional contagion” without informed consent or going through an institutional review board is unethical by any research standards I know of. Who and what should have been there to raise a flag, give pause, and help reconfigure the research into a more ethical approach? Even this work was subject to more scrutiny than usual since it was published in an academic journal - what about the mundane and subtly flawed algorithmic decisions made every day when optimizing for implicitly biased metrics?
  • building transparency tools (i.e. detectors of fake images/videos, better data provenance tracking), protective tools (i.e. ways of obfuscating one’s identity if permission to be identified is not explicitly given or required by law), and guardrails/failsafes that can halt runaway feedback loops and infuse human empathy, intuition, and values into otherwise algorithmic decision-making that have large, maybe irreversible consequences on human wellbeing (i.e. bail or no bail, hire or no hire, treat or no treat, kill or no kill).
  • I’m sure there are many more specific actions/examples we can and should add to this list…

I harbor no utopian thoughts that we will ever become perfect at “solving” the ethical dilemmas that arise with the application of any massively disruptive technology like AI/ML. Or preventing every adverse consequence (let alone spotting all of them). The immensity of the work and the ambiguity of its 1st, 2nd, 3rd order and so on effects doesn’t absolve us though. We can’t afford to rationalize away our involvement, bury our heads in the sand, or issue overblown warnings of calamity and doom that desensitize us to the real, pernicious, and deeply entrenched problems.

As vanguards and accelerators of these technologies, we have a special and added responsibility to take care. We should be stewards striving always to bend the arc of how our tools are used towards the good and just (borrowing from the MLK quote) and protecting those who are affected from the worst of the bad in each and every project we work on. This is the minimum thoughtfulness I would hope for of myself and every practitioner. The tools at our disposal and of our making have rapidly become so easily deployable, scalable, and powerful (far outpacing the development of any formal law or regulation which is reactive by nature) that holding ourselves responsible for any less would be negligent at best.

1 Like
12

I wrote a blog post a few months ago where I talked about why AI/ML scares me which is sort of related:

This blog talks about sentient malicious humans using weaponized AI, not about biases. Biases are bad, not saying they aren’t, I just am more scared of malicious humans.

(And this fear is actually the reason why I am learning about ML/AI.)

1 Like
13

New on Wired:

Cathy O’Neil, the author of “Weapons of Math Destruction” cited by @poppingtonic, has created a company to conduct external algorithm audits.

2 Likes