common pitfalls for studying the human side of machine ... · machine learning is a tool that...
TRANSCRIPT
![Page 1: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/1.jpg)
Common Pitfalls for Studying the Human Side of
Machine LearningJoshua A. Kroll, Nitin Kohli, Deirdre Mulligan
UC Berkeley School of InformationTutorial: NeurIPS 2018
3 December 2018
![Page 2: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/2.jpg)
Credit: Last Year, Solon Barocas and Moritz Hardt, "Fairness in Machine Learning", NeurIPS 2017
![Page 3: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/3.jpg)
MachineLearningFairness
![Page 4: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/4.jpg)
What goes wrong when engaging other disciplines?● Want to build technology people can trust and which supports human values● Demand for:
○ Fairness○ Accountability○ Transparency○ Interpretability
● These are rich concepts, with long histories, studied in many ways
● But these terms get re-used to mean different things!
○ This causes unnecessary misunderstanding and argument.
○ We’ll examine different ideas referenced by the same words, and examine some concrete cases
![Page 5: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/5.jpg)
Why this isn’t ethics
Machine learning is a tool that solves specific problems
Many concerns about computer systems arise not from people being unethical,but rather from misusing machine learning in a way that clouds the problem at hand
Discussions of ethics put the focus on the individual actors, sidestepping social, political, and organizational dynamics and incentives
![Page 6: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/6.jpg)
Definitions are unhelpful(but you still need them)
![Page 7: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/7.jpg)
Values Resist Definition
![Page 8: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/8.jpg)
Definitions aren’t for everyone:Where you sit is where you stand
![Page 9: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/9.jpg)
If we’re trying to capture human values, perhaps mathematical correctness isn’t enough
![Page 10: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/10.jpg)
These problems are sociotechnical problems
![Page 11: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/11.jpg)
Fairness
“What is the problem to which fair machine learning is the solution?” - Solon Barocas
![Page 12: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/12.jpg)
What is Fairness:Rules are not processes
![Page 13: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/13.jpg)
Tradeoffs are inevitable
![Page 14: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/14.jpg)
Maybe the Problem is Elsewhere
![Page 15: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/15.jpg)
What is Accountability:Understanding the Unit of
Analysis
![Page 16: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/16.jpg)
What should be true of a system, and where should we intervene on that
system to guarantee this?
![Page 17: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/17.jpg)
![Page 18: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/18.jpg)
![Page 19: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/19.jpg)
![Page 20: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/20.jpg)
![Page 21: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/21.jpg)
Transparency & Explainability are
Incomplete Solutions
![Page 22: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/22.jpg)
Transparency
![Page 23: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/23.jpg)
![Page 24: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/24.jpg)
![Page 25: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/25.jpg)
Explainability
![Page 26: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/26.jpg)
Explanations from Miller (2017)● Causal● Contrastive● Selective● Social● Both a product and a process
Miller, Tim. "Explanation in artificial intelligence: Insights from the social sciences." arXiv preprint arXiv:1706.07269 (2017).
![Page 27: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/27.jpg)
Data are not the truth
![Page 28: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/28.jpg)
![Page 29: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/29.jpg)
If length is hard to measure, what about unobservable
constructs like risk?
![Page 30: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/30.jpg)
Construct Validity
![Page 31: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/31.jpg)
Abstraction is a fiction
![Page 32: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/32.jpg)
There is no substitute for solving the problem
![Page 33: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/33.jpg)
You must first understand the problem
![Page 34: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/34.jpg)
Case One : Babysitter Risk Rating
![Page 35: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/35.jpg)
Xcorp launches a new service that uses social media data to predict whether a babysitter candidate is likely to abuse drugs or exhibit other undesirable tendencies (e.g. aggressiveness, disrespectfulness, etc.)
Using computational techniques, Xcorp will produce a score to rate the riskiness of the candidates. Candidates must opt in to being scored when asked by a potential employer.
This product produces a rating of the quality of the babysitter candidate from 1-5 and displays this to the hiring parent.
![Page 36: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/36.jpg)
With a partner, examine the validity of this approach.Why might this tool concern people, and who might
be concerned by it?
![Page 37: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/37.jpg)
What would it mean for this system to be fair?
![Page 38: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/38.jpg)
What would we need to make this system sufficiently transparent?
![Page 39: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/39.jpg)
Are concerns with this system solved by explaining outputs?
![Page 40: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/40.jpg)
Possible solutions?
![Page 41: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/41.jpg)
This is not hypothetical.
Read more here:https://www.washingtonpost.com/technology/2018/11/16/wante
d-perfect-babysitter-must-pass-ai-scan-respect-attitude/
![Page 42: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/42.jpg)
(Break)
![Page 43: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/43.jpg)
Case Two:Law Enforcement Face Recognition
![Page 44: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/44.jpg)
The police department in Yville wants to be able to identify criminal suspects in crime scene video to know if the suspect is known to detectives or has been arrested before.
Zcorp offers a cloud face recognition API, and the police build a system using this API which queries probe frames from crime scene video against the Yville Police mugshot database.
![Page 45: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/45.jpg)
What does the fact that this is a government application change about the requirements?
![Page 46: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/46.jpg)
What fairness equities are at stake in such a system?
![Page 47: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/47.jpg)
What is the role of transparency here?
![Page 48: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/48.jpg)
Who has responsibility in or for this system?What about for errors/mistakes?
![Page 49: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/49.jpg)
What form would explanations take in this system?
![Page 50: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/50.jpg)
This is not hypothetical, either.
Read more here:https://www.aclu.org/blog/privacy-technology/surveillance-technologies/amazons-face-recognition-falsely-matched-28
![Page 51: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/51.jpg)
To solve problems with machine learning, you must understand them
![Page 52: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/52.jpg)
Respect that others may define the problem
differently
![Page 53: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/53.jpg)
If we allow that our systems include people and society, it’s clear that we have to help negotiate values,
not simply define them.
![Page 54: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/54.jpg)
There is no substitute for thinking
![Page 55: Common Pitfalls for Studying the Human Side of Machine ... · Machine learning is a tool that solves specific problems Many concerns about computer systems arise not from people being](https://reader034.vdocuments.mx/reader034/viewer/2022042223/5ec9c12c4c1c21346b0b486c/html5/thumbnails/55.jpg)
Questions?