![Page 1: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/1.jpg)
The case of theThe case of theKiller RobotKiller Robot
byby
Richard EpsteinRichard Epstein
As retold byAs retold by
Netiva CaftoriNetiva CaftoriNEIUNEIU
Oct. 2004Oct. 2004
![Page 2: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/2.jpg)
When software engineering and When software engineering and computer ethics collidecomputer ethics collide
![Page 3: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/3.jpg)
The newsThe news
A robot operator, Bart A robot operator, Bart Matthews, was killed Matthews, was killed by his robot, Robbie by his robot, Robbie CX30, while at work.CX30, while at work.
A programmer, Randy A programmer, Randy Samuels, who wrote Samuels, who wrote the faulty code was the faulty code was indicted for indicted for manslaughter.manslaughter.
Silicon Techchronics is Silicon Techchronics is Randy’s employer.Randy’s employer.
![Page 4: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/4.jpg)
What really happened?What really happened?
The robot The robot malfunctioned & malfunctioned & crushed its crushed its operator to deathoperator to death
Blood all overBlood all over Decapitated headDecapitated head Num-lock key light Num-lock key light
not onnot on Numeric key pad Numeric key pad
bloodybloody
![Page 5: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/5.jpg)
Who is at fault?Who is at fault?
The programmer The programmer who made a who made a careless mistake?careless mistake?
The designer?The designer? The robotics The robotics
division?division? The company?The company? The operator of the The operator of the
robot?robot? The robot?The robot?
![Page 6: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/6.jpg)
Robbie CX 10 & 20 were Robbie CX 10 & 20 were experimental.experimental.
The robotics division chief, Johnson, put The robotics division chief, Johnson, put pressure on the project manager Reynolds pressure on the project manager Reynolds to finish the project by January.to finish the project by January.
Johnson hired 20 new programmers in June Johnson hired 20 new programmers in June against Reynolds’ will, by shifting against Reynolds’ will, by shifting resources. Robbie CX 30 had to succeed.resources. Robbie CX 30 had to succeed.
Johnson only knew manufacturing Johnson only knew manufacturing hardware. One cannot speed software hardware. One cannot speed software coding by adding more programmers.coding by adding more programmers.
Robbie CX 30 was a step ahead in Robbie CX 30 was a step ahead in sophistication.sophistication.
Johnson: Perfect software is an oxymoron.Johnson: Perfect software is an oxymoron.
![Page 7: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/7.jpg)
Programmer personalityProgrammer personality Randy’s home page: Randy’s home page:
freedom for freedom for programmersprogrammers
A hacker typeA hacker type Enormous stressEnormous stress Programmer was a Programmer was a
prima donna: could prima donna: could not accept criticism or not accept criticism or his own fallibilityhis own fallibility
Helpful but arrogantHelpful but arrogant Under pressure to Under pressure to
finish project on timefinish project on time
![Page 8: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/8.jpg)
A good programming teamA good programming team
Needs to have a whole array of Needs to have a whole array of personalities types including:personalities types including:
An interaction-oriented personAn interaction-oriented person Someone who keeps the peace and Someone who keeps the peace and
helps move things in a positive helps move things in a positive directiondirection
Task-oriented people, but not allTask-oriented people, but not all A democratic teamA democratic team Egoless programmingEgoless programming
![Page 9: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/9.jpg)
Team dynamicsTeam dynamics
The project was controversial from the The project was controversial from the beginningbeginning
Jan Anderson, a programmer, was fired Jan Anderson, a programmer, was fired after she attacked project manager after she attacked project manager Reynolds on his decision to use the Reynolds on his decision to use the waterfall methodology versus the waterfall methodology versus the prototype methodology.prototype methodology.
Reynolds was replacing a dead project Reynolds was replacing a dead project manager as a cost saving measure instead manager as a cost saving measure instead of hiring someone from outside the of hiring someone from outside the company. He never worked with robotics company. He never worked with robotics before.before.
![Page 10: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/10.jpg)
More explanationsMore explanations
The earlier Robbie models processed The earlier Robbie models processed transactions. Robbie CX30 was to transactions. Robbie CX30 was to interact with its operator, therefore interact with its operator, therefore the interface cannot be designed as the interface cannot be designed as an afterthought. Thus, a prototyping an afterthought. Thus, a prototyping model is preferable where the users model is preferable where the users can use a prototype robot while in can use a prototype robot while in the design stage.the design stage.
![Page 11: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/11.jpg)
Software development life-cycleSoftware development life-cycleWater fall methodologyWater fall methodology
RequirementsRequirements AnalysisAnalysis DesignDesign CodingCoding TestingTesting ImplementationImplementation MaintenanceMaintenance
No standard No standard practices practices
![Page 12: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/12.jpg)
Quality controlQuality controlor quality assuranceor quality assurance
Dynamic testingDynamic testing Static testingStatic testing Unit testing: Black and white box Unit testing: Black and white box
testingtesting Integration testingIntegration testing
Prof Silber attested that test results Prof Silber attested that test results were inconsistent with actual killer were inconsistent with actual killer code test resultscode test results
![Page 13: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/13.jpg)
When is the software good When is the software good enough?enough?
Errors can happen in any stageErrors can happen in any stage Design stage: 2 alternatives:Design stage: 2 alternatives:Waterfall & prototypingWaterfall & prototyping TestingTesting User interfaceUser interface The complexity of the task of The complexity of the task of
building real-world softwarebuilding real-world software Enormous stressEnormous stress
![Page 14: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/14.jpg)
The company-Sili-TechThe company-Sili-Tech Two environments: Two environments:
Worker-friendly: Chip Worker-friendly: Chip Creek facility. Creek facility. Prevent repetitive Prevent repetitive strain injuries.strain injuries.
Special training for Special training for employees.employees.
Well designed Well designed workstations.workstations.
Frequent breaks.Frequent breaks.
Worker-unfriendly: Worker-unfriendly: Silicon Valley plant.Silicon Valley plant.
Workers criticized Workers criticized for novelties.for novelties.
No exercises or No exercises or training training encouraged.encouraged.
RSI frequentRSI frequent More compensation More compensation
claimsclaims
![Page 15: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/15.jpg)
Law suitsLaw suits
The wife of the The wife of the murdered operator.murdered operator.
The indicted The indicted programmer programmer himself.himself.
More possible More possible indictments.indictments.
![Page 16: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/16.jpg)
Why was Randy Samuels indicted?Why was Randy Samuels indicted?
Sili-Tech promised to deliver robots that would Sili-Tech promised to deliver robots that would cause no bodily harm.cause no bodily harm.
So Randy was not legally responsible for the So Randy was not legally responsible for the death of the robot operator.death of the robot operator.
Waterson, the president of Sili-Tech has Waterson, the president of Sili-Tech has contributed large sums to the re-election of judge contributed large sums to the re-election of judge McMurdock, who indicted Randy.McMurdock, who indicted Randy.
Functional requirements specify the behavior of Functional requirements specify the behavior of the robot under exceptional conditions. Operator the robot under exceptional conditions. Operator intervention may be needed.intervention may be needed.
Exceptional conditions were not mentioned in Exceptional conditions were not mentioned in training of operators.training of operators.
![Page 17: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/17.jpg)
The charactersThe characters
Jane
Cindy
Prof SilberProf Gritty
Bart
![Page 18: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/18.jpg)
E-mail was not secure at Sili-TechE-mail was not secure at Sili-TechIt was found that:It was found that:
Cindy Yardley faked the test results Cindy Yardley faked the test results of the Killer codeof the Killer code
Johnson told Cindy that the robot was Johnson told Cindy that the robot was safe and all will lose their jobs if the safe and all will lose their jobs if the robot were not shipped on timerobot were not shipped on time
Randy stole some of his software, but Randy stole some of his software, but admitted he had bugs in his Killer admitted he had bugs in his Killer codecode
![Page 19: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/19.jpg)
Ethical issuesEthical issues
Email privacyEmail privacy Professional ethics: Professional ethics:
delivering bug-free delivering bug-free softwaresoftware
Plagiarism-Plagiarism-intellectual intellectual propertyproperty
Honor Honor confidentialityconfidentiality
ACM code of ethicsACM code of ethics
![Page 20: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/20.jpg)
![Page 21: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/21.jpg)
Five ethical tests:Five ethical tests:by Kallman & Grilloby Kallman & Grillo
The mom test: would you tell your The mom test: would you tell your mother?mother?
The TV test: would you tell your story The TV test: would you tell your story on national TV?on national TV?
The smell test: does it smell bad?The smell test: does it smell bad? The other person’s shoes test: would The other person’s shoes test: would
you like it if done to you?you like it if done to you? The market test: would your action The market test: would your action
make a good sale pitch?make a good sale pitch?
![Page 22: The case of the Killer Robot by Richard Epstein](https://reader035.vdocuments.mx/reader035/viewer/2022062423/56814842550346895db55852/html5/thumbnails/22.jpg)