Download - CRISP evaluation using the STEFi approach
CRISP Evaluation
16th of March 2017, CoU Meeting, Brussels
Dr Nathalie Hirschmann (Center for Technology and Society – Technische Universität Berlin)
EVALUATION
CRISP Methodology
Source: Deliverable 5.2. February 2016
Information provider
Audit and Inspection
review & decision
Auditor (third party)
EVALUATION CERTIFICATION
SurveillanceAttestationS-T-E-FiAssessment
Configuration selection and determination
R3R2R1
2
Systematisation of criteria
Video surveillance
3
performance
accuracy
robustness
fair distribution
accountability
withdrawal mechanisms
end user safety
system interference
risk
user error protection
environment
awarenesstransparent use
feeling of uneasinessphysiological invasiveness
reliability
maintenance resources
usability
ethical codes
good practice/safety codes
documentation protective measures to ensure efficiencyusability
user manual
customisation
interoperability
lifecycle costs
energy efficiency
personal data
prohibition of discrimination
due process
Systematisation of criteria
Video surveillance
4
performance
accuracyrobustness
SECURITYfair distribution
accountability withdrawal mechanismsend user safety
system interferencerisk
user error protection
environment
awarenesstransparent use TRUST
feeling of uneasiness
physiological invasiveness
reliability
maintenance resources
usability
ethical codesgood practice/safety codes
documentationprotective measures to ensure efficiency
usability
user manual
customisationEFFICIENCYinteroperability
lifecycle costsenergy efficiency personal data
prohibition of discrimination
due process
FREEDOMINFRINGEMENT
Video surveillance
5
Police authority
Private securit
yPolitical actor
…
SECURITYCitizen
Employee
Political actor
…
TRUST
Municipality
Political actor
…Infrastructure operator
EFFICIENCYData protection expert
NGO‘s
Political actor
…Lawyer
sINFRINGEMENT
FREEDOM
S-T-E-Fi experts
‘Information provider’
S-T-E-FI ASSESSMENT
CONFIGURATION
R1 R2
CERTIFICATION
10
Evaluation process
Video surveillance
‘Information provider’
S-T-E-FI -ASSESSMENT
CONFIGURATION
R2
1. Specification of the security area
Client informs about:
2. Detailed technical specifications of the system
3. Basic information on at least three application scenarios
11
Evaluation process – configuration
Video surveillance
1. Reliable answering of all assessment questions by the project leader and ‘appointed experts’
2. Identifying conflicts between and within S-T-E-Fi dimensions
3. Conflict resolution
R 1
‘Information provider’
S-T-E-FI -ASSESSMENT
12
Evaluation process – link between configuration and assessment
Video surveillance
R 1
13
Evaluation process – assessment
Video surveillance
‘Information provider’
S-T-E-FI -ASSESSMENT
R1
1. Reliable answering of all assessment questions by the project leader and ‘appointed experts’2. Identifying conflicts between and within S-T-E-Fi dimensions
3. Conflict resolution
R 2
14
Assessment: report R2
R 2
Summary of how the process was conducted
Range of stakeholders involved
Information gathered during configuration
Selected assessment questions
Results of the assessment questions
Type and number of identified conflicts
Conflict resolution
Potentially remaining, unsolved conflicts
‘Information provider’
S-T-E-FI -ASSESSMENT
CONFIGURATION
R1 R2
1. Specification of the security area
Client informs about:
2. Detailed technical specifications of the system
3. Basic information on at least three application scenarios
CERTIFICATION
15
From evaluation to certification
Video surveillance
1. Reliable answering of all assessment questions by the project leader and ‘appointed experts’2. Identifying conflicts between and within
S-T-E-Fi dimensions3. Conflict resolution
18
Assessment criteria – example
“Yes” or “No” Is the personnel trained
to increase public awareness on the
possible impacts of the system?
What kind of training does the personnel
receive and how often?
QUESTIONS:
T R U S TAwareness, Training
e.g. “Our personnel is trained on the system in the field every year. This is recorded in the personnel files.”
P R O O F
19
Conflict identification – example
RESPONSE OPTIONS NO YES
NO C- (0) C+ (1)
YES C- (0) C- (0)
Criterion A: ‘Observability’ (T)‘Are people constantly observed by the system?’
Criterion B: ‘Transparency’
(T)‘Is the system clear on what it offers?’
C – (0): no conflict; C+ (1): conflict; needs to be addressed.
24
Identification of conflicts
25
Conflict resolution
1. Ensuring compliance with applicable legal requirements
2. Implementation of technical changes to the security system and/or implementation of changes to the applied operating procedures
3. Negotiating a solution by following a practical and legitimate decision-making process
participation & systematisation: enabling/encouraging early stakeholder participation; assessment criteria are systematised according to four dimensions
systemic view: transferring assessment criteria into a matrix structure; identifying conflicts between/within dimensions and making them visible
conflict resolution: indicating identified conflicts which need to be solved to involved parties in order to move on with certification
S
Fi
T
E
27
CRISP’s approach
28
For further questions, please contact:
Dr. Leon HempelZentrum Technik und Gesellschaft – TU BerlinHardenbergstr. 16-1810623 Berlin – [email protected]