4bgfuz &ohjoffsjoh gps · 2019. 4. 24. · 4bgfuz &ohjoffsjoh gps "* 1. input space...
TRANSCRIPT
Exploratory Test Method
Exploratory Test Method consists of the following steps:
1. Input Space Definition
2. Test Case Generation
3. Test Execution in Automatic Test Loop.
4. Verification of Outputs – Coverage Measurement and Pass/Fail Decision by Metamorphic Relation
5. Statistical Risk Analysis
Especially, some AI systems using training methods such as deep learning require more systematic verification because
they have a tendency to give unexpected results for unknown cases that they have ever met in their training.
Unfortunately, it is difficult to apply existing testing techniques to trained AI systems because most of testing techniques
use requirements specification that define relations of input to output precisely. Note that trained AI systems are trained
with data and have no specification for input-output relations.
Recently, the number of high risk AI systems such as autonomous robot and autonomous driving cars is increasing. Even
though these AI systems make new business opportunities by providing unmatched performance and functionalities,
highly self control nature of these systems may cause drastic risks.
Safety Engineering for AI