Defect-based Testing for Safety-critical ML Components

Self-driving cars have the potential to revolutionize transportation, but ensuring their safety is paramount. A key challenge lies in guaranteeing "Safety of the Intended This white paper introduces a novel process for collecting adequate test data for machine learning (ML) components, drawing inspiration from defect-based software testing. It addresses the challenge of testing ML components in safety-critical applications, where the input space is complex and exhaustive testing is impractical. We propose a systematic approach that leverages ML data quality metrics and methods to enhance test data and uncover critical scenarios. The effectiveness of the proposed process is demonstrated through case studies involving stop sign recognition and railway track segmentation.