Abstract: Widely applied in various fields, deep learning (DL) is becoming the key driving force in the industry. Although it has achieved great success in artificial intelligence tasks, similar to traditional software, it has defects involving unpredictable accidents and losses due to failure. To ensure the quality of DL software, adequate testing needs to be carried out. In this paper, we propose a test set augmentation technique based on an adversarial example generation algorithm for image classification deep neural networks (DNNs). It can generate a large number of useful test cases, especially when test cases are insufficient. We briefly introduce the adversarial example generation algorithm and implement the framework of our method. We conduct experiments on classic DNN models and datasets. We further evaluate the test set by using a coverage metric based on states of the DNN.

Key words: DNN, defects, adversarial examples, coverage metric, test case generation