Int J Performability Eng ›› 2019, Vol. 15 ›› Issue (7): 1998-2007.doi: 10.23940/ijpe.19.07.p27.19982007

Previous Articles     Next Articles

Test Set Augmentation Technique for Deep Learning Image Classifiers

Qiang Chen, Zhanwei Hui*, and Jialuo Liu   

  1. Command and Control Engineering College, Army Engineering University of PLA, Nanjing, 210007, China
  • Submitted on ;
  • Contact: * E-mail address:
  • Supported by:
    This research is supported by the National Key R&D Program of China (No. 2018YFB1403400).

Abstract: Widely applied in various fields, deep learning (DL) is becoming the key driving force in the industry. Although it has achieved great success in artificial intelligence tasks, similar to traditional software, it has defects involving unpredictable accidents and losses due to failure. To ensure the quality of DL software, adequate testing needs to be carried out. In this paper, we propose a test set augmentation technique based on an adversarial example generation algorithm for image classification deep neural networks (DNNs). It can generate a large number of useful test cases, especially when test cases are insufficient. We briefly introduce the adversarial example generation algorithm and implement the framework of our method. We conduct experiments on classic DNN models and datasets. We further evaluate the test set by using a coverage metric based on states of the DNN.

Key words: DNN, defects, adversarial examples, coverage metric, test case generation