Testing deep neural networks for classification tasks through adversarial perturbations on test datasets | |
| Author | Ponakala, Rajasekhar |
| Call Number | AIT RSPR no.ICT-19-06 |
| Subject(s) | Machine learning Neural networks (Computer science) |
| Note | A research submitted in partial fulfillment of the requirements for the degree of Master of Engineering in Information and Communication Technologies, School of Engineering and Technology |
| Publisher | Asian Institute of Technology |
| Series Statement | Research studies project report ; no. ICT-19-06 |
| Abstract | Deep learning models often provide high predictive accuracy on test datasets. However, sometimes, changes in the real-world distribution of the data in production may result in incorrect functioning. Such problems have motivated research in the safety and security-critical systems domains. In this research study, I test CNN models for vehicle type classification from the AIT VISION Lab. The current vehicle type model is a GoogLeNet model trained on the task of classifying an input image as a car, van, bus, truck, or pickup truck. I used FGSM to adversarially perturb the GoogLeNet model to identify error cases. To make the model testing approach more useful in a software development context, I developed a method to include adversarial testing of models in a continuous integration (CI) framework. The method used a plugin for the Jenkins open source CI framework to visualize results. With the plugin, we can visualize the build logs for standard and adversarially trained classifiers. This method can help the software engineering team to analyze the performance of model. With this method, any kind of dataset can be used to generate adversarial examples, enabling us to detect model degradation and improve their expected error behavior under unexpected circumstances. |
| Year | 2019 |
| Corresponding Series Added Entry | Asian Institute of Technology. Research studies project report ; no. ICT-19-06 |
| Type | Research Study Project Report (RSPR) |
| School | School of Engineering and Technology (SET) |
| Department | Department of Information and Communications Technologies (DICT) |
| Academic Program/FoS | Information and Communication Technology (ICT) |
| Chairperson(s) | Dailey, Matthew N.; |
| Examination Committee(s) | Mongkol Ekpanyapong;Attaphongse Taparugssanagorn; |
| Scholarship Donor(s) | Asian Institute of Technology Fellowship; |
| Degree | Research Studies Project Report (M. Eng.) - Asian Institute of Technology, 2019 |