Towards Practical Robustness Analysis for DNNs based on PAC-Model Learning
Paper Synopsis
We introduce a novel framework for analyzing the local robustness of Deep Neural Networks (DNNs) using PAC-model learning. Our method abstracts the local behavior of DNNs into affine models with probably approximately correct (PAC) guarantees, enabling robust analysis. The framework, implemented in a tool named DeepPAC, integrates model learning into PAC robustness evaluation, offering a more faithful and accurate analysis compared to existing statistical methods. DeepPAC is designed to scale efficiently for complex DNN structures and high-dimensional inputs. Through extensive experiments on datasets like MNIST, CIFAR-10, and ImageNet, we demonstrate superior precision and practicality over state-of-the-art methods such as PROVERO and ERAN, while maintaining consistency with adversarial testing results like DeepGini. This work highlights DeepPAC as a practical and reliable approach for DNN robustness verification, even in large-scale real-world applications. Future directions include exploring more sophisticated PAC models and combining verification with testing techniques.
Cite the Paper
Renjue Li, Pengfei Yang, Cheng-Chao Huang, Youcheng Sun, Bai Xue, Lijun Zhang: Towards Practical Robustness Analysis for DNNs based on PAC-Model Learning. In 44th IEEE/ACM 44th International Conference on Software Engineering, ICSE 2022, Pittsburgh, PA, USA, May 25-27, 2022, pages 2189-2201, 2022. DOI BIB