Method of Metamorphic Testing for Image Recognition System Based on GAN

Abstract

Abstract: Image recognition is a key research field of image processing. Under the influence of the difficult determination of test results and the unbalanced category of data set samples, the test technology suitable for the robustness and stability of image recognition systems is deficient. In order to effectively test the image recognition system, this paper applies metamorphosis test method into testing process of image recognition system, generates derived data close to reality based on Generative Adversarial Network to construct a metamorphic relationship suitable for the image recognition system, introduces derivative image quality verification methods and automatic judgment methods of test results to construc a metamorphic testing framework process for image recognition systems. Finally, the feasibility and effectiveness of this method are verified through a case study of vehicle identification. The experimental results show that the method can detect the inconsistent behavior of the image recognition system in different scenes and can evaluate the robustness of system effectively.

Key words: metamorphic testing, image recognition, GAN(Generative Adversarial Network)

JIANG Jing-jie, XU Luo, LI Ning. Method of Metamorphic Testing for Image Recognition System Based on GAN[J]. Computer and Modernization, 2021, 0(02): 24-29.

References ［24］

［1］	孙瑜阳. 深度学习及其在图像分类识别中的研究综述［J］. 信息技术与信息化, 2018(1):138-140.
［2］	蒋树强,闵巍庆,王树徽. 面向智能交互的图像识别技术综述与展望［J］. 计算机研究与发展, 2016,53(1):113-122.
［3］	徐彩云. 图像识别技术研究综述［J］. 电脑知识与技术, 2013(10):2446-2447.
［4］	WEYUKER E J. On testing non-testable programs［J］. Computer Journal, 1982,25(4):465-470.
［5］	CHEN T Y. Metamorphic testing: A simple method for alleviating the test oracle problem［C］// Proceedings of the 10th International Workshop on Automation of Software TestMay. 2015:53-54.
［6］	TAO Q M, WU W, ZHAO C, et al. An automatic testing approach for compiler based on metamorphic testing technique［C］// Proceedings of the 2010 Asia Pacific Software Engineering Conference. 2010:270-279.
［7］	LU X L, DONG Y W. Testing of service-oriented software: A metamorphic testing approach［J］. Journal of Computer Applications, 2011,31(7):1756-1758.
［8］	ZHOU Z Q, XIANG S, CHEN T Y. Metamorphic testing for software quality assessment: A study of search engines［J］. IEEE Transactions on Software Engineering, 2016,42(3):264-284.
［9］	〖JP4〗MURPHY C, KAISER G E, HU L F, et al. Properties of machine learning applications for use in metamorphic testing［C］// Proceedings of the 20th International Conference on Software Engineering & Knowledge Engineering. 2008:867-872.
［10］	XIE X Y, HO J W K, MURPHY C, et al. Testing and validating machine learning classifiers by metamorphic testing［J］. Journal of Systems and Software, 2011,84(4):544-558.
［11］	XIE X Y, ZHANG Z Y, CHEN T Y, et al. METTLE: A metamorphic testing approach to validating unsupervised machine learning methods［J］. IEEE Transactions on Reliability, 2018,69(4):1293-1322.
［12］	DING J H, KANG X J, HU X H. Validating a deep learning framework by metamorphic testing［C］// Proceedings of the 2nd International Workshop on Metamorphic Testing. 2017:28-34.
［13］	DWARAKANATH A, AHUJA M, SIKAND S, et al. Identifying implementation bugs in machine learning based image classifiers using metamorphic testing［C］// The 27th ACM SIGSOFT International Symposium. 2018:118-128.
［14］	ZHANG M S, ZHANG Y Q, ZHANG L M, et al. DeepRoad: GAN-based metamorphic testing and input validation framework for autonomous driving systems［C］// Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering. 2018:132-142.
［15］	百度AI开放平台. 车辆检测服务［EB/OL］. ［2020-05-20］. https://ai.baidu.com/tech/vehicle/detect.
［16］	京东人工智能开放平台. 车辆检测服务［EB/OL］. ［2020-05-20］. http://neuhub.jd.com/market/api/266.
［17］	XU L M, TOWEY D, FRENCH A P, et al. Enhancing supervised classifications with metamorphic relations［C］// Proceedings of the 3rd International Workshop on Metamorphic Testing. 2018:46-53.
［18］	王坤峰,苟超,段艳杰,等. 生成式对抗网络GAN的研究进展与展望［J］. 自动化学报, 2017,43(3):321-332.
［19］	马春光,郭瑶瑶,武朋,等. 生成式对抗网络图像增强研究综述［J］. 信息网络安全, 2019(5):10-21.
［20］	HUANG X, LIU M Y, BELONGIE S, et al. Multimodal unsupervised image-to-image translation［C］// European Conference on Computer Vision. 2018:179-196.
［21］	SUN S Q, ZOU H W, ZHAO L T, et al. Research on detection box optimization of joint feature similarity measurement and intersection over union［J］. Computer Knowledge and Technology, 2019,15(29):190-193.
［22］	史忠科,曹力. 交通图像检测与分析(精)［M］. 北京：科学出版社, 2007.
［23］	曹蒙蒙. 基于图像内容的交通车辆识别关键技术［D］. 北京:华北电力大学, 2015.
［24］	YU F, CHEN H F, WANG X, et al. BDD100K: A diverse driving dataset for heterogeneous multitask learning［C］// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2020:2633-2642.

[1]	LI Shi-da, XIANG Jian-wen. A Weakened Joint Reinforcement Method to Improve Robustness of Image Recognition Models [J]. Computer and Modernization, 2023, 0(10): 70-76.
[2]	LI Shi-qiu. A Collaborative Representation-based Method of Discriminant Locality Preserving Projections [J]. Computer and Modernization, 2023, 0(06): 43-47.
[3]	ZHU Li-qing, LI Xiang, . Vehicle Detection of Remote Sensing Images Based on Improved YOLOv5 Algorithm [J]. Computer and Modernization, 2023, 0(05): 117-121.
[4]	MA Dong-qun, LI Bao-lin, WANG Qiu-yue, HE Xian-bo. A Method to Identify Diseases of Bridges Based on Transverse Cracks [J]. Computer and Modernization, 2021, 0(01): 43-49.
[5]	HE Tao, CHEN Jian, WEN Ying-you. HEp-2 Image Recognition Based on Deep Residual Shrinkage Network [J]. Computer and Modernization, 2021, 0(01): 38-42.
[6]	FAN Di, JU Zhi-yong. Method of Small Sample Image Recognition Based on Prototype Network [J]. Computer and Modernization, 2020, 0(03): 103-.
[7]	ZHOU Chen-yi, FENG Yu, XU Yi-bai, LU Shan. Fine-grained Image Recognition Based on Deep Neural Network #br# with Amplified Multi-attention Mechanism [J]. Computer and Modernization, 2019, 0(09): 83-.
[8]	MIAO Kai-chao1,2, WANG Chuan-hui2, ZHANG Ya-li2, ZHOU Jian-ping2, LIU Cheng-xiao2, YAO Ye-qing2. Road Visibility Estimation Method Based on AlexNet Algorithm [J]. Computer and Modernization, 2019, 0(06): 87-.
[9]	LI Chunli， LIU Zhendong， HUI Kanghua. Boundary Sample Selection Method Based on Cosine Similarity [J]. Computer and Modernization, 2017, 0(8): 66-.
[10]	LIN Qin-zhuang1, ZHONG Ying-chun1, LUO Wei-shi2. Recognition of Region of Interest from Facial Nerve Image Sequences [J]. Computer and Modernization, 2017, 0(11): 51-54.
[11]	ZENG Jin-wei1, ZHANG Peng-cheng1, LI Wen-rui2， ZHOU Xin-li3 . A Multi-label Metamorphic Relations Prediction Approach Based on RBF Neural Network [J]. Computer and Modernization, 2016, 0(12): 1-6,11.
[12]	YANG Ning. Multi-GPU Parallel Framework of Deep Convolutional Neural Networks [J]. Computer and Modernization, 2016, 0(11): 95-98.
[13]	ZHONG Zhi-wei;XU Gui-li;WANG Biao;GUO Rui-peng;TIAN Yu-peng;LI Kai-yu. Research on 3D Object Recognition Algorithm Based on PCA and Moment Invariant [J]. Computer and Modernization, 2013, 1(3): 5-8.
[14]	MAO Ya-qiang. Application of Image Recognition Technique in 3D-RPG Game Development [J]. Computer and Modernization, 2012, 1(1): 78-82.
[15]	YIN Hui-ling. Paper Currency Recognition Based on Feature Extraction of One Dimension Fourier Transform [J]. Computer and Modernization, 2011, 7(7): 34-35,3.