Retrieving Encrypted Images Using Convolution Neural Network and Fully Homomorphic Encryption




CKKS, CNN, Content based image retrieval, Homomorphic encryption, Random forest


A content-based image retrieval (CBIR) is a technique used to retrieve images from an image database. However, the CBIR process suffers from less accuracy to retrieve images from an extensive image database and ensure the privacy of images. This paper aims to address the issues of accuracy utilizing deep learning techniques as the CNN method. Also, it provides the necessary privacy for images using fully homomorphic encryption methods by Cheon, Kim, Kim, and Song (CKKS). To achieve these aims, a system has been proposed, namely RCNN_CKKS, that includes two parts. The first part (offline processing) extracts automated high-level features based on a flatting layer in a convolutional neural network (CNN) and then stores these features in a new dataset. In the second part (online processing), the client sends the encrypted image to the server, which depends on the CNN model trained to extract features of the sent image. Next, the extracted features are compared with the stored features using a Hamming distance method to retrieve all similar images. Finally, the server encrypts all retrieved images and sends them to the client. Deep-learning results on plain images were 97.94% for classification and 98.94% for retriever images. At the same time, the NIST test was used to check the security of CKKS when applied to Canadian Institute for Advanced Research (CIFAR-10) dataset. Through these results, researchers conclude that deep learning is an effective method for image retrieval and that a CKKS method is appropriate for image privacy protection.


Download data is not yet available.


Rout NK, Atulkar M, Ahirwal MK. A review on content-based image retrieval system: Present trends and future challenges. Int J Comput Vis Robot. 2021; 11(5): 461-485. doi:10.1504/IJCVR.2021.117578

Murala S, Maheshwari RP, Balasubramanian R. Local tetra patterns: A new feature descriptor for content-based image retrieval. IEEE Trans Image Process. 2012; 21(5): 2874-2886. doi:10.1109/TIP.2012.2188809

Onoufriou G, Mayfield P, Leontidis G. Fully Homomorphically Encrypted Deep Learning as a Service. Mach Learn Knowl Extr. 2021; 3(4): 819-834. doi:10.3390/make3040041.

Denning DER. Cryptography and Data Security. 1982; Addison-Wesley Longman Publishing Co., Inc. Boston,MA United States.

Yassein HR, Al-Saidi NMG, Farhan AK. A new NTRU cryptosystem outperforms three highly secured NTRU-analog systems through an innovational algebraic structure. J Discret Math Sci Cryptogr. 2020;(June). doi:10.1080/09720529.2020.1741218

Armknecht F, Katzenbeisser S, Peter A. Group homomorphic encryption: Characterizations, impossibility results, and applications. Des Codes, Cryptogr. 2013;67(2):209-232. doi:10.1007/s10623-011-9601-2

Gentry C. A Fully Homomorphic Encryption Scheme. PhD [dissertation]Stanford Univ. 2009;(September).

Plantard T, Susilo W, Zhang Z. Fully homomorphic encryption using hidden ideal lattice. IEEE Trans Inf Forensics Secur. 2013; 8(12): 2127-2137. doi:10.1109/TIFS.2013.2287732

Chung H, Kim M, Al Badawi A, Aung KMM, Veeravalli B. Homomorphic comparison for point numbers with user-controllable precision and its applications. Symmetry (Basel). 2020; 12(5): 1-22. doi:10.3390/SYM12050788

Pedrouzo-Ulloa A, Troncoso-Pastoriza JR, Gama N, Georgieva M, Pérez-González F. Revisiting multivariate ring learning with errors and its applications on lattice-based cryptography. Mathematics. 2021; 9(8): 1-42. doi:10.3390/math9080858

Liu J, Wang C, Tu Z, Wang XA, Lin C, Li Z. Secure KNN Classification Scheme Based on Homomorphic Encryption for Cyberspace. Secur Commun Networks. 2021; 2021: 1-12. doi:10.1155/2021/8759922

Amine Boulemtafes, Abdelouahid Derhab YC. A review of privacy-preserving techniques for deep learning. Neurocomputing, Elsevier. 2020; 384(4): 21-45. doi:10.1016/j.neucom.2019.11.041

Kadhim AF, Kamal ZA. Generating dynamic S-BOX based on Particle Swarm Optimization and Chaos Theory for AES. Iraqi J Sci. 2018; 59(3): 1733-1745. doi:10.24996/IJS.2018.59.3C.18

Xu Y, Zhao X, Gong J. A Large-Scale Secure Image Retrieval Method in Cloud Environment. IEEE Access. 2019; 7: 160082-160090. doi:10.1109/ACCESS.2019.2951175

Xia Z, Ma X, Shen Z, Sun X, Xiong NN. Secure Image LBP Feature Extraction in Cloud- based Smart Campus. IEEE Access. 2018; PP(c): 1. doi:10.1109/ACCESS.2018.2845456

Cao Z, Mu S, Xu Y, Dong M. Image retrieval method based on CNN and dimension reduction. In 2018. Int. Conf. Secur. Pattern Anal. Cybern, SPAC 2018; 2018: 441-445. doi:10.1109/SPAC46244.2018.8965601

Verma M, Raman B. Local neighborhood difference pattern : A new feature descriptor for natural and texture image retrieval. Multimed Tools Appl Springer Sci. 2018; 77: 11843–11866. doi:10.1007/s11042-017-4834-3

Selvam S, Kannan ST. A New Architecture for Image Retrieval Optimization with HARP Algorithm. Asian J Comput Sci Technol. 2017; 6(1): 1-5.

Du A, Wang L, Cheng S, Ao N. A privacy-protected image retrieval scheme for fast and secure image search. Symmetry. 2020; 12(2):1-17. doi:

Khokher A, Talwar R. Content-based image retrieval at the end of the early years. IEEE Trans Pattern Anal Mach Intell. 2000;22(12):1349-1380.

He Q, He H. A novel method to enhance sustainable systems security in cloud computing based on the combination of encryption and data mining. Sustain. 2021;13(1):1-17. doi:10.3390/su13010101

Kuo CH, Chou YH, Chang PC. Using deep convolutional neural networks for image retrieval. In: Visual Information Processing and Communication. IS&T Int. Symp. Electron. Imaging Sci Technol. 2016: 1-6. doi:10.2352/ISSN.2470-1173.2016.2.VIPC-231

Huang HK, Chiu CF, Kuo CH, Wu YC, Chu NNY, Chang PC. Mixture of deep CNN-based ensemble model for image retrieval. 2016 IEEE 5th Glob Conf Consum Electron GCCE 2016. 2016; (2): 5-6. doi:10.1109/GCCE.2016.7800375

Khan UA, Javed A, Ashraf R. An effective hybrid framework for content based image retrieval (CBIR). Multimed Tools Appl. 2021; 80(17): 26911-26937. doi:10.1007/s11042-021-10530-x

Ali F, Mohammed AH. Content Based Image Retrieval (CBIR) by statistical methods. Baghdad Sci J. 2020;17:694-700. doi:10.21123/bsj.2020.17.2(SI).0694

Challa RK, Gunta VK. A modified symmetric key fully homomorphic encryption scheme based on Read-Muller Code. Baghdad Sci J. 2021; 18(2): 899-906. doi:10.21123/bsj.2021.18.2(Suppl.).0899

Syed D, Refaat SS, Bouhali O. Privacy preservation of data-driven models in smart grids using homomorphic encryption. Inf. 2020;11(7):1-17. doi:10.3390/info11070357

Lou Q, Jiang L. SHE: A fast and accurate deep neural network for encrypted data. Adv Neural Inf Process Syst. 2019; 32(NeurIPS):1-9.

Obla S, Gong X, Aloufi A, Hu P, Takabi D. Effective Activation Functions for Homomorphic Evaluation of Deep Neural Networks. IEEE Access. 2020; 8: 153098-153112. doi:10.1109/ACCESS.2020.3017436

Kwabena OA, Qin Z, Qin Z, Zhuang T. MSCryptoNet: Multi-Scheme Privacy-Preserving Deep Learning in Cloud Computing. IEEE Access. 2019; 7: 29344-29354. doi:10.1109/ACCESS.2019.2901219

Clet P-E, Stan O, Zuber M. BFV, CKKS, TFHE: Which One Is the Best for a Secure Neural Network Evaluation in the Cloud? Springer International Publishing; 2021. doi:10.1007/978-3-030-81645-2_16

Zhang Q, Zhang M, Chen T, Sun Z, Ma Y, Yu B. Recent advances in convolutional neural network acceleration. Neurocomputing. 2019; 323: 37-51. doi:10.1016/j.neucom.2018.09.038

Tzelepi M, Tefas A. Deep convolutional learning for Content Based Image Retrieval. Neurocomputing. 2018; 275: 2467-2478. doi:10.1016/j.neucom.2017.11.022

Bologna G. A Simple Convolutional Neural Network with Rule Extraction. Appl Sci. 2019; 9(12): 2411. doi:10.3390/app9122411

Zaid Khalaf Hussien BND. Anomaly Detection Approach Based on Deep Neural Network and Dropout. Baghdad Sci J. 2020; 17:701-709.

Will MA, Ko RKL. A Guide to Homomorphic Encryption. Elsevier Inc. 2015. doi:10.1016/B978-0-12-801595-7.00005-7

Shrestha R, Kim S, Integration of IoT with Blockchain and Homomorphic Encryption: Challenging Issues and Opportunities. Adv Comput. 2019; 115. 1st ed. Elsevier Inc. doi:10.1016/bs.adcom.2019.06.002

Hee CJ, Andrey K, Miran K, Yongsoo S. Homomorphic Encryption for Arithmetic of Approximate Numbers. In: Inte Conf T Appl Crypt and Inf S Springer. 2017: 409–437. doi:10.1007/978-3-319-78381-9_14