Deep Learning Function Implications For Handwritten Character Recognition Using Improved Mean Square Error Function
DOI:
https://doi.org/10.21123/bsj.2024.10024Keywords:
Loss functions; Machine learning; pattern recognition; Handwritten character recognition; Deep learning; Mean Square ErrorAbstract
Loss functions are used for estimating the accuracy of the classification models in Machine Learning. The error loss value measures whether the predicted labels match the true labels or are close to them. Handwritten character recognition is a pattern recognition problem. The progression of pattern recognition applications for offline image classification is rapidly improved by using deep learning techniques. Convolution Neural Networks (CNN) and the activation function are the most commonly used in deep learning. In addition, different loss functions are used to evaluate the performance of the model such as Categorical Cross-Entropy (CCE) and Mean Square Error (MSE). In deep learning techniques, the size of the datasets plays an important role in obtaining high performance, but with traditional MSE, the loss values reach zero in the very early stages of the training process when the dataset size is large, and yet the model accuracy is still in progress. This study proposes a developed loss function via enhancing MSE. The proposed improved MSE is based on dividing the square error by the sum of the predicted label probabilities instead of the total of the sample number. Five datasets are used to test the performance of the proposed modified MSE with the proposed CNN pipeline model in addition to the modified VGG16. The datasets are AHCD, AIA9K, HIJJA, Self-collected, and MNIST. The loss rates of the proposed loss function showed a significant improvement in the accuracy rates for all datasets with the improved MSE in comparison to the CCE.vvv
Received 25/10/2023
Revised 05/07/2024
Accepted 07/05/2024
Published Online First 20/09/2024
References
Janocha K, Czarnecki WM. On loss functions for deep neural networks in classification. arXiv preprint arXiv: 170205659. 2017 Feb 18; 25. https://doi.org/10.4467/20838476SI.16.004.6185
Edalatifar M, Tavakoli MB, Ghalambaz M, Setoudeh F. Using deep learning to learn physics of conduction heat transfer. J Therm Anal Calorim. 2020 July 31; 146: 1435–1452. https://doi.org/10.1007/s10973-020-09875-6
Pham H. A New Criterion for Model Selection. Math. 2019 Dec 10; 7(12): 1215. https://doi.org/10.3390/math7121215
Ren J, Zhang M, Yu C, Liu Z, editors. Balanced mse for imbalanced visual regression. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2022 Mar 30. https://doi.org/10.48550/arXiv.2203.16427
Mazaal AR, Karam NS, Karam GS. Comparing Weibull Stress – Strength Reliability Bayesian Estimators for Singly Type II Censored Data under Different loss Functions. Baghdad Sci J. 2021; 18(2): 0306. https://doi.org/10.21123/bsj.2021.18.2.0306
Lawgali A, Bouridane A, Angelova M, Ghassemlooy Z. Handwritten Arabic character recognition: Which feature extraction method?. Int J Adv Sci Technol. 2011; 34: 1-8.
Hasasneh N, Hasasneh A, Salman N, Eleyan D. Towards offline Arabic handwritten character recognition based on unsupervised machine learning methods: A perspective study. 2019.
Lorigo LM, Govindaraju V. Offline Arabic handwriting recognition: a survey. IEEE Trans Pattern Anal Mach Intell. (TPAMI). 2006; 28(5): 712-24. https://doi.org/10.1109/TPAMI.2006.102
Hinton GE, Osindero S, Teh Y-W. A fast learning algorithm for deep belief nets. Neural Comput (NECO). 2006; 18(7): 1527-54. https://doi.org/10.1162/neco.2006.18.7.1527
Goyal P, Pandey S, Jain K. Deep learning for natural language processing. SpringerLink. 2018; 138-43. https://doi.org/10.1007/978-1-4842-3685-7
Chen Z. Deep-learning Approaches to Object Recognition from 3D Data. Case Western Reserve University. 2017.
Rosasco L, Vito ED, Caponnetto A, Piana M, Verri A. Are loss functions all the same? Neural Comput (NECO).2004; 16(5): 1063-76. https://doi.org/10.1162/089976604773135104
Deng L, Gong Y, Lu X, Lin Y, Ma Z, Xie M. STELA: A Real-Time Scene Text Detector With Learned Anchor. IEEE Access. 2019;7:153400-7. https://doi.org/10.1109/ACCESS.2019.2948405
Najadat HM, Alshboul AA, Alabed AF, editors. Arabic handwritten characters recognition using convolutional neural network. 2019 10th International Conference on Information and Communication Systems (ICICS). 2019. https://doi.org/10.1109/IACS.2019.8809122
El Atillah M, El Fazazy K, Riffi J. Classification of Arabic Alphabets Using a Combination of a Convolutional Neural Network and the Morphological Gradient Method. Baghdad Sci J. 2024; 21(1): 0252. https://doi.org/10.21123/bsj.2023.7877
Saifullah, Ren Z, Hussain K, Faheem M. K-means online-learning routing protocol (K-MORP) for unmanned aerial vehicles (UAV) adhoc networks. Ad Hoc Networks. 2024; 154: 103354. https://doi.org/10.1016/j.adhoc.2023.103354
Saini D, Garg R, Malik R, Prashar D, Faheem M. HFRAS: design of a high-density feature representation model for effective augmentation of satellite images. Signal, Image and Video Processing. 2023 Nov 11. https://doi.org/10.1007/s11760-023-02859-7
Zhang Z, Sabuncu M, editors. Generalized cross entropy loss for training deep neural networks with noisy labels. Adv Neural Inf Process Syst. 2018; May 20: 8792–8802. https://doi.org/10.48550/arXiv.1805.07836
Zhu X, Zhou H, Yang C, Shi J, Lin D, editors. Penalizing top performers: Conservative loss for semantic segmentation adaptation. Proceedings of the European Conference on Computer Vision (ECCV); Springer, Cham; 2018; 11211: 587–603. https://doi.org/10.1007/978-3-030-01234-2_35
Tang Y. Deep learning using linear support vector machines. arXiv preprint arXiv: 13060239. 2013 Jun 2. https://doi.org/10.48550/arXiv.1306.0239
Mohapatra P, Rolinek M, Jawahar C, Kolmogorov V, Pawan Kumar M, editors. Efficient optimization for rank-based loss functions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).2018 Jun 28; 3693-3701. https://doi.org/10.1109/cvpr.2018.00389
Willmott CJ, Matsuura K. Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Clim Res. 2005; 30(1): 79-82. https://doi.org/10.3354/cr030079
Willmott CJ, Matsuura K, Robeson SM. Ambiguities inherent in sums-of-squares-based error statistics. Atmos. Environ.2009 Jan; 43(3): 749-52. https://doi.org/10.1016/j.atmosenv.2008.10.005
Chai T, Draxler RR. Root mean square error (RMSE) or mean absolute error (MAE)?–Arguments against avoiding RMSE in the literature. Geosci Model Dev. 2014 Jun 30; 7(3): 1247-50. https://doi.org/10.5194/gmd-7-1247-2014
Chatterjee S, Hammad A, Katzin EN, Hua J. Virtual wallet card selection apparatuses.methods and systems. 2013 Nov 5.
Ghazvini A, Abdullah SNHS, Hasan MK, Kasim DZAB. Crime spatiotemporal prediction with fused objective function in time delay neural network. IEEE Access. 2020 Jun 18; 8: 115167-83. https://doi.org/10.1109/ACCESS.2020.3002766
Nayef BH, Abdullah SNHS, Sulaiman R, Alyasseri ZAA. Optimized leaky ReLU for handwritten Arabic character recognition using convolution neural networks. Multimed Tools Appl. 2022 Oct 19; 81(2): 2065-94. https://doi.org/10.1007/s11042-021-11593-6
Pandit V, Schuller B. On Many-to-Many Mapping Between Concordance Correlation Coefficient and Mean Square Error. arXiv preprint arXiv: 190205180. 2019 Feb 14. https://doi.org/10.48550/arXiv.1902.05180
Sharma N. A Beginner’s Guide to Loss functions for Regression Algorithms November 14, 2021 .
El-Sawy A, Loey M, El-Bakry H. Arabic handwritten characters recognition using convolutional neural network. WSEAS Trans Comput Res. 2017; 5: 11-9.
Torki M, Hussein ME, Elsallamy A, Fayyaz M, Yaser S. Window-based descriptors for arabic handwritten alphabet recognition: A comparative study on a novel dataset. arXiv preprint arXiv: 14113519. 2014 Nov. https://doi.org/10.48550/arXiv.1411.3519
Altwaijry N, Al-Turaiki I. Arabic handwriting recognition system using convolutional neural network. Neural Comput Appl. 2020 Jun 28 ; 1-13. https://doi.org/10.1007/s00521-020-05070-8
Mudhsh M, Almodfer R. Arabic handwritten alphanumeric character recognition using very deep neural network. info. 2017 Aug 31; 8(3): 105. https://doi.org/10.3390/info8030105
Alani AA. Arabic handwritten digit recognition based on restricted Boltzmann machine and convolutional neural networks. info. 2017 Nov 9 ; 8(4): 142. https://doi.org/10.3390/info8040142
de Sousa IP. Convolutional ensembles for Arabic handwritten character and digit recognition. PeerJ Comput Sci. 2018 Oct 15 ; 4: e167. https://doi.org/10.7717/peerj-cs.167.
Younis KS. Arabic handwritten character recognition based on deep convolutional neural networks. Jordanian J Comput Inf Techno (JJCIT). 2017 Feb 18; 3(3): 186-200. https://doi.org/10.48550/arXiv.1702.05659
Downloads
Issue
Section
License
Copyright (c) 2024 Bahera H. Nayef , Siti Norul Huda Sheikh Abdullah, Manal Mohammed
This work is licensed under a Creative Commons Attribution 4.0 International License.
How to Cite
