Deep Learning Function Implications For Handwritten Character Recognition Using Improved Mean Square Error Function

Authors

  • Bahera H. Nayef Department of Computer Science, College of Science, Al Nahrain University, Baghdad, Iraq. https://orcid.org/0000-0002-1844-5839
  • Siti Norul Huda Sheikh Abdullah Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, 43600 Bangi, Selangor, Malaysia.
  • Manal Mohammed Faculty of Information Science and Technology, Universiti Kebangsaan Malaysia, 43600 Bangi, Selangor, Malaysia & Faculty of Administrative Science, Hadhramout University, AL-Mukalla, Yemen.

DOI:

https://doi.org/10.21123/bsj.2024.10024

Keywords:

Loss functions; Machine learning; pattern recognition; Handwritten character recognition; Deep learning; Mean Square Error

Abstract

Loss functions are used for estimating the accuracy of the classification models in Machine Learning. The error loss value measures whether the predicted labels match the true labels or are close to them. Handwritten character recognition is a pattern recognition problem. The progression of pattern recognition applications for offline image classification is rapidly improved by using deep learning techniques. Convolution Neural Networks (CNN) and the activation function are the most commonly used in deep learning. In addition, different loss functions are used to evaluate the performance of the model such as Categorical Cross-Entropy (CCE) and Mean Square Error (MSE). In deep learning techniques, the size of the datasets plays an important role in obtaining high performance, but with traditional MSE, the loss values reach zero in the very early stages of the training process when the dataset size is large, and yet the model accuracy is still in progress. This study proposes a developed loss function via enhancing MSE. The proposed improved MSE is based on dividing the square error by the sum of the predicted label probabilities instead of the total of the sample number. Five datasets are used to test the performance of the proposed modified MSE with the proposed CNN pipeline model in addition to the modified VGG16. The datasets are AHCD, AIA9K, HIJJA, Self-collected, and MNIST. The loss rates of the proposed loss function showed a significant improvement in the accuracy rates for all datasets with the improved MSE in comparison to the CCE.vvv

References

Janocha K, Czarnecki WM. On loss functions for deep neural networks in classification. arXiv preprint arXiv: 170205659. 2017 Feb 18; 25. https://doi.org/10.4467/20838476SI.16.004.6185

Edalatifar M, Tavakoli MB, Ghalambaz M, Setoudeh F. Using deep learning to learn physics of conduction heat transfer. J Therm Anal Calorim. 2020 July 31; 146: 1435–1452. https://doi.org/10.1007/s10973-020-09875-6

Pham H. A New Criterion for Model Selection. Math. 2019 Dec 10; 7(12): 1215. https://doi.org/10.3390/math7121215

Ren J, Zhang M, Yu C, Liu Z, editors. Balanced mse for imbalanced visual regression. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2022 Mar 30. https://doi.org/10.48550/arXiv.2203.16427

Mazaal AR, Karam NS, Karam GS. Comparing Weibull Stress – Strength Reliability Bayesian Estimators for Singly Type II Censored Data under Different loss Functions. Baghdad Sci J. 2021; 18(2): 0306. https://doi.org/10.21123/bsj.2021.18.2.0306

Lawgali A, Bouridane A, Angelova M, Ghassemlooy Z. Handwritten Arabic character recognition: Which feature extraction method?. Int J Adv Sci Technol. 2011; 34: 1-8.

Hasasneh N, Hasasneh A, Salman N, Eleyan D. Towards offline Arabic handwritten character recognition based on unsupervised machine learning methods: A perspective study. 2019.

Lorigo LM, Govindaraju V. Offline Arabic handwriting recognition: a survey. IEEE Trans Pattern Anal Mach Intell. (TPAMI). 2006; 28(5): 712-24. https://doi.org/10.1109/TPAMI.2006.102

Hinton GE, Osindero S, Teh Y-W. A fast learning algorithm for deep belief nets. Neural Comput (NECO). 2006; 18(7): 1527-54. https://doi.org/10.1162/neco.2006.18.7.1527

Goyal P, Pandey S, Jain K. Deep learning for natural language processing. SpringerLink. 2018; 138-43. https://doi.org/10.1007/978-1-4842-3685-7

Chen Z. Deep-learning Approaches to Object Recognition from 3D Data. Case Western Reserve University. 2017.

Rosasco L, Vito ED, Caponnetto A, Piana M, Verri A. Are loss functions all the same? Neural Comput (NECO).2004; 16(5): 1063-76. https://doi.org/10.1162/089976604773135104

Deng L, Gong Y, Lu X, Lin Y, Ma Z, Xie M. STELA: A Real-Time Scene Text Detector With Learned Anchor. IEEE Access. 2019;7:153400-7. https://doi.org/10.1109/ACCESS.2019.2948405

Najadat HM, Alshboul AA, Alabed AF, editors. Arabic handwritten characters recognition using convolutional neural network. 2019 10th International Conference on Information and Communication Systems (ICICS). 2019. https://doi.org/10.1109/IACS.2019.8809122

El Atillah M, El Fazazy K, Riffi J. Classification of Arabic Alphabets Using a Combination of a Convolutional Neural Network and the Morphological Gradient Method. Baghdad Sci J. 2024; 21(1): 0252. https://doi.org/10.21123/bsj.2023.7877

Saifullah, Ren Z, Hussain K, Faheem M. K-means online-learning routing protocol (K-MORP) for unmanned aerial vehicles (UAV) adhoc networks. Ad Hoc Networks. 2024; 154: 103354. https://doi.org/10.1016/j.adhoc.2023.103354

Saini D, Garg R, Malik R, Prashar D, Faheem M. HFRAS: design of a high-density feature representation model for effective augmentation of satellite images. Signal, Image and Video Processing. 2023 Nov 11. https://doi.org/10.1007/s11760-023-02859-7

Zhang Z, Sabuncu M, editors. Generalized cross entropy loss for training deep neural networks with noisy labels. Adv Neural Inf Process Syst. 2018; May 20: 8792–8802. https://doi.org/10.48550/arXiv.1805.07836

Zhu X, Zhou H, Yang C, Shi J, Lin D, editors. Penalizing top performers: Conservative loss for semantic segmentation adaptation. Proceedings of the European Conference on Computer Vision (ECCV); Springer, Cham; 2018; 11211: 587–603. https://doi.org/10.1007/978-3-030-01234-2_35

Tang Y. Deep learning using linear support vector machines. arXiv preprint arXiv: 13060239. 2013 Jun 2. https://doi.org/10.48550/arXiv.1306.0239

Mohapatra P, Rolinek M, Jawahar C, Kolmogorov V, Pawan Kumar M, editors. Efficient optimization for rank-based loss functions. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).2018 Jun 28; 3693-3701. https://doi.org/10.1109/cvpr.2018.00389

Willmott CJ, Matsuura K. Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Clim Res. 2005; 30(1): 79-82. https://doi.org/10.3354/cr030079

Willmott CJ, Matsuura K, Robeson SM. Ambiguities inherent in sums-of-squares-based error statistics. Atmos. Environ.2009 Jan; 43(3): 749-52. https://doi.org/10.1016/j.atmosenv.2008.10.005

Chai T, Draxler RR. Root mean square error (RMSE) or mean absolute error (MAE)?–Arguments against avoiding RMSE in the literature. Geosci Model Dev. 2014 Jun 30; 7(3): 1247-50. https://doi.org/10.5194/gmd-7-1247-2014

Chatterjee S, Hammad A, Katzin EN, Hua J. Virtual wallet card selection apparatuses.methods and systems. 2013 Nov 5.

Ghazvini A, Abdullah SNHS, Hasan MK, Kasim DZAB. Crime spatiotemporal prediction with fused objective function in time delay neural network. IEEE Access. 2020 Jun 18; 8: 115167-83. https://doi.org/10.1109/ACCESS.2020.3002766

Nayef BH, Abdullah SNHS, Sulaiman R, Alyasseri ZAA. Optimized leaky ReLU for handwritten Arabic character recognition using convolution neural networks. Multimed Tools Appl. 2022 Oct 19; 81(2): 2065-94. https://doi.org/10.1007/s11042-021-11593-6

Pandit V, Schuller B. On Many-to-Many Mapping Between Concordance Correlation Coefficient and Mean Square Error. arXiv preprint arXiv: 190205180. 2019 Feb 14. https://doi.org/10.48550/arXiv.1902.05180

Sharma N. A Beginner’s Guide to Loss functions for Regression Algorithms November 14, 2021 .

El-Sawy A, Loey M, El-Bakry H. Arabic handwritten characters recognition using convolutional neural network. WSEAS Trans Comput Res. 2017; 5: 11-9.

Torki M, Hussein ME, Elsallamy A, Fayyaz M, Yaser S. Window-based descriptors for arabic handwritten alphabet recognition: A comparative study on a novel dataset. arXiv preprint arXiv: 14113519. 2014 Nov. https://doi.org/10.48550/arXiv.1411.3519

Altwaijry N, Al-Turaiki I. Arabic handwriting recognition system using convolutional neural network. Neural Comput Appl. 2020 Jun 28 ; 1-13. https://doi.org/10.1007/s00521-020-05070-8

Mudhsh M, Almodfer R. Arabic handwritten alphanumeric character recognition using very deep neural network. info. 2017 Aug 31; 8(3): 105. https://doi.org/10.3390/info8030105

Alani AA. Arabic handwritten digit recognition based on restricted Boltzmann machine and convolutional neural networks. info. 2017 Nov 9 ; 8(4): 142. https://doi.org/10.3390/info8040142

de Sousa IP. Convolutional ensembles for Arabic handwritten character and digit recognition. PeerJ Comput Sci. 2018 Oct 15 ; 4: e167. https://doi.org/10.7717/peerj-cs.167.

Younis KS. Arabic handwritten character recognition based on deep convolutional neural networks. Jordanian J Comput Inf Techno (JJCIT). 2017 Feb 18; 3(3): 186-200. https://doi.org/10.48550/arXiv.1702.05659

Downloads

Issue

Section

article

How to Cite

1.
Deep Learning Function Implications For Handwritten Character Recognition Using Improved Mean Square Error Function. Baghdad Sci.J [Internet]. [cited 2024 Oct. 9];22(4). Available from: https://bsj.uobaghdad.edu.iq/index.php/BSJ/article/view/10024