Wavelet-Attention Swin for Automatic Diabetic Retinopathy Classification
DOI:
https://doi.org/10.21123/bsj.2024.8565Keywords:
APTOS Data Set, Diabetic Retinopathy, Swin-B, Swin-T, Wavelet-AttentionAbstract
Diabetic retinopathy (DR) is a complication of diabetes that affects the eyes by damaging the blood vessels in the retina. High blood sugar levels can cause leakage or blockage of these vessels, leading to vision loss or blindness. Early detection of DR is crucial to prevent blindness, but manually analyzing fundus images can be time-consuming, especially with a large number of images. Swin-Transformers have gained popularity in medical image analysis, reducing calculations and yielding improved results. This paper introduces the WT Attention-Db5 Block, which focuses attention on the high-frequency domain using Discrete Wavelet Transform (DWT). This block extracts detailed information from the high-frequency field while retaining essential low-frequency information. The study discusses findings from the 2019 Blindness Detection challenge (APTOS 2019 BD) held by the Asia Pacific Tele-Ophthalmology Society.The proposed WT-Swin model achieves significant improvements in classification accuracy. For Swin-T, the training and validation accuracies are 99.14% and 98.91%, respectively. For binary classification using Swin-B, the training accuracy is 99.01%, the validation accuracy is 99.18%, and the test accuracy is 98%. In multi-classification, the training and validation accuracies are 93.19% and 86.34%, respectively, while the test accuracy is 86%.In conclusion, early detection of DR is essential for preventing vision loss. The WT Attention-Db5 Block integrated into the WT-Swin model shows promising results in classification accuracy
Received 11/02/2023
Revised 12/06/2023
Accepted 14/06/2023
Published Online First 20/01/2024
References
Farooq MS, Arooj A, Alroobaea R, Baqasah AM, Jabarulla MY, Singh D, et al. Untangling computer-aided diagnostic system for screening diabetic retinopathy based on deep learning techniques. Sensors MDPI. 2022 24; 22(5): 1803. https://doi.org/10.3390/s22051803
Alyoubi WL, Abulkhair MF, Shalash WM. Diabetic retinopathy fundus image classification and lesions localization system using deep learning. Sensors MDPI. 2021; 21(11): 3704. https://doi.org/10.3390/s21113704
Hameed EK, Al-Ameri LT, Hasan HS, Abdulqahar Z. The Cut-off Values of Triglycerides-Glucose Index for Metabolic Syndrome Associated with Type 2 Diabetes Mellitus. Baghdad Sci J. 2021; 19(2): 340-346. http://dx.doi.org/10.21123/bsj.2022.19.2.0340
Qureshi I, Ma J, Abbas Q. Diabetic retinopathy detection and stage classification in eye fundus images using active deep learning. Multimed Tools Appl. 2021; 80: 11691-11721. https://doi.org/10.1007/s11042-020-10238-4
Hasan DA, Zeebaree SR, Sadeeq MA, Shukur HM, Zebari RR, Alkhayyat AH. Machine Learning-based Diabetic Retinopathy Early Detection and Classification Systems-A Survey. 1st Babylon Int Conf Inf Technol Sci. 2021 28 (pp. 16-21). IEEE. https://doi.org/10.1109/BICITS51482.2021.9509920
Dutta S, Saini K. Securing data: A study on different transform domain techniques. WSEAS Trans Syst Control. 2021; 16: 110-120. https://doi.org/10.37394/23203.2021.16.8
Liu J, Ding J, Ge X, Wang J. Evaluation of total nitrogen in water via airborne hyperspectral data: potential of fractional order discretization algorithm and discrete wavelet transform analysis. Remote Sens. MDPI. 2021; 13(22): 4643. https://doi.org/10.3390/rs13224643
Liu Z, Hu H, Lin Y, Yao Z, Xie Z, Wei Y, et al. Swin transformer v2: Scaling up capacity and resolution. Proc IEEE/CVF Conf CVPR 2022: 12009-12019.
https://doi.org/10.48550/arXiv.2111.09883
Xie Z, Lin Y, Yao Z, Zhang Z, Dai Q, Cao Y, et al. Self-supervised learning with swin transformers. arXiv preprint arXiv. 2021 10; 2105: 1-8. https://doi.org/10.48550/arXiv.2105.04553
Hao Li, Zhijing Yang, Xiaobin Hong, Ziying Zhao, Junyang Chen, Yukai Shi, et al. DnSwin: Toward real-world denoising via a continuous Wavelet Sliding Transformer. Knowl Based Syst . 2022, 14; 255: 109815. https://doi.org/10.1016/j.knosys.2022.109815
Zhao X, Huang P, Shu X. Wavelet-Attention CNN for image classification. Multimedia Systems. 2022 ; 28(3): 915-24. https://doi.org/10.1007/s00530-022-00889-8
Kobat SG, Baygin N, Yusufoglu E, Baygin M, Barua PD, Dogan S, et al. Automated Diabetic Retinopathy Detection Using Horizontal and Vertical Patch Division-Based Pre-Trained DenseNET with Digital Fundus Images. Diagnostics. 2022 15; 12(8): 1975. https://doi.org/10.3390/diagnostics12081975
Chen D, Yang W, Wang L, Tan S, Lin J, Bu W. PCAT-UNet: UNet-like network fused convolution and transformer for retinal vessel segmentation. Raja G, editor. PLoS One. 2022 , 24; 17(1): e0262689. https://dx.plos.org/10.1371/journal.pone.0262689
Gupta IK, Choubey A, Choubey S. Mayfly optimization with deep learning enabled retinal fundus image classification model. Comput Electr Eng. 2022 , 1; 102: 108176. https://doi.org/10.1016/j.compeleceng.2022.108176
Atwany MZ, Sahyoun AH, Yaqub M. Deep learning techniques for diabetic retinopathy classification: A survey. IEEE Access. 2022 8: 28642 - 28655. https://doi.org/10.1109/ACCESS.2022.3157632
Omran M, AlShemmary EN. Towards accurate pupil detection based on morphology and Hough transform. Baghdad Sci J. 2020 ,1; 17(2): 583-590. http://dx.doi.org/10.21123/bsj.2020.17.2.0583
Jaskari J, Sahlsten J, Damoulas T, Knoblauch J, Särkkä S, Kärkkäinen L, et al. Uncertainty-aware deep learning methods for robust diabetic retinopathy classification. IEEE Access. 2022; 10: 76669-76681. https://doi.org/10.1109/ACCESS.2022.3192024
Zia F, Irum I, Qadri NN, Nam Y, Khurshid K, Ali M, et al. A multilevel deep feature selection framework for diabetic retinopathy image classification. CMC 2022; 70(2): 2261-2276. https://doi.org/10.32604/cmc.2022.017820
Ashour M A H. Optimized Artificial Neural network models to time series. Baghdad Sci J. 2022 ;19(4): 0899-0904. https://doi.org/10.21123/bsj.2022.19.4.0899
Liu Y, Guan L, Hou C, Han H, Liu Z, Sun Y, et al. Wind Power Short-Term Prediction Based on LSTM and Discrete Wavelet Transform. Appl Sci. 2019 , 15; 9(6): 1108. https://doi.org/10.3390/app9061108
Nobre J, Neves RF. Combining principal component analysis, discrete wavelet transform and XGBoost to trade in the financial markets. Expert Systems with Applications. 2019, 1; 125: 181-94. https://doi.org/10.1016/j.eswa.2019.01.083
Freire PK de MM, Santos CAG, Silva GBL da. Analysis of the use of discrete wavelet transforms coupled with ANN for short-term streamflow forecasting. Appl Soft Comput ASC. 2019; 80: 494–505. https://doi.org/10.1016/j.asoc.2019.04.024
Tymchenko B, Marchenko P, Spodarets D. Deep learning approach to diabetic retinopathy detection. arXiv preprint arXiv: 2020 , 3; 2003: 02261.
Bodapati JD, Naralasetti V, Shareef SN, Hakak S, Bilal M, Maddikunta PKR, et al. Blended Multi-Modal Deep ConvNet Features for Diabetic Retinopathy Severity Prediction. Electronics. 2020 , 30; 9(6): 914. https://doi.org/10.3390/electronics9060914
Khalaf M, Dhannoon BN. MSRD-Unet: Multiscale Residual Dilated U-Net for Medical Image Segmentation. Baghdad Sci J. 2022 , 5; 19(6(Suppl.)): 1603-1611. https://doi.org/10.21123/bsj.2022.7559
Downloads
Issue
Section
License
Copyright (c) 2024 Baghdad Science Journal
This work is licensed under a Creative Commons Attribution 4.0 International License.
How to Cite
