A New Model Design for Combating COVID -19 Pandemic Based on SVM and CNN Approaches

: In the current worldwide health crisis produced by coronavirus disease (COVID-19), researchers and medical specialists began looking for new ways to tackle the epidemic. According to recent studies, Machine Learning (ML) has been effectively deployed in the health sector. Medical imaging sources (radiography and computed tomography) have aided in the development of artificial intelligence(AI) strategies to tackle the coronavirus outbreak. As a result, a classical machine learning approach for coronavirus detection from Computerized Tomography (CT) images was developed. In this study, the convolutional neural network (CNN) model for feature extraction and support vector machine (SVM) for the classification of axial lung CT-scans into two groups (COVID-19 and NonCOVID-19) had been proposed. A dataset used is 960 slices of CT scan collected from Iraqi patients /Ibn Al-Nafis teaching hospital. The performance metrics are used in this study (accuracy, recall, precision, and F1 scores). The results indicate that the proposed approach generated a high-quality model for the collected dataset, with an overall accuracy of 98.95% and an overall recall of 97 %


Introduction:
ML is a subset of AI technology in which algorithms search through large data sets for patterns, learn from them, and perform jobs without being told how to fix the problem. ML algorithms have a lot of potential in the healthcare industry because of the volume of data collected for each patient. As a result, there are currently a number of effective machine learning implications in healthcare 1,2 . Coronavirus is one of the contagious diseases generated by the coronavirus (SARS-CoV-2). Because until the beginning of 2021, there is no vaccine or licensed medicine for coronavirus, social isolation has been identified as the most effective control and prevention strategy and it is an important aspect of managing and preventing the virus's spread 3 . Diagnostic tests and antibody tests are the two types of standardized tests used to identify coronavirus. These procedures are time-consuming, necessitate the use of specific materials and tools, and are poor at generating true positive rates. As a result, current diagnostic and tracking procedures for coronavirus illness are useless 4 . The secondary diagnosis of coronavirus relies mostly on chest CT imaging. It may take a few seconds for the coronavirus chest CTassisted diagnosis approach based on ML to provide reliable test findings. ML models are one of the most promising tools for assisting radiologists in early coronavirus detection. Moreover, it reduces radiologists' workload, improves detection accuracy and efficiency, and provides coronavirus patients with timely and accurate treatment 5 .

Applications of Machine Learning in COVID-19 Fighting:
ML methods can be separated into supervised and unsupervised learning. Under these methods, there are many common machine learning techniques used in various applications such as Knearest neighbor (KNN), SVM, decision trees (DT ), K-means, and principle component analysis (PCA), Fig. 1 shows a classification of ML methods 6 . In this paper, the use of the supervised and unsupervised algorithms in combating COVID-19 and limiting its spread will be introducing.

Supervised learning techniques
Supervised learning techniques are machine learning approaches that are based on past data and current data to predict future events. The learning process for machines begins with a process of training the data set to predict output values where the examples are correctly labeled according to the category to which they belong 5 . Implementation of some of the most significant supervised ML approaches, such as SVM, logistic regression (LR), and KNN, in tackling the coronavirus pandemic will be discussed.

Support vector machine (SVM)
SVM is a controlled ML algorithm and powerful tool that can be deployed for both classification and regression challenges. It is widely used in the health sector because of its excellent precision and performance. As a result of its excellent performance, SVM has recently been utilized to battle the coronavirus epidemic 7,8 . Initially, M.Turkoglu built a model named COVIDetectioNet that relied on SVM to detect coronavirus using X-ray pictures. The dataset consisted of 6092 images categorized as normal, COVID-19, and Pneumonia, and the suggested model attained an accuracy of 99.1%. 9 . For accurate detection of coronavirus patients using X-ray images the authors use SVM as a classifier, the maximum accuracy, sensitivity, and specificity were achieved when fusion SVM with CNN and the sobel filter were used, with 99.02 %, 100 %, and 95.23 %, respectively 10 . Furthermore, C. Zhou et al proposed a model for coronavirus detection, they used a mix of image regrouping and ResNet-SVM to achieve their goal. SVM was used in the final step for the recognition of the feature extracted, this model accomplished an accuracy of 93% and sensitivity of 88% 11 . In another study SVM frame was utilized to ride the issues of early diagnosis of coronavirus, It is the first research to design an SVM frame to perform simultaneous multi-job and multi-modal learning for medical image analysis using CT images the accuracy accomplished by this model reached 92.57% and the sensitivity of 91.44% 12 . In the presented work, SVM is proposed as a classifier of CT images into normal and abnormal COVID-19 groups.

Logistic Regression
LR is a ML technique for modeling the prospect of a specific category or occurrence. This strategy is used when the data are linearly distributed and the outcome is binary or dichotomous. That is to say, for binary classification problems, logistic regression is commonly utilized 13 . LR was recently used to mitigate the coronavirus pandemic. The classification of patients with COVID-19 and now whether the infected has risked or not using dataset collected from Kaggle and achieving an accuracy of 92% 14 . On the other hand, the LR and Multinomial Nave Bayes were employed to classify a clinical report, and they used classical and ensemble ML methods to divide textual clinical reports into four categories. Term frequency/inverse document frequency (TF/IDF), Bag of words (BOW), and report length were utilized to implement feature engineering. The results revealed that LR and Multinomial Nave Bayes had 96.2 % performance in identifying covid-19, outperforming other ML techniques 15 . Another study used LR for forecasting the spread of coronavirus based on the confirmed data 16 .

K Nearest-Neighbored (KNN)
The KNN algorithm is a supervised ML technique that could be utilized to handle both classification and regression issues. It saves all patients' cases diagnosed and classifies new data or cases using a similarity score. It's most typically used to classify a data point depending on its neighbors' classifications 17 . Several researchers used the KNN algorithm for fighting COVID-19, Mukherjee et al.
proposed an enhanced KNN for the detection of coronavirus, they used a mathematical function to determine the value of k not randomly choose it. This algorithm was applied to 7 benchmarks COVID-19 datasets collected from different countries. The outcome showed that the enhanced KNN classifier is well than the KNN classifier and the enhanced KNN along with features selection is well than KNN without feature selection 18 . Another study uses nine machine learning algorithms, KNN with SVM, bagging, boosting, SVM, bidirectional long shortterm memory, DT, naïve Bayes, RF, and multinomial LR as the classifier. These algorithms used with Noise filter algorithms are used to eliminate noise from the datasets applied in this research to show its impact of it on the machine learning algorithms. The result showed that ML models have produced good results, KNN achieved an accuracy of 100% after performing noise filtering operations for the prediction of coronavirus cases 19

Decision Tree (DT)
The DT algorithm is a type of supervised learning algorithm. The aim of using a DT is to establish a training model that can be used to forecast the category or value of the input variables by learning simple decision rules learned from prior data (training data). Because of its ease of use and resilience, DT is widely employed in a variety of fields 21 . Recently, the DT algorithm was known in the medical field. For example, a decision tree model was established to detect the severity of the condition of children with COVID-19. The clinical laboratories and epidemiological information of 105 unwell children were obtained from a Chinese hospital from February 1 to March 3, 2020. COVID-19 positivity was found in 105 children, including 41 females and 64 males. The female hitting rate (39.05%) is lower than the male infection rate (60.95 %). The proposed approach performed admirably, receiving a perfect F1 score of 100 % 22 . Yoo et al. suggested a deep learning-based decision tree classifier to detect coronavirus from CXR images. The classifier's first binary decision tree differentiates between normal and anomalous CXR images, while the second tree detects aberrant photos with symptoms of tuberculosis, and the third tree does the same for coronavirus images. The first and second decisions were both 98 % and 80 % accurate, respectively, while the third decision was 95 % accurate on average 23 . In order to forecast the death rate in patients infected with coronavirus, a comparative analysis of different ML approaches including DT, RF, KNN, SVM, LR, and ANN was used. This research employed a dataset of 117,000 coronavirus ill cases from both males and females. For the prediction of the death rate, the model had a 93% accuracy rate. While employing 10-fold crossvalidation, DT was able to reach an accuracy of 90.63% 24 . A summary of some applications of supervised ML methods for coronavirus fighting is shown in Table. 1.

Unsupervised Learning Technique
Unsupervised learning analyzes and clusters unlabeled datasets utilizing ML methods. These algorithms detect underlying patterns or data groups even without human participation. Because of its ability to find similarities and divergences in information, it is the greatest choice for exploratory data analysis, cross-selling tactics, customer segmentation, and image recognition. Implementation of some of the important unsupervised ML methods which are PCA and Kmeans in fighting the coronavirus pandemic will be discussed.

Principle Component Analysis(PCA)
PCA is a widely used unsupervised ML approach in a wider range of usage including exploratory data analysis, dimension reduction, information fusion, and data de-noising. The primary objective of PCA is to produce the most significant data features. It guarantees that numerous feature variables are decreased without sacrificing information 25 . The authors used PCA for reducing the features of images extracted from CNN, this method achieved good accuracy of 97.84% 26 . In another research aligned big genome sequences were subjected to PCA, and numerical values were transformed from symbols using an established approach for protein sequence cluster analysis. This research developed a quick tool for analyzing high-volume genome sequences like coronavirus, which was effectively used for over 20,000 sequences and might give mutation direction data for coronavirus studies 27 . The RNA expression patterns of 16 coronavirus patients and 18 healthy control subjects were analyzed using PCA-based unsupervised feature extraction (PCAUFE). From 60,683 potential probes, 123 genes were identified as crucial for COVID-19 development, including immune-related genes 28 . Ozturk et al classified coronavirus from Xray and CT images. They used four feature extraction methods to extract the features vector and the feature vector is decreased in size by PCA to refuse interconnected features in the feature vector. Since the PCA architecture can work independently from the number of samples, it has provided very efficient classification 29 .

K-means Learning
One of the most basic and often used unsupervised ML algorithms is k-means learning. In other terms, the K-means algorithm finds k centroids and then assigns each data point to the cluster with the fewest centroids. In simple terms, k-means clustering allows us to automatically cluster data into numerous groups by identifying distinct types of groups in unlabeled datasets without the need for data training. The purpose of this centroid-based strategy is to minimize the summation of distances among data points and their associated clusters by connecting each cluster to a centroid 30 . The authors utilized unsupervised ML algorithms (k-means) to identify groups of countries that depend on the data. Disease incidence rates, air pollution measures, socioeconomic status, and health system coverage were all submitted to the algorithm as inputs. The model was shown to be able to stratify nations based on the number of verified coronavirus cases (p0.001) 35 . In another study, the K-means method was used in clustering coronavirus cases in Indonesia's provinces using November 2020 data. This study looked more into the statistical side. There are three variables originally applied in this paper, i.e. numbers of coronavirus certain cases, recovered, and death 36 . J. Wu and S. Sha used k-mean to learn about the pattern of the coronavirus pandemic in the United States and the implications for mitigating the disease. They applied the k-means algorithm to discover the pattern of disease curves in different cases 30 . In this study, the 3M k-means algorithm was utilized to identify those who had been in close contact with COVID-19 infected patients, as well as to determine the risk of infection. This algorithm decreases the amount of data used by ignoring the excluded risk categories and significantly reducing computational time, achieves significant cost savings, and provides psychological security for working people to resume work. Therefore, the use of this algorithm can help in controlling the epidemic coronavirus 37 . On other hand L. M. Abdulrahman et al was used to study the adverse reactions of the global COVID-19 vaccine based on the cluster ML algorithms. The K-means, Density-Based Algorithm, Hierarchical Clustering Algorithm, and Expectation and Maximization (EM) algorithms were compared in this study. K-means have outperformed the rest of the algorithms by using them extensively in different types of the coronavirus vaccine reverse vaccine dataset 38 . A summary of some applications of unsupervised ML methods for coronavirus fighting is shown in Table 2.

The Proposed Model Design
The overall algorithmic framework is given in Fig. 2.

COVID-19 Dataset Data collection and Acquisition
The data were collected from Iraqi patients at Ibn Al-Nafis teaching hospital in Baghdad, Iraq. The scanning device was Toshiba (Aquilion 64-slice multi-slice helical CT). The following scanning parameters were used to capture the CT images:  Tube voltage 120 kV  Current 500 mA  Slice thickness of 5 mm  Scanning with no contrast introduced.
The reconstructed image is in DICOM format and has a spatial resolution with a size of 512× 512 (rows and columns). The overall chest CT scans composed were of 217 cases, these included (52, 89, 76) for coronavirus, Non coronavirus, and excluded cases respectively. The ground truth concerning this data was established based on the diagnosis decisions of three experienced radiologists. Cases are eliminated based on a variety of criteria, including circumstances in which the contrast was employed for purposes other than diagnosis, the total cases for each class are listed in Table 3.

Data Preparation and Interpretation
The total axial CT images acquired from collected cases included 480 images for each class, 480 images for the COVID-19 case, and 480 images for the Non-COVID19. Three radiological professionals with more than 10 years of experience chose and annotated the number of slices for each patient depending on the major symptoms of COVID-19 and their diffusion in the lung area. These symptoms vary depending on the severity of the illness in each patient:  Early-stage: inflammatory infiltration with patchy or segmental pure Ground Glass Opacities in the subpleural or peri-bronchovascular sections of one side or both lungs and Vascular dilation.  The introductory stage: more widespread pure GGOs in numerous lobes of both lungs, consolidation of some lesions, thickening of the interlobular septum, and the appearance of a crazy-paving pattern; air bronchograms are normal.  The advanced stage: the lesions were disseminated in both lungs, consisting mostly of GGOs and united. Parenchymal bands and a small degree of pleural effusion characterize the advanced stage.
CT images are obtained in DICOM format, which has a depth of 16 bits. Because high-resolution photographs need significant computational power, considerable processing time, and huge memory. When dealing with deep learning, the images were transformed to bitmap format (.bmp). This reduced the size of the image. Another crucial step is data anonymization, which safeguards patient-related private data. During image conversion, the metadata is stripped from the photos. All the images used in the classification model were pre-processed by resizing the images to (256 x 256) to obtain impersonation by a different set of features. Samples of the collected CT scans are shown in Fig. 3.

Methods:
Image preprocessing Pre-processing is a very important step before the images are fed into the models, in order to increase efficiency and reduce complexity 39 . The size of input images was chosen experimentally taking into account the computational power of the system to train the model, therefore images were resized to a smaller size equal to (256×256), and rescale the values of the pixels for the 8 bits image (between 0 and 255) to the scale of [0, 1]. The data set is separated into 80% for training and 20% for testing.

Feature Extraction
Convolutional neural networks (CNNs) are artificial neural networks that are utilized to extract features from data and classify it. The structure of CNN consists of two parts, the initial part work for feature extraction included (the input layer, convolutional layer, activation layer, pooling layer, and the flatten layer), and another part work for classification including the dense fully connected layer and the output layer 40 41 . The benefit of CNN is that it can extract features on its own using its convolution kernels; further, there are a set of filters in the convolutional layers known as kernels or convolutional filters. These filters convolve with the input image in order to give the features map that uses after that to train the model. In our model, the number of filters was used 10,10, 20, and 30 respectively for every layer to yield a tensor of feature maps. The most common activation function is Rectifier Linear Unit (ReLU) was used in this work for calculating the feature maps that are produced from a convolutional layer, The Relu function is defined as 42 The pooling layer comes to minimize the dimensions of the features map while trying to preserve the contained information via the pooled maximum or average value for the defined kernel size (usually 2x2). In this work we used (max pooling) with a kernel size of 2x2 and strides of 2, 2, stride refer to the number of steps that the filter moves along both dimensions(x,y)vertically and horizontally. Finally, a flattened layer was used to obtain a feature vector of 27000 lengths, that represents the number of features extracted from CNN layers. Fig.4 shows the deep CNN as feature extraction with four convolutional layers and three max pooling.

SVM Classification
SVM is a statistical learning algorithm established by Vapnik to fix classification problems by finding the best hyperplane value and the results obtained from the optimal classification 43 . As illustrated in Fig. 5, the SVM classifier depends on calculating the best hyperplane to differentiate between two classes. The hyperplane is considered to be optimal if the distance between the hyperplane and the neighboring data from every class is the greatest distance possible. 9 . In a classification issue that containing two classes and n elements, learning data is X = {xi, yi} and i = {1,2,……, n}, where x represents the class labels corresponding to -1 or + 1 as a result of the calculations shown in Eq.2 and y represents the input vector of the features in size N.
where w and b denote the weight vector and hyperplane respectively. The basic method of SVM work is linear classification, the hyperplane is found in the classification problems that can be linearly separated which ensures the greatest possible separation between the training data of two classes that are near to one another, as an outcome of the optimization solution the problem appears in Eq.3 12 . ||w|| 2 is minimum, where d denotes the distance between the supports vector and the hyperplane. In classification problems a positive slack variable (ξ ) and a smoothing parameter (C) are added for handling the non-linearity as displayed in Eq.4 12 . ( In this study SVM model was configured with a linear kernel, and the regularization parameter C=1.0 was suggested by 44 and give the best classification result.

Results:
The dataset was divided in this study into two separate datasets at random, 80% and 20 % used for training and testing, respectively. These models were performed using HP laptop Intel® Core™ i7-10510U, RAM (8 GB), and Microsoft Windows 10 Pro. The code was done by Jupyter Notebook which is a web application available online using python programming language and Keras API with TensorFlow platform for ML. The SVM classifier uses the deep feature had been extracted from CNN network. The classification is then carried out, and the performance of all classification models is evaluated. For our investigation, the confusion matrices were employed to assess the model's performance. The following formulas were utilized to determine the Accuracy, Precision, Recall (sensitivity), and F1 score for the implemented models based on the predicted values: Accuracy = ((TP+TN) / (TP+FP+FN+TN)) *100% … 5 Precision = (TP / (TP+FP)) *100% … 6 Recall = (TP / (TP+FN)) *100% … 7 F1 Score = (2*(Recall * Precision) / (Recall + Precision)) *100% … 8 TP, FP, TN, and FN are the acronyms for True Positive, False Positive, True N negative, and False Negative, respectively. TP is the ratio of positive (Covid-19) that the model properly identifies as Covid-19 for a particular test dataset and model. FP is the ratio of negative (normal) to (positive) that the model erroneously classifies (Covid-19). The ratio of negative (normal) that is rightly categorized as normal is TN, whereas the ratio of positive (coronavirus) that is wrongly labeled as negative is FN (normal). Fig. 6 specifies the accuracy of the training and testing phases. The accuracy obtained by the proposed model is 98.95%. The precision, recall and f1 scores are 100 %, 97 % and 98.94%, respectively The confusion matrix gives us an idea about the performance of the model by calculating the accuracy of the model, the confusion matrix in Fig.7 shows the good performance obtained in this study, Among 192 images used to test the model, two images were misclassified by our model these images belong to the NonCOVID-19 cases. The threshold value used in this study is 0.5, any value greater than threshold refers to the COVID-19 class and under it belongs to the NonCOVID-19 In this article first, the applications of ML in coronavirus fighting have been offered. The applications of ML methods include supervised ML methods which are SVM, LR, KNN and unsupervised ML methods such as PCA and Kmeans have been introduced. All the common and practical information including writer's name, methodology, sample kind and outcome result of the proposed model, related to the employment of ML for fighting COVID-19 have been abstracted and offered in Tables 1 and 2. The research based on traditional supervised and unsupervised ML techniques (SVM, DT, K-means, PCA, etc.) showed low performance when applied alone. While it is performing well when combined with other techniques. Therefore, it was found that the grouping of CNN-based features and supervised classifier algorithm gives good results for classifying axial lung CT-images due to the intrinsic features provided by CNN feature extraction layers followed by the optimal parameter values of the SVM classifier. Combining the features obtained by pretrained network Resnet with SVM classifier usually had a good performance and reasonable accuracy levels on classification 95.33% when applied to X-ray images of lung 8 . Features that were chosen using the Relief feature selection algorithm from the layers of the Alexnet architecture, and combined with SVM classifier have further enhanced classification accuracy 99.18% on X-ray images, reflecting the efficiency of CNN features 9 .
Using SVM classifier to classify a set of CNN features, were provided good results (98.24%) on the classification of X-ray images, and when fusion CNN with SVM and sobel filter on the same data gave excellent results (99.02) 10 . In this study, a CNN deep learning algorithm that was built from scratch was Used to extract features and feed it's to SVM as a classifier for the discovery of coronavirus from lung CT images. The dataset utilized for model development was locally taken from the Iraqi health center. When comparing the proposed model in this study and other related studies listed in Table. 1, it has been stated that the proposed model has achieved relatively good results for the data set used in this study. Table 4 below shows the performance metrics of some previous methods introduced for solving similar classification problems in comparison with the model proposed in this study.

Conclusion:
The application of AI-based technologies, such as ML models, is one of the most promising strategies for assisting radiologists in the early identification of coronavirus. Besides, it decreased the workload of the technicians, increased the accuracy and efficiency of the detection, given appropriate time response and exact medicament for the patients of coronavirus. This study highlights the latest usages of ML to combat coronavirus. It was discovered that the grouping of CNN-based features and ML classifier algorithms were mostly used for coronavirus pandemic prediction and give good results for classifying axial lung CT-images. For that, this paper offered an overview of several research and studies to mitigate the coronavirus pandemic and limit its spread. In our methodology, a model for processing computed tomography images of the lung and detecting coronavirus disease was presented based on extracted features using CNN model and fed to SVM for classification CT-images. In comparison to the other related works, this work used a real dataset from Iraqi health centers. So, it is worth noting that many obstacles faced the study when compared to other works that used free available datasets; including the process of data collection, archiving, and avoiding poor quality data that leads to inaccurate predictions. Also, the most important thing is the risk of infection virus during the data collection. This work is vitally important for Iraqi healthcare centers by finding quick and accurate methods to mitigate the overload lies on the radiologist for detecting COVID-19 where it achieved a 98.95% accuracy rate in diagnosing the disease. For future works more data can be collected from multi-healthy centers further generalizing the model in its performance for COVID-19 diagnosis.