A BALANCED INTRUSION DETECTION SYSTEM FOR WIRELESS SENSOR NETWORKS IN A BIG DATA ENVIRONMENT USING CNN-SVM MODEL

. Wireless Sensor Networks (WSNs) were exposed to several distinct safety issues and attacks regarding gathering and sending data. In this scenario, one of the most prevalent WSN assaults that may target any tier of the protocol stack is the Denial of Service (DoS) attack. The current research suggested various strategies to find the attack in the network. However, it has classification challenges. An effective ensemble deep learning-based intrusion detection system to identify the assault in the WSN network was, therefore, suggested in this research to address this issue. The data pre-processing involves converting qualitative data into numeric data using the One-Hot Encoding technique. Following that, Normalization Process was carried out. Then Manta-Ray Foraging Optimization is suggested to choose the best subset of features. Then Synthetic Minority Oversampling Technique (SMOTE) oversampling creates a new minority sample to balance the processed dataset. Finally, CNN– SVM classifier is proposed to classify the attack kinds. The Accuracy, F-Measure, Precision, and Recall metrics were used to assess the outcomes of 99.75%, 99.21%, 100%, and 99.6%, respectively. Compared to existing approaches, the proposed method has shown to be extremely effective in detecting DoS attacks in WSNs.


Introduction.
Networking systems are used by people worldwide to solve problems, find fresh perspectives, and fulfil fundamental needs.One of the most recent and well-liked technical developments is the creation of sensors, which permit users to accept data from a long distance.Devices of the Internet of Things (IoT) are using sensors more frequently these days [1].Wireless Sensor Networks are regarded as an important research topic.Many uses, including telecommunications, the military, healthcare, research, and agriculture, can benefit from the technology [2] and see natural catastrophes like earthquakes, floods, and volcanoes [3].All of the nodes involved in the design of these applications are presumed to be reliable and cooperative.However, in actual deployments, nodes are subject to various intrusions and assaults that can substantially impact the system's performance and the network's capacity to function.Sadly, protecting this form of network from numerous harmful assaults is a challenging issue [4].A DoS assault is the most frequent danger that WSNs encounter [5].DoS attacks may harm a network service by blocking real traffic from connecting to the network by saturating it with many fraudulent requests.DoS assaults may occur at any tier of the TCP/IP protocol stack [6,7].The WSN's security must be verified using the Intrusion Detection System (IDS).
IDSs keep an eye on system activity to spot and stop malicious traffic.Attacks may be immediately detected by choosing the usual network traffic design and size.IDS monitors and examines network-generated events to spot anything unusual and notify sensor nodes of an intrusion [8].Data received from sensors may be used in real time by intrusion detection/prevention systems and cyber threat analysts to identify useful information.With the aid of this data, security solutions can be created to identify weaknesses and assaults.
One of the most popular ways to help IDSs better identify and distinguish intruders is to combine them with artificial intelligence (AI) approaches [9].To detect DoS in WSN, the AI-based classification algorithm has been applied.This study used deep learning techniques to assess the defences against DoS attacks in WSN.To verify the efficiency of these defence methods in identifying and categorizing the many types of DoS assaults, they were assessed on a unique Wireless Sensor Network DataSet called WSN-DS, which included several normal and attack circumstances.The primary goal of attack detection, classified as a classification challenge, is determining if an assault is Flooding, Black Hole, Normal, Time Division Multiple Access (TDMA), or Gray Hole.Thus, this study suggested a unique deep-learning architecture to address the issues of attack detection.This study's primary contribution is as follows.
Pre-processing the data represents the initial stage of the suggested method.Here, the One-Hot Encoding approach changed the categorical data into numeral information.Following that, the normalization process is carried out to determine the normalized value using average and standard deviation values.
− Then, to select the best subset of aspects, this research proposed a Manta Ray Foraging Optimization, which makes every possible feature combination and returns the set of attributes that performs the best.Moreover, to balance the dataset, SMOTE oversampling approach is employed, which increases the classification performance.− Finally, this research proposed a CNN-SVM-based deep learning approach to classify the attacks.Therefore, the suggested multi-class intrusion detection technique can properly categorize particular class assaults.− This article is organized as follows: Section 2 reviews the existing artificial intelligence-based malicious threat detection approaches in WSN.Section 3 describes the proposed deep learning-based attack detection framework.Next, model implementation and their consecutive outcomes were discussed in Section 4. Finally, Section 5 concludes this research paper.assaults, a LightGBM algorithm is then used.But this approach does not employ a distributed mechanism to further cut down on time.
For intrusion detection, in [15] SVM had the maximum accuracy compared to other algorithms, whereas Gradient Boosting Classifier had the minimum accuracy rating.It was discovered after examining the efficacy of several machine-learning techniques.However, this study is still not focused on multi-class classification and considering features to choose the most crucial aspects and improve accuracy.
In paper [16] the LightGBM method and autoencoders were suggested for constructing a network intrusion detection system, with the latter being used for feature selection and the former for training and detection.A foundation for intrusion detection is provided when data collection, including network intrusion behaviours, is fed into an autoencoder because there is a significant difference in restructural error between the input data and the output data produced by the autoencoder.An adequate threshold is determined based on the reconstruction error to discriminate between defensive and offensive actions symmetrically.However, this strategy was limited to binary class classification.
A unique feature selection technique called dynamic recursive feature selection algorithm (DRFSA), which determines how many characteristics are appropriate for categorization, was suggested by the authors in their paper published in Science [17].Employing fuzzy temporal constraints as a smart addition to the decision tree algorithm is also suggested to categorize network users and traffic better accurately.Convolution neural networks are also used for data classification in huge volumes.No distributed environment was used to implement this work.
In paper [18] the authors suggested a network intrusion detection system based on LightGBM and adaptive synthetic (ADASYN) oversampling technology.The authors initially used information preprocessing to normalize and one-hot encode the original data to lessen the impact of the top or lowest ranking for all requirements.Next, the ADASYN oversampling method was used to deal with the poor rate of minority assault detection brought on by an imbalance in the training data.Finally, the system's temporal complexity is reduced while preserving detection accuracy by using the LightGBM ensemble learning model.A big data environment is not considered, though.
In study [19] the authors suggested a classification-based network attack detection method for massive amounts of data.The SMOTE + Tomek-Link -Hybrid Deep Learning (STL-HDL) method combines a hybrid deep learning network here with data balancing.The STL ensemble method was employed in the research for data balancing.In study [20] the authors suggested a passive-aggressive online classifier and an information gain ratio-based model.First, the important parts of the sensor data are chosen using the information gain ratio.Second, several forms of Denial of Service assaults have been identified and categorized using the online Passive Aggressive algorithm.However, the data balance has not been taken into account.
Study [21] suggested a light-weight Intelligent Intrusion Detection Model for WSN.Utilizing the k-nearest neighbour algorithm (kNN) and the sine-cosine algorithm (SCA) allows for the intelligent identification of various threats, including unknown assaults, while significantly improving classification accuracy and reducing false alarm rates.The polymorphic mutation (PM) technique is employed to compensate for the decrease in optimization precision, and compact machine SCA (CSCA) is employed to reduce computation time and space.On the other hand, the big data environment is not considered.
In [22] the authors developed a memory-efficient, light-weight anomaly detection system that maintains high accuracy while reducing computing complexity.Concepts of dimension reduction and one-class learning were used in the study.The One-Class Support Vector Machine (OCSVM) was employed for one-class learning and size reduction, and the Candid Covariance-Free Incremental Principal Component Analysis (CCIPCA) approach was employed.The normal reference model must be adaptively learned over time to guarantee that sensor data is accurate because it gets rigid.
In a WSN, in study [23] the authors suggested a technique used for monitoring and thwarting DoS and distributed DoS assaults.The suggested approach uses an RNN as a classifier.However, the dispersed environment is not taken into account in this work.
Paper [24] provided a unique perspective on the malicious security threats in WSNs using an ensemble learning-based quick intrusion detection system that can handle continuous and dynamic data streaming.Although it generates superior results, the pre-processing with data reduction and parameter adjustment is not carried out, which might increase the classifier's effectiveness.
In [25] the authors developed a CNN-LSTM network to identify and categorize DoS intrusion threats.To enhance the accuracy of the classification findings, this strategy does not balance the dataset and yields worse performance outcomes.
1300 Информатика и автоматизация. 2023.Том 22 № 6. ISSN 2713-3192 (печ.)ISSN 2713-3206 (онлайн) www.ia.spcras.ru To identify DoS assaults targeted specifically at WSNs, the authors in [26] suggested a special DoS Intrusion Detection System (DDS).STLGBM-DDS, the proposed system, integrates the LightGBM machine learning algorithm, data balance and feature selection techniques to create an ensemble intrusion detection system.It was created using the big data platform Apache Spark.The data imbalance's impact on system performance was reduced by treating the data imbalance utilizing the SMOTE and the STL sampling methods.Information Gain Ratio is also employed as a feature selection strategy during the data preparation.However, using incorrect neural network parameters has impacted classification accuracy.
Considering the literature review cited earlier, almost all studies that advocate for deploying intrusion detection systems acknowledge that the data imbalance problem is still unresolved.Furthermore, most research has overlooked feature selection.To illustrate the need for large data environments as the amount of WSNs data increases daily, not all required work is completed in one.
3. Proposed Method.WSN deployments have increased dramatically during the past several years.WSNs are being used in various applications across numerous sectors because of their tiny size and low cost.WSNs are exposed to several distinct security concerns and attacks regarding data collection and transmission.The DoS attack is among the most widespread WSN assaults that can target any protocol stack layer.Attack detection is a crucial responsibility in this case for protecting the network and the data.To provide security, this research proposed a deep learning framework.This framework consists of the following steps: data pre-processing, data normalization, feature selection, data balancing, and attack classification shown in Figure 1.The initial data pre-processing stage involves preparing the raw dataset for categorization algorithms -the One-Hot Encoding technique, which transforms the categorical data into numeric data.Then the Normalization Process was carried out to determine the normalized value using mean and standard deviation values.After normalization, removing unnecessary and redundant features and choosing the best subset of features is crucial using a feature selection approach that uses a process called Manta-Ray Foraging based Feature Selection that assesses each feature set using a fitness function.It implies that this method attempts to make every possible feature combination and returns the set of attributes that performs the best.After that, we must balance the imbalanced dataset to increase classification performance.The assault detection dataset contains imbalanced classes, which hurts classification performance.Oversampling with under-sampling approaches was used in our research.Noise and unwanted data size growth are caused when oversampling techniques are used alone.The imbalanced class problem is addressed in this research using the SMOTE approach.Following that, the dataset must be divided for testing and training purposes.

WSN-DS Dataset
Attacks are finally identified using the CNN-SVM-based Deep Learning approach, in which the Convolution layer, Maxpooling layer, Batch Normalization, Dropout layer, and Dense layer are employed.Finally, the SVM classifier is utilized to classify the attack kinds.The ADAM optimizer will tune the deep learning model's hyper-parameter to detect intrusions with a high detection accuracy by minimizing the error rate.As a result, the proposed multi-class intrusion detection approach can accurately classify the specific classes such as Flooding, Blackhole, Normal, scheduling, or Grayhole.The following sections are arranged to explain the proposed method process in detail.

Dataset Description.
A WSN-DS dataset was constructed in [27] to detect and classify the attacks on WSNs specifically.The Low-energy adaptive clustering hierarchy (LEACH) protocol was selected to create the dataset since it is one of the most popular and regularly utilized routing protocols in WSNs.The data were gathered in the NS-2 simulation environment.The dataset includes 23 characteristics that were obtained using the LEACH routing technique.The LEACH protocol is a routing protocol that employs 23 characteristics to indicate the status of each sensor node in the wireless network.There are 23 features, including Id, Time, Is_CH, Who_CH, RSSI, Dist_To_CH, M_D_CH, and A_D_CH.Energy that is now being used, consumed, ADV_S, ADV_R, JOIN_S, JOIN_R, ADV_SCH_S, ADV_SCH_R, rank, DATA_S, DATA_R, Data_Sent_BS, Dist_CH_BS, Send_code, and attack_type.However, as indicated in This WSN-DS dataset served as the basis for all evaluations in the fundamental research.The dataset was selected as that includes DoS assaults specific to WSNs.

Data Normalization and Encoding.
Constructing the raw dataset suitable for classification algorithms is the first phase in data preprocessing.Due to this, the One-Hot Encoding technique was employed in this work to first convert the dataset's category values to quantitative data, after which the normalization step was carried out by Equation (1).
where,   is the original value,   ′ are the normalized value,  and  are the mean and standard deviation values, respectively.As a result, each numerical value in the data set was changed to a number between 0 and 1.
For instance, the Attack Type attribute in the WSN-DS dataset contains textual descriptors like Normal, Grayhole, Blackhole, TDMA, and Flooding.AI-systems are prevented from using these textual descriptions to continue their calculations.Consequently, employing our suggested One-Hot Encoding in conjunction with the normalizing method is required to transform these descriptors into numeric values.
The process of One-Hot encoding can be applied in the WSN-DS dataset: 1. Identify the categorical variables: The attack type is described in textual format in this data set.
3. Assign binary values: Assign a value of 1 to the appropriate binary column for each data instance corresponding to the respective category.
4. Remove the original categorical variable: Once the One-Hot encoding is applied, the original categorical variable.
The classification labels were changed into numbers via the One-Hot Encoding process, as shown in Table 3. Certain characteristics with large numerical values are prevented after normalization and encoding from impairing the algorithm's performance and having a negative impact.These data are then used in the feature selection method; this is covered in greater detail in the section below.

Feature Selection.
Feature selection is an approach that determines the most effective feature subset by reducing noise and eliminating unnecessary and redundant features in the WSN-DS dataset.It can enhance computation performance by reducing the computational load.This research proposed a Manta Ray Foraging (MRF) optimization to choose the features, which examines each feature set that simulates manta ray foraging behaviours.Three foraging operators are used in this strategy: chain foraging, cyclone foraging, and somersault foraging.

Chain Foraging.
The manta ray chain prefers to devour plankton as its main diet.Hence the MRF optimization algorithm indicates that a place with much plankton is the finest.According to the optimum location and the manta ray in front of it, the MRF algorithm modifies the manta ray's position by omitting the first one.The chain foraging update process is shown in Equation (2): where,    denotes where the  ℎ the manta ray is at iteration;  1,2,3,4 are randomly produced integers in the range [0, 1] that are distinct from one another;  is the weight coefficient;    is the maximum level of plankton concentration.The optimum plankton site, therefore, dictates the position update.

Cyclone Foraging.
In addition to moving in the path of the plankton, each manta ray follows the manta ray in front of it as it swims.Equation ( 4) provides a mathematical representation of the cyclone foraging updating strategy. .
where, ∝ represent the weight coefficient; . determines the most iterations possible; and  5.6 represent the different randomly produced numbers between [0, 1].
The MRF algorithm's exploration and exploitation are mostly driven by this step, which uses the best plankton as a benchmark (forcing the manta rays to move to a random place in the explore regions that is both distant from their current location and the best site for prey).Equations ( 6) and (7) show this hypothesized process.
( ) where,    is the fabricated random place within the permitted range;  and  are the maximum and minimum values that can be found at a specific place; and  5,6,7,8,9,10,11 is the various randomly generated integers between 0 and 1.
( ) ( ) where  is the somersault factor; and  12,13 are various randomly generated integers between 0 and 1.
Algorithm The Id and ADV R features in the WSN-DS dataset show substantial correlations among themselves, even though many other characteristics in the dataset do not.
The correlation between these two features can be calculated using Pearson's correlation coefficient to measure the linear relationship between 2 variables.The corr ( ) function computes the pairwise correlations between the dataset's selected features (Id and ADV R).The Id property was thus dropped from the dataset due to this action.Id, ADV R, and SCH_R, respectively, have the attributes that have the least effect on the class, as shown in Table 4.As a result, the dataset did not include these attributes.The number of features has decreased due to the feature selection procedure, leaving the WSN-DS dataset with only 16 features.
3.4.Data Balancing.The WSN-DS contains unevenly distributed classes, which has a detrimental effect on the precise classification.In study [28] the authors suggested the heuristic oversampling approach known as SMOTE to address the class imbalance in datasets.By oversampling the data from the minority class, this strategy creates fake data.By creating synthetic data, SMOTE also solves the overfitting issue brought on by random oversampling techniques.As a result, this study suggested using a SMOTE to modify the sparse spreading of the minority class samples and usually avoid overfitting.
1308 Информатика и автоматизация.2023.Том 22 № 6. ISSN 2713-3192 (печ.)ISSN 2713-3206 (онлайн) www.ia.spcras.ru____________________________________________________________________ ИСКУССТВЕННЫЙ ИНТЕЛЛЕКТ, ИНЖЕНЕРИЯ ДАННЫХ И ЗНАНИЙ SMOTE selects samples that are near the feature space.From the minority class, a random sample is taken, and the sample's nearest k neighbours are then identified.Following the random selection among the closest neighbours, the distinction between the two sample features is multiplied by a value between 0 and 1 and added to the selected sample value.A line then connects the two sample features, and synthetically samples are produced this way.The kNN algorithm are randomly picked the neighbours depending on the oversampling needed.
The following definition applies to SMOTE samples, which are linear combinations of the minority class's two comparable samples where v is the variation between the two samples; and   is the randomly chosen model of r based on the closest neighbour digit.In order to address the class imbalance issue, SMOTE introduces new samples at random between neighbourhood samples from minority classes, greatly boosting the percentage of samples from minority classes.Figure 2 shows the raw dataset, and Figure 3 illustrates the after-data balancing.Following that, the dataset is split into sections for testing and training.Finally, to classify the attacks, this research proposed a deep learning-based framework described in the following section.

CNN-SVM-based classification.
Attacks are finally identified using the CNN-SVM based Deep Learning approach, in which the Convolution layer, max-pooling layer, batch normalization, Dropout layer, and dense layer are employed.Finally, the SVM classifier is utilized to provide the classification of the attack kinds.
CNN: The most analytical information can be gathered from raw data by CNN.In the proposed framework, the most recognizable features are taken from the raw data using a 5x5 kernel/filter.The input layer's n×n input neurons are convolved with an m×m filter in the convolutional layer, producing an output with the dimensions (n-m+1) × (n-m+1).Each layer's output serves as the following layer's input.
SVM: SVM attempts to denote a multi-dimensional dataset in a space where a hyperplane separates the data into different classes.The SVM classifier can minimize the generalization error.

CNN-SVM:
The hybrid CNN-SVM model, SVM functions as a binary classifier and takes the place of CNN's softmax layer in this scenario, is introduced in the current work.Although CNN is a feature extractor, SVM is a binary classifier.Figure 4  the input features from the convolution layer 3 data as input.These new automatically generated training data features are initially used to train the SVM classifier.To identify the assaults utilized for testing, the trained SVM classifier is employed.5, the hyper-parameter of the deep learning model will be tuned by the ADAM optimizer to identify intrusions with high detection accuracy.Consequently, the proposed CNN-SVM method can accurately categorize certain categories like Flooding, Blackhole, Normal, TDMA, or Grayhole.
4. Expriments and Evaluations.This segment details the performance and comparative findings of the suggested strategy and the implementation outcomes.The proposed method is implemented on the balanced WSN-DS dataset.For the evaluation purpose, we have split the dataset into testing and training in the ratio of 50:50.The simulation was carried out using the MATLAB 2021a tool operated in Windows 7 operating systems with 8GB RAM size of Intel premium processor.
4.1.Evaluation Parameters.This section describes the various performance indicators of the recommended deep learning system for classifying attacks in WSN.
Figure 5 demonstrates that after 32 iterations, the MRF optimization technique decreases the fitness function and converges on the value.To achieve the best characteristics, the optimization was done 100 times.As a result, the MRF technique has a good performance in terms of feature selection.
Figure 6 shows the multi-intrusion detection system outcomes of the suggested CNN-SVM-based method.Figure 6 classifies the assaults in the WSN-DS dataset.A confusion matrix displays the assaults properly detected on the main diagonal (top left to bottom right).The erroneous labels are visible in the remaining cells, referred to as true negatives or false negatives.The suggested hybrid model, therefore, produces superior outcomes.From that confusion matrix, we obtain the following performance values.The efficiency of the unique strategy for attack categorization is assessed using the performance of our suggested method and several measures, including accuracy, F1 Score, precision, and recall.The following formulas were used to assess the Accuracy, Precision, Recall, and F1 Score performance metrics.
Figure 10 shows the entire recall comparison.Using CNN-SVM increases the recall of the suggested method.We are comparing the recall of 1314 Информатика и автоматизация. 2023.Том 22 № 6. ISSN 2713-3192 (печ.)ISSN 2713-3206 (онлайн) www.ia.spcras.ru____________________________________________________________________ ИСКУССТВЕННЫЙ ИНТЕЛЛЕКТ, ИНЖЕНЕРИЯ ДАННЫХ И ЗНАНИЙ our suggested method to the baseline DT, RF, NB, LR, MLP, CNN, LSTM, and CNN-LSTM such as 0.98%, 0.98%, 0.93%, 0.97%, 0.96%, 0.98%, 0.99%, and 0.98%.Consequently, our original, distinctive technique has a 100.0%recall compared to standard methods.ISSN 2713-3206 (online) www.ia.spcras.ru0.98%, 0.98%, and 0.98%.Because of this, our innovative, distinctive method has a 0.9997% F-Score, which is better than baseline methods.The overall comparison of the performance parameters is tabulated in Table 7.  selection algorithm.The peculiar feature structure of the WSN-DS dataset is thought to be the root cause of this circumstance.The NB algorithm had the worst results.Since it could not recognize data from the TDMA and Flooding classes with a high rate, the Naive Bayes method performed the worst.Figures 8-11 and Table 7 compare classification algorithms based on accuracy, precision, recall, and F1-Score characteristics.When the outcomes of these parameters are analyzed, the suggested technique yields the best results for each parameter.

Fig. 1 .
Fig. 1.Architecture of the proposed approach

Fig. 4 .
Fig. 4. Hybrid CNN-SVM approach A 28×28 normalized data matrix is provided to the CNN layer as an input.The convolutional layers have a stride of size two and a 5×5 convolutional filtering.The feature map layer's convolution layers 1, 2, and 3 extract values that are also considered distinguishing properties of the input data.Here, dropout is a powerful regularization technique that can significantly reduce overfitting in Deep Neural Network (DNNs).By randomly dropping out neurons during training, the network learns to generalize better to new data.The CNN is trained after going through multiple epochs and until the training process converges.Here, the SVM classifier takes the place of CNN's last layer.The SVM classifier considers 1310 Информатика и автоматизация. 2023.Том 22 № 6. ISSN 2713-3192 (печ.)ISSN 2713-3206 (онлайн) www.ia.spcras.ru

Fig. 7 .
Fig. 7. Performance parameters of the proposed approach

Fig. 8 .
Fig. 8.Comparison of Accuracy Consequently, our original distinguishing technique has a 99.96% accuracy rate, outperforming conventional techniques.Figure9shows the comparison of complete precision.Using CNN-SVM enhances the suggested method's accuracy.As compared to the baseline, our suggested strategy achieves greater precision than DT, RF, NB, LR, MLP, CNN, LSTM, and CNN-LSTM such as 0.97%, 0.98%, 0.88%, 0.97%, 0.96%, 0.98%, 0.99%, and 0.98%.Our innovative distinguishing method has a 0.9994% precision, outperforming conventional techniques.Figure10shows the entire recall comparison.Using CNN-SVM increases the recall of the suggested method.We are comparing the recall of
Neural Network-Long Short Term Memory (CNN-LSTM) is utilized for classification.However, a WSN-DS real-time dataset has not been utilized to evaluate how well the proposed strategy worked.

Table 1 ,
there are only 19 features total in the dataset file, in addition to the class label.

Table 1 .
A detailed description of the attributes of the WSN-DS datasetThe WSN-DS dataset has 374.661 samples.The samples in the dataset are separated into five groups, four of which correspond to different forms of DoS attacks: Black hole, grey hole, flooding, TDMA/Scheduling, and normal.Table2displays the precise data distribution of each class.

Table 2 .
The number of samples in each class of WSN-DS Attack

Table 3 .
Findings from Data Normalization and Encoding

Table 4 .
Feature selection results of the proposed approach

Table 6 .
Performance Evaluation

Table 7 .
The striking aspect of the results is that, on the WSN-DS dataset, the CNN and LSTM deep learning algorithms outperform the CNN-SVM hybrid strategy separately.Data balancing enhances the performance of the DoS intrusion detection system greatly.In research published in the literature, hybrid approaches frequently outperformed individual deep learning techniques, whereas individual methods outperformed hybrid methods on the WSN-DS dataset with the feature 1316 Информатика и автоматизация. 2023.Том 22 № 6. ISSN 2713-3192 (печ.)ISSN 2713-3206 (онлайн) www.ia.spcras.ru Comparison of Performance Parameters Methods Accuracy (%) Precision (%) Recall (%) F-Score (%)