APPLE LEAF DISEASE CLASSIFICATION USING IMAGE DATASET: A MULTILAYER CONVOLUTIONAL NEURAL NETWORK APPROACH

. Agriculture is one of the prime sources of economic growth in Russia; the global apple production in 2019 was 87 million tons. Apple leaf diseases are the main reason for annual decreases in apple production, which creates huge economic losses. Automated methods for detecting apple leaf diseases are beneficial in reducing the laborious work of monitoring apple gardens and early detection of disease symptoms. This article proposes a multilayer convolutional neural network (MCNN), which is able to classify apple leaves into one of the following categories: apple scab, black rot, and apple cedar rust diseases using a newly created dataset. In this method, we used affine transformation and perspective transformation techniques to increase the size of the dataset. After that, OpenCV crop and histogram equalization method-based preprocessing operations were used to improve the proposed image dataset. The experimental results show that the system achieves 98.40% training accuracy and 98.47% validation accuracy on the proposed image dataset with a smaller number of training parameters. The results envisage a higher classification accuracy of the proposed MCNN model when compared with the other well-known state-of-the-art approaches. This proposed model can be used to detect and classify other types of apple diseases from different image datasets.


Introduction. The cultivation area and production of apples in
Russia are leading in the world, and it is one of Russia's most important economic crops [1].Due to the economic growth, apple planting area and production increase every year.Global food security is threatened by climate change (a comprehensive analysis of the individual and combined effects of ozone trends on global climate change 2000-2050) [2] and plant diseases (at least 10% of the world's food production is lost due to plant diseases) [3].In developing countries like Bangladesh, the identification of plant diseases is done by visual observation by the farmers, where proper identification of the diseases usually depends on the skill, experience, and ability of the farmers.There is always a risk of incorrect detection and classification of plant diseases.Sometimes, the agronomists may fail to properly identify leaf disease.This is a big challenge to accurately identify leaf disease, timely treat, and protect crop from damage.
In recent years, computer vision and deep learning techniques have been widely used in agriculture, and a number of methods for the identification of crop diseases have been acquired [4,5].Popular methods that are widely used to detect crop diseases include artificial neural network (ANN) [6], K Nearest Neighbors (KNN) algorithm [7], and so on.There are several diseases that attack apples, the major ones are apple scab (Venturia inaequalis), Apple black rot (Botryosphaeria obtusa), and apple cedar rust (Gymnosporangium juniperi-virginianae).The proper care of apple trees using fertilizers can help the farmers.However, apple leaf diseases cause low production, financial losses, and a decrease in the quality of fruit industry products.Therefore, there is a necessity of accurate methods for detecting apple leaf diseases and preventing losses by taking proper actions.
To solve this problem, early detection of apple leaf diseases is necessary.Manual detection of apple leaf diseases is carried out either by farmers or agricultural scientists.Manual detection is a challenging and time-consuming task.To address this problem, many researchers around the world have introduced various state-of-the-art systems for automatic detection of apple leaf diseases through different machine learning [8,9] and deep learning [10,11] methods.However, these existing systems use a very high number of training parameters and a large number of datasets.Therefore, the training and prediction time of these systems is very long and requires high computing power.There are two noticeable advantages of deep learning methods over machine learning methods.First, they automatically extract various features from raw data, so there is no need for an additional feature extraction module.Second, deep learning methods reduce the time required to process large, high-dimensional datasets.
This paper proposes an MCNN model for automatic apple leaf disease classification with fewer training parameters.In addition, the research includes effective data augmentation and image preprocessing operations.The proposed method boosts the apple leaf disease recognition accuracy and reduces computational time.The rest of the paper is arranged into four sections.Section 2 discusses several state-of-the-art methods for the automatic detection and classification of apple leaf diseases described in the literature.In Section 3, the methods used to design the multilayer convolution neural network are described.The results from the apple leaf disease classification model are presented in Section 4 and Section 5 conclusions.
2. Related Works.Modern technology makes the agriculture sector more advanced by early disease detection and reduction of human efforts.There are various techniques for automated detection and classification systems using deep learning, such as brain tumor detection using MRI images [12], tomato crop disease classification using deep learning [13], and so on.
In order to improve the accuracy of deep neural networks, Xie et al. [14] proposed a real-time grape leaf disease detector using improved deep convolutional neural networks.Initially enlarges the grape leaf disease image using digital imaging technology and creates a grape leaf disease (GLDD) dataset.Based on GLDD and the Faster R-CNN detection algorithm, a deep learning-based Faster DR-IACNN model with a higher feature extraction capability for grape leaf disease detection is presented.The implemented detection model Faster DR-IACNN achieves a precision of 81.1% mAP on GLDD, and the detection speed reaches 15.01 FPS.Sun et al. [15] proposed three major steps to detect maize leaf blight diseases using a convolutional neural network.They recommended a way to consolidate dataset preprocessing, fine-tuning network, and detection module.They claimed in their paper, that data preprocessing reduced the influence of high-intensity light on image identification and improved accuracy.They used the loss function with Generalized Intersection Over Union (GIoU) to optimize the detection processes.The proposed model achieved the highest mean average precision (mAP) 91.83%.Sabrol et al. [16] proposed a system to identify tomato plant disease by using a tree classifier model from the tomato leaf image dataset.The classification was conducted by extracting color, shape and texture features from tomato plant images.Five types of tomato diseases and one healthy were classified which used 382 tomato images and overall 97.3% of classification accuracy was achieved.
Aiming at the complexity of identifying apple leaves disease, Yadav et al. [17] presented an automatic technique for apple leaves disease detection using a convolutional neural network.This model uses preprocessing and fuzzy c-mean clustering for the identification and classification of apple leaves disease.Authors claim that the proposed model provided overall 98% accuracy.Also, Baranwal et al. [18] presented an automatic technique for apple leaves disease detection using a convolutional neural network at SUSCOM-19 which was organized by Amity University in India.The dataset with 1000 samples of healthy leaf image and 1526 unhealthy leaf images are used in their project.Image Compression and filtering techniques with convolutional neural networks are used for the automatic identification of apple leaves disease.The convolutional neural network model is used similarly to the LetNet architecture and the proposed model has offered 98.54% apple leaves disease detection accuracy.
Traditional deep learning methods have large progress in the leaf disease recognition field.Bin Liu et al. [19] introduced a novel architecture of a deep convolutional neural network based on AlexNet for the identification of apple leaves diseases using a dataset that included 13689 images.The Google Net's inception is utilized to enhance and boost the feature extraction ability.The image processing techniques are applied to avoid overfitting in the training processes.Nesterov's Accelerated Gradient (NAG) optimization algorithm is applied to train the CNN-based model.This proposed novel CNN model architecture automatically detects leaf diseases with 97.62% accuracy.Kerkech et al. [20] proposed a model for automatic vine disease detection using the registration method on Unmanned Aerial Vehicle (UAV) image dataset.Also, they used a traditional CNN model to classify each pixel according to different instances.They claimed that the proposed method achieved more than 92% of apprehension at the level of grapevine and 87% at the leaf level.
Sanga et al. [21] proposed a disease detection application for banana plants using five different convolutional neural network architectures (VGG-16, ResNet-152, ResNet-50, ResNet-18, and InceptionV3).They found that ResNet-152 architecture performed an accuracy of 99.2%, which is better than other models.They also proposed a mobile application, so that farmers could easily detect diseases by uploading banana leaf images with their smartphones.Chohan et al [22] proposed a similar work using VGG-19 and InceptionV3 CNN architectures for automatic plant disease detection and classification using the PlantVillage image dataset.They used data augmentation to enlarge the dataset.They claimed in their paper, that the VGG-19 model performed with 98% training accuracy, and the InceptionV3 model performed with 95% testing accuracy.
Dealing with YOLO architecture, Wu et al. [23] introduced the YOLOv4 model and data augmentation methods for the apple picking robot to identify apples quickly and accurately.They used EfficientNet architecture and convolution layer (Conv2D), instead of Cross Stage Partial Darknet53 (CSPDarknet53) due to the extensive size and calculation of the YOLOv4 Model.The apple identification is performed on 2670 samples and the test result shows that the EfficientNet-B0-YOLOv4 model is better than YOLOv3, YOLOv4, and Faster R-CNN.The proposed EfficientNet-B0-YOLOv4 model offers accurate identification results, and it can be used for the vision system of picking robots in the apple industry.Coulibaly et al. [24] introduced a disease detection system for millet crops in the agricultural sector.Their approach was used to extract millet crops leaf's features based on the transfer learning technique of the neural network model.The test dataset used to evaluate the performance of the model includes 18 images with diseases and 9 images without diseases.A pretrained VGG16 model had been used to transfer its learning ability to their proposed CNN network, where the best accuracy achieved 95%, precision of 90.50%, recall of 94.50% and the f1-score of 91.75%.
In this study, we proposed a multilayer convolutional neural network 3. Material and Methods.This section discusses the materials and methods that elaborate the proposed model.Figure 1 shows the flow diagram of the overall systems, which lists the main steps.The detailed work of these processes is presented in the following subsections.Fig. 1.Workflow for the whole system 3.1.Proposed Dataset.Dataset is one of the most important parts of a training network; we analyzed 480 images of apple leaves with high resolution.All the images collected for the dataset were downloaded from the Internet and searched for apple leaf diseases from various sources in different languages such as English, Russian, Bengali, and Hindi.The images in the dataset were grouped into three different classes.The dataset images consist of apple black root, apple cedar rust, and apple scab which are three common apple leaf diseases (Figure 2).
In order to avoid duplicate images, the comparing procedure was applied to remove duplicate apple leaf images using a developed python script.The script removed all duplicate apple leaf images by comparing metadata such as image name, size, and date.After that, the images were evaluated by human experts.The next step was to increase the apple leaf images dataset with augmented images.The main idea of this study is to train MCNN to learn the features of apple leaf images.Therefore, the chance to learn appropriate features was increased by using more augmented apple leaf images for the MCNN network.The image data augmentation process is described in Section 3.2.Finally, an apple leaf image dataset containing 3840 augmented images and 1141 images for validation has been created (Table 1).All apple leaf images are in RGB color and JPG format.

Data Augmentation.
The main purpose of using data augmentation is to enlarge the dataset and introduce slight distortions in images, which helps to reduce overfitting during the training phase.OpenVX supports two commonly used image enlargement methods, which are affine transformation and perspective transformation [25].In the affine transformation, we need three points from the input images and their corresponding locations in the output images.Affine transformation creates an  2×3 matrix, which defines a pixel coordinate mapping from the output to the input image following Equation (1): , .
x M x M y M y M x M y M Where,  0 ,  0  ,  = coordinates of a pixel in the input and output images, and M = affine matrix.Perspective transformation creates a 3 × 3 transformation matrix.It requires 4 points in the input image and corresponding points in the output image.The perspective transformation is defined by an  3×3 matrix following Equation (2).
Where, ,  = pixel coordinates of the output image, and   ,   ,   = uniform pixel coordinates of the input image.These two transformations are applied in the proposed dataset (Figure 3), where the first image is an affine transformation; the second image represents a perspective transformation.Table 1 shows the diseases along with the number of original images and the number of augmented images for each class.Additionally, it shows validation images for the disease classification model.Separating the data into training and test sets helps us find the best model and prevent overfitting.We run the experiments across a whole range of train-validation sets (where, 80% of the whole dataset is used for training, and the rest 20% for validation).

Image Preprocessing.
Preprocessing is a familiar name for operation on images, and it helps to improve the image data for further processes.The two different methods are used in the preprocessing stage, namely the OpenCV crop method for resizing the images and the histogram equalization method for contrast enhancement.The images captured may have different shapes and proportions, so the pictures are preprocessed and taken to the same size, removing noise, background, and distortions.The dimension of the input image is 256 × 256 to reduce the training time.So we resized our image dataset to 256 × 256 using the OpenCV 3.4.3with Python 3.7 framework [26].Further, the contrast of the apple leaf images is improved using the histogram equalization method [27] following Equation (3).
Where,   = cumulative frequency,    = minimum value of cumulative function,   � , � = intensity of current pixel,  ×  = product of the number of pixels in rows and columns, and L = number of intensities.

MCNN Network Model.
Convolutional neural networks which are specifically designed to deal with 2D shapes were first introduced by Lecun et al. in 1998 [28].The proposed multilayer convolutional neural network (MCNN) consists of convolution layers, batch normalization layers, ReLU layers, max-pooling layers, and fully connected and softmax layers.The MCNN model has three blocks; the first block includes convolution, batch normalization, ReLU activation function, and maxpooling layers.The rest of the blocks include a convolution layer, ReLU function, max-pooling followed by a fully connected layer, and a softmax layer as shown in Figure 4.
The convolution operation was used to extract features such as color and edges from an apple leaf image.In this work, the size of the filter has been fixed in all the convolution layers, but the number of filters has been changed.In the first convolution layer, the number of filters is 8, while in the second and third convolution layers are 16 and 32 respectively.The main function of these layers is to extract properties from the input image.The batch normalization layer was used to speed up the training of multilayer convolutional neural networks and reduce the sensitivity for network initialization.Rectified Linear Unit (ReLU) activation function has been used to eliminate negative values, which can be represented by Equation ( 4) [29].
The max-pooling layer contains parameters such as the number of filters and the number of step sizes, which were used to reduce samples by choosing the maximum value and excluding the remaining value.The features were extracted from convolution1, Relu1, max-polling1, convolution2, Relu2, max-polling2, convolution3, Relu3, max-polling3.The fully connected layer belongs to the three classes of apple leaf diseases in this work.The class number represents the number of neurons used to connect each input to all neurons.The softmax function has been used to calculate the probability of each target class with the range from 0 to 1.It returns the probabilities for each class and the target class, which have a more high probability.
In order to verify the performance, the proposed network is trained using the apple leaf disease identification dataset [30].It includes a set of 5461 images of apple black root, apple cedar rust, and apple scab leaves.The proposed method is experimented with using Google Colab (Jupyter Notebook) deep learning framework with a single 12GB NVIDIA Tesla K80 GPU.In addition, the performance of the MCNN model is shown using the Confusion Matrix Method (CMM) on the proposed dataset (Figure 6).The CMM method was applied to evaluate the classification accuracy for each class.The proposed MCNN model correctly predicts an average of 90 images out of 100 images.Also, the CMM method can be used to find the True Positive and False Positive Rates according to Equation ( 5) and Equation ( 6) [32] From Table 3, it can be observed that the proposed MCNN model achieved 98.47% testing accuracy, which is more than the testing accuracies in the research works done by Khamparia et al. [33], with a testing accuracy of 86.78%, Tiwari et al. [34], with a testing accuracy of 97.8%, and Mohameth et al. [35], with a testing accuracy of 98%.The testing accuracy of the proposed model is slightly lesser than the testing accuracies in the research works done by Ferentinos [36] with a testing accuracy of 99.5%.However, in the proposed work only 688,315 training parameters are used, which is much less as compared to the number of training parameters in the state-of-the-art systems.The proposed model can be used for automatic plant disease detection on low computational power systems with less training time and prediction time.The experimental outcome has shown the higher performance of this work compared to other methods.Also, acceptable results have been achieved.It's indicated that deep learning is a significant method for leaf disease classification.
In the future, the detection efficiency of the MCNN method will be improved by the utilization of advanced deep learning-based image classification techniques with a web application for a real-time disease monitoring system.Additionally, we intend to expand our dataset to include more apple leaf images in order to build better models for the future.

(
MCNN), data augmentation methods, and image preprocessing methods to overcome the drawbacks of the previous methods and provide a practical solution.The limitations of the previous study and the main contributions of this study are summarized as follows:1.The previous models have limitations in properly taking advantage of data augmentation techniques.The proposed model uses various image augmentation techniques such as affine transformation and perspective transformation to enhance the proposed dataset.2.All of the studies reviewed above used a large number of learning parameters.Training a model with a large number of training parameters requires high computing power.Proposed multilayer convolutional neural network that reduces the dimensionality of the input of apple leaf images using image preprocessing operations such as the OpenCV crop method and the histogram equalization method.Reducing the dimension of apple leaf images before classification reduces the number of training parameters.

Fig. 2 .
Fig. 2. Apple leaf disease images from the proposed dataset: (a) apple black rot leaves; (b) apple cedar rust leaves; (c) apple scab leaves

Fig. 4 . 4 .
Fig. 4. The architecture of the proposed multilayer convolutional neural network Other programming languages can be used such as Matlab, R, etc., to implement the proposed MCNN model.To train the MCNN network, the Adam optimizer [31] has been used with a batch size of 10 and 40 epochs.The training hyper-parameters used in this experiment are shown in Table 2.Table 2. Training parameters of the proposed MCNN model Parameter Description Number of epoch 40 Learning rate 1e-3 Batch Size 10 Steps 100 Optimizer Adam (adaptive moment estimation)

Fig. 5 .
Fig. 5. Accuracy and loss graphs during model trainings: (a) training and validation accuracy; (b) training and validation loss

Table 1 .
Dataset for image classification of apple leaf disease .

Table 3 .
Comparison of different state-of-the-arts with the proposed model This paper has presented an automated apple leaf disease classification using a multilayer convolutional neural network model.The MCNN model uses apple leaf images to determine the existence of disease with the highest detection rate of apple leaves.The proposed model encompasses the OpenCV crop method and histogram equalization method based on image preprocessing, affine transformation and perspective transformation-based augmentation, adaptive moment estimation-based parameter optimization, and MCNN-based classification.