DETECTION AND CLASSIFICATION OF BRAIN TUMOURS FROM MRI IMAGES USING FASTER R-CNN

: Magnetic resonance imaging (MRI) is a useful method for diagnosis of tumours in human brain. In this work, MRI images have been analysed to detect the regions containing tumour and classify these regions into three different tumour categories: meningioma, glioma, and pituitary. Deep learning is a relatively recent and powerful method for image classification tasks. Therefore, faster Region - based Convolutional Neural Networks (faster R - CNN), a deep learning method, has been utilized and implemented via TensorFlow library in this study. A publicly available dataset containing 3,064 MRI brain images (708 meningioma, 1426 glioma, 930 pituitary) of 233 patients has been used for training and testing of the classifier. It has been shown that faster R - CNN method can yield an accuracy of 91.66% which is higher than the related work using the same dataset.


INTRODUCTION
Cancer is one of the major causes of death today. According to the reports of World Health Organization (WHO), it is estimated that 9.6 million people worldwide died of cancer in 2018 (https://www.who.int/cancer/en/). 30-50% of these were preventable with early diagnosis. Between types of cancer, brain tumour is one of the deadliest ones. According to statistics, it is estimated that 17,760 adults will die from brain tumours in 2019 [1].
The complex structure of the human brain complicates the diagnosis of the tumour in the brain region. MRI, a useful method for obtaining high quality brain images, is widely utilized for tumour diagnosis. Especially in brain imaging, the MRI method provides a unique appearance in visualization of soft tissues with spatial resolution and contrast resolution.
Detection of brain tumours may be considered as an image segmentation problem where the tumour is labelled on the image. In order to solve this problem, various image processing methods as well as machine learning algorithms have been applied to MRI images by the researchers. Among the studies based on image processing, transform-based approaches and thresholding methods are common. For example, Salem and Alfonso utilized Fast Fourier Transform (FFT) and Minimal Redundancy-Maximal Relevance (MRMR) techniques are used for automatic classification of MRI brain tumour images [4]. In another work, Remya et al. used discrete wavelet transform (DWT) together with Fuzzy C-Means method for segmentation of tumours from MRI images [5]. While thresholding and clustering are common methods for tumour segmentation [6][7][8], it is also possible to combine these with transform-based methods [9].
Even though it is possible to detect tumours using these methods, it takes more processing to classify the tumours. For this purpose, methods based on machine learning are utilized. Shree and Kumar used Berkeley Wavelet Transformation (BWT) and Support Vector Machine (SVM) to detect normal and abnormal tissues from MRI brain images [10]. In the study by Kumar and Vijayakumar the principle component analysis (PCA) and Radial Basis Function (RBF) are used for segmentation and classification of brain tumour and obtained better results [11]. Another well-known method for medical image classification is artificial neural networks (ANN). Ullah et al., used ANN to classify MRI brain images as normal or abnormal. The features of the images were extracted by Haar wavelet and statistical moments [12]. For a similar purpose, Muneer and Joseph utilized ANN together with (PCA) to reduce the dimensionality of the feature vector [13].
Deep learning is one of the machine learning methods that uses neural network architecture with possibly hundreds of hidden layers between the input and output layers [14]. It has been applied in various problems such as image classification [15], object detection [16,17] and speech recognition [18,19]. A common deep learning architecture is convolutional neural networks (CNN) where one of three types of operations are performed: convolution, pooling, or rectified linear unit (ReLU). A typical CNN can decide whether an image contains an object but without its location information. On the other hand, region-based CNN (R-CNN), an extended version of CNN, is mainly used for locating objects in images [20].
Deep learning is a highly effective method in solving medical image analysis problems including lung cancer diagnosis [21], tibial cartilage segmentation [22] and brain tumour detection [23][24][25][26]. Using deep learning methods, automatic segmentation has been successfully performed on large amounts of MRI images [23][24][25]. In particular, CNNbased algorithms for automated MRI segmentation for brain tumour yielded successful results by distinguishing distinctive features [26]. Among more recent solutions to this problem, Sajid et al. proposed a CNN architecture that takes local and contextual information into account [27]. Their method involves a pre-processing step to normalize the images and a post-processing step to eliminate the false positives. The obtained sensitivity and specificity values for glioma detection are 0.86 and 0.91, respectively. In another study for brain tumour detection, a hybrid method utilizing Neutrosophy together with CNN is proposed [28]. The performance of the method is compared with regular CNN, SVM and K-nearest neighbours (KNN) and reported to be outperforming them with an accuracy of 95.62%.
In this work, MRI brain images are analysed using faster R-CNN method to detect and locate tumours in the images. Also, the detected tumours are classified into one of the tumour categories: meningioma, glioma, and pituitary. This method has been chosen because it can perform classification with higher accuracy and speed than regular R-CNN. Also, performance of faster R-CNN method for detecting the type of the tumour has not been investigated on this problem yet. Besides, effects of selecting different thresholds at the output layer have been analysed in detail by computing several performance metrics such as accuracy, precision, sensitivity, specificity, and f-score.
All the coding is performed with Python programming language (version 3.6.6) and implementation of deep learning model is done via TensorFlow library (https://www.tensorflow.org/). The rest of the paper is organized as follows. Theoretical background information about faster R-CNN is mentioned in Section 2. Section 3 contains explanation of the dataset used as well as details of model training. Next, results and discussion are given in Section 4 and the paper is concluded with Section 5.

BACKGROUND
The process of building a CNN involves six main steps: Convolution, Rectified Linear Unit (ReLU), Pooling, Flattening, Fully connected layers, and Softmax function ( Fig. 1). At the initial step, a number of convolution filters are applied to the input image to activate certain features from the images. In order to increase the training speed, negative values are mapped to zero and positive values remained unchanged in ReLU step. The purpose of pooling step is to simplify the output by performing nonlinear downsampling, and hence reducing the number of parameters that the network needs to learn about. These three operations are repeated over tens or hundreds of layers, with each layer learning to detect different features. In the flattening step, all two-dimensional arrays are transformed into one single linear vector. Such a process is needed for fully connected layers to be used after convolutional layers. Fully connected layers are able to combine the entire local features of the previous convolutional layers. The procedure is finished with application of softmax function to provide the final classification output.
It is possible to tell the class of the objects within an image via CNN; however, it does not return any location information. In order to find the object location R-CNN method has been developed. In R-CNN, some candidate regions of the image are selected with some region proposal functions. These regions are cropped and resized to be classified by CNN method. One improved method of R-CNN is fast R-CNN where the entire image is processed rather than cropping the image and resizing the cropped parts. Region proposal functions used in fast R-CNN as well. However, unlike CNN, these regions are not processed directly; CNN features corresponding to each region proposal are pooled by fast R-CNN method. Generation of region proposals in a faster and better way is achieved by region proposal network (RPN). Faster R-CNN method introduces RPN directly in the network thereby enables a higher classifier performance [29].

MATERIALS AND METHOD 3.1 The Dataset
The brain image dataset contains 3,064 MRI slices from 233 patients. Each of these images contains one of the three types of brain tumours: meningioma, glioma, or pituitary. The images have an in-plane resolution of 512×512 with pixel size 0.49×0.49 mm 2 . The slice thickness is 6 mm and the slice gap is 1 mm [11]. Sample images from each of the classes are shown in Fig. 2 and the distribution of the images having these tumour types is given in Tab. 1.

Training
In order to train and test the faster R-CNN classifier, the tumour locations in all of the images should be labelled and saved to a file. For this purpose, graphical image annotation tool called LabelImg [30] is used. Labelimg is written in Python and uses Qt for its graphical interface. Annotations are saved as XML files in PASCAL VOC format, the format used by ImageNet (http://www.image-net.org/).
After labelling process, 80% of the images in the database are randomly selected as training samples. In order to prevent the training set to be dominated by one of the classes, same proportion of images is selected from each class. The remaining 20% is left for testing. A faster R-CNN model is trained on the training data. The parameters regarding the model are as follows:  The loss function that is being minimized during training is provided in Fig. 3. At the beginning of the training, very high loss rate is observed. For example, in the first iteration total loss is approximately 90%. After 140.000 iterations, total loss is decreased to a value about 2%. This is an expected situation indicating that the algorithm performs learning process. Basically, two different loss functions are minimized in this problem. One is related to the correct classification rate and the other one is related to the location of the correctly classified object. Since graphs of both functions are similar, only Total Loss graph is given here as the sum of these two functions.

RESULTS AND DISCUSSION
The 612 samples (constituting 20% of the dataset) are used for testing the faster R-CNN model. For each test image three parameters are returned: (i) class label of the detected tumour, (ii) probability of the tumour being in that class and (iii) location of the tumour. Sample classification results are shown in Fig. 4.
In TensorFlow library, the minimum softmax probability value for assigning a class label to a detected tumour is set as 0.8 by default. This means that a tumour is left unclassified if the associated probability value is smaller than 0.8. By changing this probability threshold, we observed the change in number of unclassified and misclassified samples. Three confusion matrices belonging to three different threshold values (0.8, 0.66, and 0.5) are given in Tabs. 2, 3, and 4, respectively.
Five performance metrics, given in equations 1-5, are calculated using the classification results. Computation of these quantities requires definition of the confusion matrix elements, that are True Positive (TP), True Negative (TN), False Positive (FP) and False Negative (FN). Since values of these elements change according to the reference class label, three different values for each of the performance metrics are calculated for each of the confusion matrices in Tabs. 2, 3, and 4. The related results are given in Tabs. 5, 6, and 7.      It is clearly seen from the confusion matrices that the number of unclassified samples decreases as the decision threshold is decreased. Also, the related performance measures are improved by decreasing the threshold. Considering the accuracy values in Tabs. 5, 6 and 7, 87.42% of the images may correctly be classified when the decision threshold is set to 0.8 and this accuracy is increased to 91.66% when the threshold is set as 0.5. This means that, even though the output probability of the classifier is relatively low for some of the images, the tumour can be correctly detected. On the other hand, majority of the unclassified and misclassified samples belong to "Glioma" class, meaning that the classifier cannot learn this class as effectively as the other two. However, specificity value related to "Glioma" class is the highest for three of the thresholds.

CONCLUSION
In this study, MRI brain images have been used for detection of tumours using a deep learning method, namely faster R-CNN. A model has been trained using 2452 images containing three types of tumours. It may be concluded that faster R-CNN is suitable algorithm for this problem as the achieved classification accuracy is 91.66% which is higher than the related work using the same dataset. Also, it has been observed that some of the tumours are correctly detected with low scores. Therefore, the results are evaluated using three different levels of detection scores. Among the three types of tumours, Glioma has the lowest detection rate with a sensitivity of 88.11% and Pituitary has the highest rate with a sensitivity of 95.19%. Since only one test sample is predicted as false positive for Glioma class, the precision value of the Glioma is the highest, on the other hand high number of false negative samples causes its sensitivity value to be low.
One major advantage of deep learning is that the classification accuracy is improved as the number of training examples is increased. However, this is not the case for earlier machine learning methods in which no increment is observed above a certain amount of training examples. Among deep learning methods, faster R-CNN has been proven to be performing faster with a higher classification accuracy [29]. This method also returns the class label and location of the object together. These properties of faster R-CNN make it suitable to be applied to tumour detection problem where it is important to show the location of the tumour. Therefore, contributions of the study may be listed as: (i) There is a potential for improving the performance of the method when more data is available, (ii) tumours are located and classified simultaneously, (iii) classification performance is improved by changing the threshold value at the output layer.
A careful attention should be paid to the unclassified and misclassified samples. The presence of unclassified samples is something related to detections with low scores. Such samples may be thought to be classified as "no tumours" even though they contain a tumour. In the future, this situation may be overcome by adding healthy images (i.e. images containing no tumours) to the dataset as the fourth class. As for reduction of the number of misclassified samples (particularly the samples from Glioma class), a preprocessing step is planned to be added to the proposed method. The pre-processing step may include algorithms that will emphasize the unobvious features in the given images. Unsharp masking and histogram equalization are two possible examples of these algorithms. Besides, as can be seen from Section 3.2, a set of constant parameters has been used for generating the CNN model. In the future, some optimization algorithms (such as genetic algorithm, particle swarm optimization, simulated annealing, etc.) may be utilized to find the best parameter set giving the highest classification accuracy.