Hybrid ConvNeXt–ViT Feature Extraction for Rice Blast Disease Severity Classification

INTRODUCTION

1.

India's economy depends heavily on agriculture, which contributes around 18% to the national GDP and employs nearly two-thirds of the workforce. This sector helps to improve economic growth through agricultural exports [1]. As a staple food crop, rice farming supports rural livelihoods and contributes to agricultural GDP through production and processing activities [2]. However, rice productivity is challenged by multiple factors, including climate-related changes such as variations in rainfall, temperature, droughts, and floods. Additionally, biotic stresses such as pest infestations and diseases, particularly blast and brown spot, continue to reduce rice yield stability and threaten rice production [3].

Rice blast is one of the most destructive diseases in rice farming, making rice production very challenging. This disease can cause damage to rice plants at any stage of growth and is caused by the fungus Magnaporthe oryzae. The disease first appears as small lesions on the leaves, later spreads to the grains, panicle axes, nodes, necks, and leaf sheaths [4]. As the infection worsens, both the number and spread of lesions on the leaf surface increase, reflecting higher levels of disease severity. This severity is commonly assessed on a scale of 0 to 9, representing the proportion of plant tissue damaged by lesions [5]. Severe blast outbreaks can result in yield losses ranging from 10% to as high as 50% under favorable environmental conditions [6,7].

Proper categorization of blast disease severity is required to determine the level of infection and minimizing yield losses. Also, it simplifies decision-making for sustainable rice production and allows for more effective use of resources [8].

Traditionally, specialists use visual inspection to identify the severity of rice blast by looking at leaves for lesions and other symptoms. This method is laborious and time-consuming because it relies on experts' experience and judgment. Discrepancies between the evaluator’s decisions and early-stage symptoms can lead to inconsistent or inaccurate assessments [9].

Most existing studies focus on the detection of rice leaf diseases, while relatively few have addressed the more challenging task of disease severity classification. A few studies concentrate on two to three severity levels, such as mild, moderate, and severe. For effective disease management, models capable of classifying rice blast disease into severity levels and capturing subtle differences between infection stages are needed. To address this, a newly segregated rice blast severity-level image dataset is created and used to train the proposed model. The proposed model presents a ConvNeXt–ViT model for rice blast disease severity classification. It comprises two parallel feature-extraction branches: ConvNeXt captures fine-grained spatial and textural information, and the Vision Transformer (ViT) learns global contextual dependencies. Features extracted from both branches are fused to generate the final severity-level prediction. This work presents the following major contributions:

RELATED WORKS

2.

The Whale Optimization Algorithm and Adaptive Particle Swarm Optimization were combined in a hybrid bio-inspired optimization approach [10] to enhance rice disease classification using a CNN. The PlantVillage dataset was used to test the strategy, with a focus on segmentation, pre-processing, and optimal feature selection. ANN, SVM, and conventional CNN models were used to compare performance. According to the results, the hybrid approach attained a high accuracy of 97.5%. However, the reliance on the PlantVillage dataset, which is composed of controlled laboratory images, raises concerns about the model's generalizability to real-world field conditions where lighting, background, and image quality vary considerably. Furthermore, the study focuses on disease identification without addressing severity-level classification. The proposed ConvNeXt–ViT framework addresses these limitations by training on a field-representative dataset with seven fine-grained severity levels.

Wang [11] used a convolutional block attention module to develop a double-branch DCNN incorporating a Convolutional Block Attention Module (CBAM) to improve feature representation for rice disease classification. The dual-branch design allowed the model to capture complementary feature representations from different network pathways. The results show that their model outperforms the conventional models VGG-16, MobileNet-V2, EfficientNet-B1, ResNet-50, ResNet50 + CBAM, GoogLeNet, and Inception-V2 in terms of accuracy (98.73%). The model was evaluated solely on disease identification without addressing severity-level classification, which is a critical requirement for agronomic decision-making in real field conditions.

A hybrid model was developed for classifying blast disease severity into four levels based on the percentage of leaf area infected [12]. The dataset, comprising 1908 images from various sources (Mendeley, Kaggle, GitHub, and UCI), was used to train the system. Features were extracted using a CNN, and severity detection was performed with an SVM. The resulting model achieved 97% accuracy, demonstrating the viability of combining deep feature extraction with traditional classifiers for severity prediction. However, the four-level severity categorization remains coarse for practical field applications that demand more granular disease monitoring, and the hybrid CNN-SVM design may limit end-to-end optimization since the feature extraction and classification stages are not jointly trained. Furthermore, the aggregation of images from heterogeneous sources without standardization may introduce inconsistencies in image quality and annotation.

Ritharson et al. [13] introduced the DeepRice framework for the classification of rice leaf disease into five categories—healthy, blast, blight, tungro, and brown spot—with each disease further divided into mild and severe levels. A customized VGG16 model achieved a high accuracy of 99.94%, demonstrating the effectiveness of CNN-based approaches. However, the binary severity classification is relatively coarse and may not fully capture the progressive nature of disease development. Additionally, relying on a single CNN model limits the ability to capture both local and global features. In contrast, the proposed ConvNeXt–ViT framework uses a dual-branch architecture to model both aspects and provides classification across seven fine-grained severity levels for more detailed assessment.

Roy and Kukreja [14] addressed the challenge of rice leaf disease classification and severity estimation using Vision Transformers (ViTs) — an area that had been largely unexplored before their work. Their dataset comprised 3,345 images spanning ten disease categories annotated at three severity levels: mild, moderate, and severe. The contribution lay in leveraging the self-attention mechanism of ViT to jointly model disease identity and progression stage within a single framework. The model achieved a weighted average F1-score of 77.94% for severity prediction. While this work demonstrated the feasibility of transformer-based models for rice disease analysis, its relatively modest F1-score and coarse three-level severity highlight the need for more discriminative architectures capable of fine-grained severity classification — a gap addressed by the proposed ConvNeXt–ViT model in this study.

A multimodal deep learning framework integrating visual and environmental modalities — such as temperature, humidity, and rainfall — alongside RGB leaf images was proposed in [15] for simultaneous rice leaf disease classification and severity prediction. A dual-output CNN architecture was employed to jointly predict disease category and severity level within a unified training pipeline, eliminating the need for separate models for each task. However, the dependency on environmental sensor data introduces practical limitations in resource-constrained or remote agricultural settings where such ancillary data may not be readily available, and the fusion of heterogeneous modalities adds architectural complexity.

PROPOSED METHODOLOGY

3.

Dataset

3.1.

A dataset of 1,440 rice blast images was collected from Mendeley Data [16]. These images were manually annotated into seven severity levels by an agriculture university professor specializing in plant pathology, following standard plant pathology practices. The severity levels were defined based on the estimated percentage of lesion area coverage (1–10%, 11–20%, 21–30%, 31–40%, 41–50%, 51–60%, and 61–70%), as shown inFig. 1.Fig. 1 also shows the number of images in each severity level. This newly segregated rice blast severity-level image dataset has been uploaded to the Mendeley Data repository [17].

Fig. 1. Rice blast disease severity levels Dataset.

Proposed ConvNeXt–ViT model

3.2.

To meet the input requirements of the ConvNeXt and Vision Transformer backbones, all rice blast imager were scaled to 224 × 224 pixels and converted from BGR to RGB format. For the validation and testing sets, resizing and normalization were applied. Data augmentation, such as random rotations and horizontal and vertical flips, was applied to the training set to enhance generalization and prevent overfitting.

Fig. 2 illustrates the Hybrid ConvNeXt–ViT [18] architecture used for disease-specific severity prediction for Blast disease. The image is first pre-processed and then passed through two parallel feature-extraction paths: ConvNeXt [19] and the Vision Transformer (ViT). ConvNeXt processes the input images through stacked convolutional stages to emphasize local details [20], while the ViT branch models long-range dependencies by analyzing relationships among image patches using transformer encoder layers. The features learned from both branches are then combined and forwarded to a classification module consisting of a fully connected layer followed by a softmax function. Attention-based characteristics and localized convolutional information are successfully combined in this hybrid architecture [21].

Fig. 2. Hybrid ConvNeXt–ViT architecture

The ConvNeXt backbone focuses on capturing fine-grained spatial features such as lesion boundaries, texture variations, spot geometry, and chlorotic regions, which are essential for distinguishing different severities of rice diseases [22]. The ConvNeXt backbone begins with a stem convolution, followed by multiple ConvNeXt blocks that progressively refine feature representations, and concludes with global pooling.

The Stem is also called patchify Convolution. The Stem of the ConvNeXt backbone is implemented as a 2D convolutional layer designed to convert the raw input image into a compact grid of patch embedding. It uses a convolution kernel of size applied with a stride of 4 and zero padding, meaning that the filters slide across the image in non-overlapping steps. Given an input image $X$ ∈ $R$^3×224×224 this operation 4×4, partitions the leaf image into 4×4 pixel patches and transforms each patch into a learned feature representation $F$_stem($X$) [23] as expressed in Eq. (1).

(1)

$$F_{stem}(X)\ = \ {Conv}_{4 \times 4\ ,\ s = 4\ }(X)$$

For ConvNeXt-Base, the Stem generates 128 output channels, producing an output feature map of size 56×56×128. The output shape is denoted as tensor $T$_1, and it is expressed in Eq. (2).

(2)

$$T_{1}\ \in R^{56\ \times \ 56\ \times \ 128}\$$

ConvNeXt transforms it through a sequence of hierarchical convolutional stages. The complete transformation is expressed in Eq. (3).

(3)

$$F_{conv}\ = B_{4}(D_{3}(B_{3}(D_{2}(B_{2}(D_{1}(B_{1}(T_{1}))))))))\$$

Where $B$_$i$ stands for the ConvNeXt block in the $i$^th stage, each of which is in charge of using GELU activation, normalization, and depth-wise convolutions to gradually improve the spatial and contextual representations. The component $D$_$i$ represents the down-sampling layer located between consecutive stages, which reduces the spatial dimensions while increasing channel depth, thereby enabling hierarchical feature abstraction. Finally, $F$_conv corresponds to the final feature map, containing the highest-level semantic and texture descriptors extracted by the network.

(4)

$$B_{i}\left( T_{i} \right)\ = T_{i}\ + \ MLP\left( \sigma\left( DWConv\left( T_{i} \right) \right) \right)$$

The ConvNeXt block at stage $i$, as defined in Eq. (4), is designed to refine the input feature map. The input tensor Tᵢ, which contains spatial and semantic information extracted from earlier layers. In the first step, a depth-wise convolution, DWConv(Tᵢ), is applied to process each channel independently using individual spatial kernels. This preserves channel-wise structure and captures disease visual cues such as lesion edges and texture variations. The resulting features $\sigma$ (.) pass through a GELU activation [24] and a multi-layer perceptron (MLP) to enable interaction across channels. Finally, a residual connection adds the original tensor $T$ᵢ back to the transformed output, which ensures that both original and refined information are retained.

(5)

$$D_{i}\left( T_{i}^{'} \right)\ = {Conv}_{2 \times 2,\ s = 2}\ (T_{i}^{'})\$$

The down-sampling layer between the $i$^th Stage and ($i$+1)^th Stage of the ConvNeXt is represented by the operation $D$_$i$($T$^'_$i$) shown in Eq. (5). It applies a 2×2 convolution with stride 2 to reduce the spatial dimensions of the feature map by half. As a result, the output of $D$_$i$ becomes a more compact feature representation that contains richer high-level information. This down-sampled tensor then forms the input to the next hierarchical stage of ConvNeXt, enabling deeper layers to process disease regions with an enlarged receptive field and enhanced discrimination capacity.

(6)

$$z_{conv}\ \in \ GAP(F_{conv}\ )$$

Following the hierarchical progression of ConvNeXt blocks and down-sampling layers, the final extracted feature map obtained from Eq. (3) is $F$_conv, which is subsequently passed through a Global Average Pooling (GAP) layer. The GAP operation shown in Eq. (6) compresses the spatial dimensions $F$_conv by computing the average value of each channel across all spatial locations. This transforms the feature map $z$_conv from a tensor of size 7×7×1024 into a compact vector of size 1024, shown in Eq. (7).

(7)

$$z_{conv}\ \in \ {\ R}^{1024}$$

The ViT model first converts the input rice leaf image $x$ into a sequence of fixed-size patches, a process known as patch embedding. The image is divided into ‘n’ patches [25], and linearly projected using a learnable weight matrix $E$. This produces the patch embedding represented in Eq. (8).

(8)

$$x_{i}\ = \ \left\lbrack x_{1}E;\ x_{2}E;\ x_{3}E;\ldots\ldots\ldots x_{n}E;\ \right\rbrack$$

Where $E$ is a learnable weight matrix for patch projection, and ‘$x$_$i$ $E$’ is the linear embedding of the $i^{th}$ patch. Through this step, the 2D image is effectively transformed into a 1D token sequence compatible with the transformer encoder. Following patch embedding, a learnable classification token $x$_lcls and positional embedding $E$_pos are added to retain spatial information that would otherwise be lost during the flattening process [26]. The sequence of embedded patches $Z$₀ is denoted in Eq. (9).

(9)

$$Z_{0}\ = \ \left\lbrack x_{1cls}\ ;x_{1}E;\ x_{2}E;\ x_{3}E;\ldots\ldots\ldots x_{n}E;\ \right\rbrack\ + \ E_{pos}$$

After positional encoding, the patch sequence passes through multiple transformer encoder layers, each containing a Multi-Head Self-Attention (MHSA) mechanism followed by a Feed Forward Network (FFN) [27]. In the self-attention operation, the attention output is computed and shown in Eq. (10).

(10)

$$Attention(Q,K,V)\ = \ Softmax\left( \frac{QK^{T}}{\sqrt{}d_{k}} \right)V$$

Where the query, key, and value matrices are denoted by $Q$, $K$, and $V$. The input token sequence $Z$₀ is linearly projected using learnable weight matrices to produce these matrices, $Q$ =$Z$₀ $W$_$Q$, $K$ = $Z$₀ $W$_$K$, and $V$ = $Z$₀ $W$_$V$, where $W$_$Q$, $W$_$K$, $W$_$V$ are learnable projection matrices. Then ViT uses multiple attention heads [28] to capture diverse contextual relationships within the rice leaf image. The multi-head attention output is defined in Eq. (11).

(11)

$$MultiHead\left( Z_{0} \right) = \ \left\lbrack {head}_{1;\ldots..\ldots..}{head}_{h} \right\rbrack W^{o}$$

Where $W$^o is the output of the projection matrix and head_$i$ = Attention ($Q$_$i$, $K$_$i$, $V$_$i$). A residual connection is used to combine the output of the Multi-Head Self-Attention (MHSA) module with the original input $Z$₀ through ($Z$₁ = $Z$₀ + MultiHead($Z$₀)). Each token is then passed independently through a Feed-Forward Network (FFN). Eq. (12) represents the FFN used within each Transformer encoder layer [29]. The FFN is applied independently to every token and is defined as

(12)

$$z_{vit}\ = \ FFN(Z_{1})\ = \ GELU(Z_{1}W_{1}\ + \ b_{1}\ )W_{2}\ + \ b_{2}$$

In Eq. (12), $W$₁​ and $W$₂ are learnable weight matrices that project the token into a higher-dimensional intermediate space and then back to the model dimension. The terms $b$_1​ and $b$₂ are learnable bias parameters associated with each linear transformation. The non-linear Gaussian Error Linear Unit (GELU) activation function introduces smooth gating behavior, enabling richer and more expressive feature transformations. This ViT branch outputs a high-level embedding that encapsulates long-range dependencies, subtle texture differences, and global disease-spread patterns. After several encoder layers, a pooler operation is applied to obtain the final 768-dimensional feature vector $z$_vit [30], and it is shown in Eq. (13).

(13)

$$z_{vit}\ \in \ {\ R}^{768}$$

Following the extraction of complementary feature representations from ConvNeXt and the ViT branch [31], the hybrid architecture fuses [32] these two modalities to create a unified descriptor for severity-level prediction. The 1024-dimensional ConvNeXt feature vector given in Eq. (6) and the 768-dimensional ViT feature vector presented in Eq. (13) are concatenated to generate a joint feature embedding $z$_fused [33] described in Eq. (14). The fused output has a total dimensional size of 1792.

(14)

$$z_{fused}\ = \ \left\lbrack z_{conv}\ ;\ z_{vit}\ \right\rbrack\ \in \ R^{1792}$$

This fused vector is then transformed through a series of nonlinear mappings to enhance representational capacity. The first fusion layer applies a linear transformation followed by a ReLU activation, producing the intermediate representation $h$₁ as defined in Eq. (15)

(15)

$$h_{1}\ = \ ReLU(W_{1}\ z_{fused} + b_{1})$$

Where $b$₁ denotes the bias vector, and $W$₁ is the weight matrix of the first fusion layer. Together, they constitute the learnable parameters of the first nonlinear mapping applied to the fused feature vector. A subsequent nonlinear transformation is applied to refine the features further and introduce additional abstraction, resulting in $h$₂, as described in Eq. (16). In this transformation, the term $b$₂ serves as the bias vector of the second fusion layer.

(16)

$$h_{2}\ = \ ReLU(W_{1}\ h_{1} + b_{2})$$

To reduce overfitting and improve generalization, dropout regularization is applied to the second fusion layer. As a result, the fusion block produces a compact 512-dimensional representation illustrated in Eq. (17).

(17)

$$h_{2}\ \in \ R^{512}$$

Following the feature fusion stage, the transformed representation $h$₂ is forwarded to the final classification module, where the model determines the disease-specific severity level. After the fused feature vector has gone through a fully connected layer, a softmax activation with a weight matrix $W$_c and a bias vector $b$_c is employed to transform its output ŷ into class-probability values. This operation is expressed in Eq. (18). The resulting vector ŷ encodes the predicted probability values over all possible severity classes for the identified disease, serving as the final output of the Hybrid ConvNeXt–ViT severity-level prediction pipeline.

(18)

$$ŷ\ = \ softmax(W_{c}\ h_{2} + b_{c})$$

For the total number of severity classes ‘sc’, the probability value corresponding to each class k, denoted as ŷ_k, is obtained using the formulation shown in Eq. (19). This equation gives the probability associated with each class based on the output logits across all classes ‘sc’. The predicted severity level ĉ, corresponds to the class with the maximum probability is expressed in Eq. (20).

(19)

$$ŷ_{k} = \frac{e^{z_{k}}}{\sum_{j = 1}^{sc}e^{z_{j}}}$$

(20)

$$\widehat{c}\ = \underset{k}{argmax}{\widehat{y}}_{k}$$

RESULTS

4.

Python was used in a Jupyter Notebook environment to carry out all experiments. An NVIDIA A100 GPU was utilised to speed up both training and inference processes throughout the model's implementation on Google Colab.

The suggested ConvNeXt–ViT hybrid model is consistent with the performance findings for blast disease severity classification, which are presented inTable 1. Every evaluation measure shows improvement as the number of training epochs rises. The model performs moderately throughout the first ten epochs, with metric values near 0.80, indicating that it has started selecting important blast-related information.

As the epochs increase to 20 and 30, the model shows improvement with Accuracy rising to 0.8525 and 0.8802, respectively. The same rising trend was seen in other parameters, which showed improved classification consistency and feature extraction. All evaluation measures surpassed 0.90 by 40 epochs, demonstrating a high degree of blast disease symptom detection. The accuracy reached at 0.9263 after 50 epochs, indicating the best performance.

The bar chart inFig. 3 depicts an increase in every performance metric as the number of epochs increases, supporting the numerical results recorded inTable 1.

Fig. 3. Comparison of Results of the Hybrid Model ConvNeXt+ViT for Blast disease

The Blast disease model’s accuracy and loss curves of training and validation sets are shown inFig. 4. Both accuracy curves rise quickly during the initial epochs, indicating fast learning of key Blast features, and later stabilize above 0.9, reflecting strong generalization. The training and validation loss drop sharply at the beginning and continue to decline smoothly, with both curves closely aligned. This tight convergence indicates stable learning without overfitting.

Fig. 4. Training and validation performance curves for the blast disease model.

Fig. 5 presents the confusion matrix for Blast severity classification. The confusion matrix provides detailed class-wise performance insights for the proposed model. It can be observed that the majority of predictions lie along the diagonal, indicating accurate classification across all severity levels. Misclassifications are minimal and primarily occur between adjacent severity classes, such as 21–30% and 31–40%, or 41–50% and 51–60%, which is expected due to their visual similarity. Importantly, there are no significant misclassifications across distant classes, demonstrating the model’s ability to preserve severity order. These results confirm that the proposed model performs consistently across different classes despite class imbalance.

Fig. 5. Confusion matrix for Blast severity classification.

Table 2 presents a comparative evaluation of individual backbone models and hybrid fusion strategies. To ensure comparison with transformer-based approaches, a Vision Transformer (ViT) model was implemented as a baseline. The ViT achieved an accuracy of 0.7972, slightly outperforming the ConvNeXt model (0.7742), indicating the effectiveness of transformer-based global feature learning. However, the performance of the standalone ViT remains significantly lower than that of the proposed hybrid model.

To take advantage of the complementary features of both architectures, several hybrid ConvNeXt–ViT fusion strategies were tested. Early Fusion, which combines features at the initial stage, improved the accuracy to 0.8295, showing the enhanced discriminative capability. Intermediate Fusion, which integrates features at a later stage, achieved an even higher accuracy of 0.8618, indicating more confident predictions of severity levels.

The proposed ConvNeXt–ViT Late Fusion model with pre-trained weights, using a patch size of 8 and a batch size of 32, achieves the highest performance, attaining an accuracy of 0.9263, with corresponding precision, recall, and F1-score of 0.9253, 0.9263, and 0.9175, respectively. In comparison, training the same model without pre-trained weights leads to a drop in accuracy to 0.9032, highlighting the importance of transfer learning in improving generalization. Further analysis shows that the model’s performance is influenced by key hyper parameters. For instance, reducing the batch size to 16 lowers the accuracy to 0.8525, while increasing the patch size to 16 results in an accuracy of 0.8986.

Overall, the findings indicate that hybrid models perform better than individual models. And the proposed ConvNeXt–ViT Late Fusion model with pre-trained weights gives the accurate classification of blast disease severity among the fusion strategies. These results showcase the advantages of integrating transformer-based and convolutional representations for the analysis of disease severity.

Furthermore, the significant improvement over the standalone ViT baseline demonstrates that combining convolutional local feature extraction with transformer-based global context modelling leads to accurate classification.

Table 3 summarizes the performance of the proposed ConvNeXt–ViT model in comparison with several classical CNN-based architectures for rice blast disease severity classification. ResNet-34 records the lowest accuracy of 76.50% among the baseline models, stating that severity patterns are not well presented. With accuracies of 85.25% and 86.64%, EfficientNet and MobileNetV2 performed better, demonstrating their capacity to extract more useful features. VGG16 further improved the classification accuracy to 87.10%, while Xception achieved the highest accuracy among the classical CNN models at 88.94%. Comparatively, the suggested ConvNeXt–ViT model outperformed all traditional CNN-based techniques with 0.9263 accuracy, 0.9253 precision, 0.9263 recall, and an F1-score of 0.9175.

CONCLUSIONS AND FUTURE WORK

5.

A hybrid ConvNeXt–ViT deep learning model for rice blast disease severity classification was introduced in this study. A severity-level annotated dataset comprising seven classes was utilized to address the limitations of existing severity categorization approaches. Experimental results demonstrate that the Late Fusion strategy consistently outperforms both Early and Intermediate Fusion approaches, confirming that independent feature learning before integration yields more discriminative representations. The incorporation of pre-trained weights proved instrumental in enhancing model generalization, particularly under the constraint of limited annotated agricultural data. The proposed ConvNeXt–ViT Late Fusion model achieved a classification accuracy of 92.63%, with a precision of 0.9253, recall of 0.9263, and F1-score of 0.9175, outperforming all evaluated conventional deep learning architectures. Overall, the proposed framework establishes a reliable and accurate solution for automated rice blast severity classification. The integration of convolutional and transformer-based learning paradigms within a unified architecture presents a broadly applicable strategy for fine-grained agricultural disease classification.

Future enhancements of this work will aim to support multi-disease severity detection by including other major rice diseases in the proposed model. Expanding the dataset with field-acquired images under diverse environmental conditions will further improve model robustness and real-world applicability. In addition, future work will focus on validating the model on external datasets, as they become available, to further assess its generalization capability across different data sources.