Note: The complete dataset and all the scripts are available at: https://github.com/tbgicgeb/iRSVPred-2.0
Figure: Flowchart depicting the methodology used for the development of iRSVPred 2.0 web server and an Android-based mobile application.
The distribution of the images into the original dataset used for training and evaluation of AI-based models.
The distribution of original and augmented images into a final dataset used for training and evaluation of AI-based models.
| Predicted variety | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1121 | 1509 | 1637 | 1718 | 1728 | BAS-370 | CSR-30 | DHBT-3 | PB-1 | PB-6 | Unknown | ||
| Actual variety | 1121 | 49 | 0 | 0 | 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 1509 | 0 | 50 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| 1637 | 0 | 0 | 40 | 0 | 0 | 0 | 0 | 0 | 7 | 3 | 0 | |
| 1718 | 0 | 0 | 0 | 50 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| 1728 | 5 | 0 | 0 | 0 | 45 | 0 | 0 | 0 | 0 | 0 | 0 | |
| BAS-370 | 0 | 0 | 0 | 0 | 0 | 50 | 0 | 0 | 0 | 0 | 0 | |
| CSR-30 | 0 | 0 | 0 | 0 | 0 | 0 | 50 | 0 | 0 | 0 | 0 | |
| DHBT-3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 50 | 0 | 0 | 0 | |
| PB-1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 50 | 0 | 0 | |
| PB-6 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 50 | 0 | |
| Unknown | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 142 | |
| Evaluation metrics | Precision | 91 | 100 | 100 | 98 | 100 | 100 | 100 | 100 | 88 | 94 | 100 |
| Recall | 98 | 100 | 80 | 100 | 90 | 100 | 100 | 100 | 100 | 100 | 100 | |
| Accuracy | 99 | 100 | 98.5 | 99.9 | 99.2 | 100 | 100 | 100 | 99 | 99.5 | 100 | |
A confusion matrix showing precision and recall of an AI model (deployed on iRSVPred 2.0) with original images (n=642) in an external dataset.
| Predicted variety | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1121 | 1509 | 1637 | 1718 | 1728 | BAS-370 | CSR-30 | DHBT-3 | PB-1 | PB-6 | Unknown | ||
| Actual variety | 1121 | 1710 | 16 | 0 | 59 | 15 | 0 | 0 | 0 | 0 | 0 | 0 |
| 1509 | 13 | 1757 | 0 | 27 | 3 | 0 | 0 | 0 | 0 | 0 | 0 | |
| 1637 | 0 | 0 | 1590 | 0 | 20 | 0 | 3 | 0 | 135 | 52 | 0 | |
| 1718 | 84 | 60 | 0 | 1648 | 8 | 0 | 0 | 0 | 0 | 0 | 0 | |
| 1728 | 32 | 30 | 0 | 17 | 1704 | 0 | 6 | 0 | 11 | 0 | 0 | |
| BAS-370 | 0 | 0 | 0 | 0 | 0 | 1800 | 0 | 0 | 0 | 0 | 0 | |
| CSR-30 | 7 | 4 | 0 | 0 | 0 | 0 | 1747 | 3 | 30 | 9 | 0 | |
| DHBT-3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 1800 | 0 | 0 | 0 | |
| PB-1 | 0 | 0 | 0 | 0 | 2 | 0 | 14 | 0 | 1784 | 0 | 0 | |
| PB-6 | 0 | 0 | 4 | 0 | 0 | 0 | 9 | 1 | 13 | 1773 | 0 | |
| Unknown | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 0 | 5110 | |
| Evaluation metrics | Precision | 93 | 94 | 100 | 94 | 97 | 100 | 98 | 100 | 90 | 97 | 100 |
| Recall | 95 | 98 | 88 | 92 | 95 | 100 | 97 | 100 | 99 | 98 | 100 | |
| Accuracy | 99 | 99.4 | 98.7 | 99.9 | 99.4 | 100 | 99.6 | 99.98 | 99.1 | 99.6 | 99.99 | |
A confusion matrix showing precision and recall of an AI model (deployed on RSVPred 2.0) with augmented images (n=23,112) in an external dataset.
| S. No. | Model name | Images used in training | Images used in internal validation | Training accuracy | Validation accuracy | Validation loss |
|---|---|---|---|---|---|---|
| 1 | MAM-I | 61632 | 15408 | 98 | 93.4 | 0.24 |
| 2 | MAM-II | 61632 | 15408 | 100 | 97.7 | 0.16 |
| 3 | Inception_ResNet_V2 | 73944 | 18504 | 99.9 | 96.5 | 0.135 |
Overall performance of iRSVPred 2.0 prediction model “InceptionResNetV2” with respect to previous iRSVPred models MAM-I and MAM-II.
| S. No. | Class | Number of Images (iRSVPred) | Number of Images (iRSVPred 2.0) |
|---|---|---|---|
| 1 | 1121 | 1500 | 1800 |
| 2 | 1509 | 1500 | 1800 |
| 3 | 1637 | 1500 | 1800 |
| 4 | 1718 | 1500 | 1800 |
| 5 | 1728 | 1500 | 1800 |
| 6 | BAS-370 | 1500 | 1800 |
| 7 | CSR-30 | 1500 | 1800 |
| 8 | DHBT-3 | 1500 | 1800 |
| 9 | PB-1 | 1500 | 1800 |
| 10 | PB-6 | 1500 | 1800 |
| 11 | Unknown | 4260 | 5112 |
The class-wise distribution of original and augmented images in the external validation datasets from iRSVPred and iRSVPred 2.0.
| S. No. | Class | No. of images used for validation | Multiple augmentation model (251 epochs) accuracy (%) | Multiple augmentation model (502 epochs) accuracy (%) | Average of MAM-I and MAM-II | Inception_ResNet_V2 model accuracy (%) |
|---|---|---|---|---|---|---|
| 1 | 1121 | 50 | 74 | 72 | 73 | 99 |
| 2 | 1509 | 50 | 92 | 92 | 92 | 100 |
| 3 | 1637 | 50 | 76 | 84 | 80 | 98.5 |
| 4 | 1718 | 50 | 26 | 62 | 44 | 99.9 |
| 5 | 1728 | 50 | 86 | 74 | 80 | 99.2 |
| 6 | BAS-370 | 50 | 98 | 100 | 99 | 100 |
| 7 | CSR-30 | 50 | 86 | 64 | 75 | 100 |
| 8 | DHBT-3 | 50 | 72 | 98 | 85 | 100 |
| 9 | PB-1 | 50 | 86 | 90 | 88 | 99 |
| 10 | PB-6 | 50 | 80 | 64 | 72 | 99.5 |
| 11 | Unknown | 50 | 100 | 100 | 100 | 100 |
The comparative performance of iRSVPred models with external validation dataset of original 642 images in terms of class-wise accuracy.
| S. No. | Class | Multiple augmentation model (251 epochs) accuracy (%) | Multiple augmentation model (502 epochs) accuracy (%) | Average of MAM-I and MAM-II | Inception_ResNet_V2 model accuracy (%) |
|---|---|---|---|---|---|
| 1 | 1121 | 61.4 | 68.53 | 64.96 | 99 |
| 2 | 1509 | 85.33 | 88.2 | 86.8 | 99.4 |
| 3 | 1637 | 59.07 | 77.47 | 68.27 | 98.7 |
| 4 | 1718 | 38.07 | 64.47 | 51.27 | 98.9 |
| 5 | 1728 | 64.73 | 55.6 | 60.2 | 99.4 |
| 6 | BAS-370 | 89.87 | 96.13 | 93 | 100 |
| 7 | CSR-30 | 87.53 | 65.67 | 76.6 | 99.6 |
| 8 | DHBT-3 | 83.73 | 95.4 | 89.6 | 99.98 |
| 9 | PB-1 | 78.27 | 83.47 | 80.87 | 99.1 |
| 10 | PB-6 | 69.87 | 59.53 | 64.7 | 99.6 |
| 11 | Unknown | 99.51 | 99.27 | 99.4 | 99.9 |
The comparative performance of iRSVPred models with external validation dataset of original and augmented images.
The graphical representation of the average accuracy of iRSVPred MAMs with respect to prediction accuracy of InceptionResNetV2 model (iRSVPred 2.0) for external validation dataset.
The graphical representation of the average accuracy of iRSVPred MAMs with respect to prediction accuracy of InceptionResNetV2 model (iRSVPred 2.0) for external validation datasets of original and augmented images.