Benchmarking models (YOLOv5, YOLOv8, YOLO-NAS) which performs better.

[rexxsd]

I was trying to benchmark(ing) the models (YOLO : v5, v8 & NAS) while training them on same dataset, the evaluation metrics for v5 & v8 were satisfactory as mentioned below:

YOLOv5: Model Summary:
Precision: 0.914 | Recall : 0.969 | mAP@50: 0.968 | mAP90: 0.625

YOLOv8: Model summary (fused): 168 layers, 11125971 parameters, 0 gradients, 28.4 GFLOPs
Class Images Instances Box(P R mAP50 mAP50-95): 100%|██████████| 7/7 [00:04<00:00, 1.42it/s]
all 215 230 0.966 0.948 0.98 0.781

But the outcomes for YOLO-NAS was not even close to them where I got the following outcomes:

YOLO-NAS
SUMMARY OF EPOCH 273
├── Training
│ ├── Ppyoloeloss/loss = 1.3271
│ │ ├── Best until now = 1.3328 (arrow_lower_right -0.0057)
│ │ └── Epoch N-1 = 1.3328 (arrow_lower_right -0.0057)
│ ├── Ppyoloeloss/loss_cls = 0.6218
│ │ ├── Best until now = 0.6233 (arrow_lower_right -0.0014)
│ │ └── Epoch N-1 = 0.6256 (arrow_lower_right -0.0038)
│ ├── Ppyoloeloss/loss_dfl = 0.8454
│ │ ├── Best until now = 0.8469 (arrow_lower_right -0.0016)
│ │ └── Epoch N-1 = 0.8469 (arrow_lower_right -0.0016)
│ └── Ppyoloeloss/loss_iou = 0.113
│ ├── Best until now = 0.1135 (arrow_lower_right -0.0004)
│ └── Epoch N-1 = 0.1135 (arrow_lower_right -0.0004)
└── Validation
├── F1@0.50 = 0.0982
│ ├── Best until now = 0.1203 (arrow_lower_right -0.0221)
│ └── Epoch N-1 = 0.1097 (arrow_lower_right -0.0115)
├── Map@0.50 = 0.9115
│ ├── Best until now = 0.9312 (arrow_lower_right -0.0197)
│ └── Epoch N-1 = 0.9246 (arrow_lower_right -0.013)
├── Ppyoloeloss/loss = 1.8552
│ ├── Best until now = 1.8142 (arrow_upper_right 0.0409)
│ └── Epoch N-1 = 1.8432 (arrow_upper_right 0.012)
├── Ppyoloeloss/loss_cls = 0.8396
│ ├── Best until now = 0.7983 (arrow_upper_right 0.0414)
│ └── Epoch N-1 = 0.8198 (arrow_upper_right 0.0198)
├── Ppyoloeloss/loss_dfl = 1.1295
│ ├── Best until now = 1.0111 (arrow_upper_right 0.1184)
│ └── Epoch N-1 = 1.1407 (arrow_lower_right -0.0112)
├── Ppyoloeloss/loss_iou = 0.1803
│ ├── Best until now = 0.1628 (arrow_upper_right 0.0175)
│ └── Epoch N-1 = 0.1812 (arrow_lower_right -0.0009)
├── Precision@0.50 = 0.0516
│ ├── Best until now = 0.064 (arrow_lower_right -0.0124)
│ └── Epoch N-1 = 0.058 (arrow_lower_right -0.0064)
└── Recall@0.50 = 1.0
├── Best until now = 1.0 (= 0.0)
└── Epoch N-1 = 0.9957 (arrow_upper_right 0.0043)

Whats the reason behind this, why these values are having such a huge differences in NAS as compare to v5 & v8, what are the factors affecting these difference.If any one can help regarding this it would be helpful .

[SkalskiP]

Hi @rexxsd 👋🏻! That looks very interesting. Do you have a single table that would sum up your results?