Differences in results between yolo and my code.

[AliAldubaikhi]

Hello,

I have a problem. I used the following code to extract the number of FP and FN , but when I compare it to the validation results in Yolo, I found a big difference. Is there a solution to this problem or an explanation for it?

import json
from pathlib import Path
from PIL import Image  


predictions_path = Path('C:/Users/ENG-LT-SL-15/Desktop/predictions.json')
with open(predictions_path) as f:
    predictions = json.load(f)



def load_ground_truth_annotations(ground_truth_path):
    with open(ground_truth_path) as f:
        ground_truth_list = json.load(f)

    
    ground_truth_dict = {}
    for annotation in ground_truth_list:
        image_id = annotation['image_id']
        bbox = annotation['bbox']
        if image_id not in ground_truth_dict:
            ground_truth_dict[image_id] = []
        ground_truth_dict[image_id].append(bbox)

    return ground_truth_dict



def convert_bbox_to_yolo(image_width, image_height, bbox):
    x_min, y_min, width, height = bbox
    x_center = (x_min + width / 2) / image_width
    y_center = (y_min + height / 2) / image_height
    w = width / image_width
    h = height / image_height
    return [x_center, y_center, w, h]



ground_truth_path = Path('C:/Users/ENG-LT-SL-15/Desktop/annotations.json')  # ضع مسار ملف التعليقات الحقيقية
ground_truth = load_ground_truth_annotations(ground_truth_path)



def iou(boxA, boxB):
    # box format: [x_center, y_center, width, height] (YOLO format)
    xA_min = boxA[0] - boxA[2] / 2
    yA_min = boxA[1] - boxA[3] / 2
    xA_max = boxA[0] + boxA[2] / 2
    yA_max = boxA[1] + boxA[3] / 2

    xB_min = boxB[0] - boxB[2] / 2
    yB_min = boxB[1] - boxB[3] / 2
    xB_max = boxB[0] + boxB[2] / 2
    yB_max = boxB[1] + boxB[3] / 2

    # Calculate the intersection
    inter_x_min = max(xA_min, xB_min)
    inter_y_min = max(yA_min, yB_min)
    inter_x_max = min(xA_max, xB_max)
    inter_y_max = min(yA_max, yB_max)

    inter_area = max(0, inter_x_max - inter_x_min) * max(0, inter_y_max - inter_y_min)

    
    boxA_area = (xA_max - xA_min) * (yA_max - yA_min)
    boxB_area = (xB_max - xB_min) * (yB_max - yB_min)

    
    union_area = boxA_area + boxB_area - inter_area

    if union_area == 0:
        return 0

    
    iou_value = inter_area / union_area
    return iou_value



fp_images = set()  # Images with false positives
fn_images = set()  # Images with false negatives



def get_image_dimensions(image_folder, image_id):
    image_path = image_folder / f"{image_id}.jpg"  # assuming images are in .jpg format, adjust as needed
    with Image.open(image_path) as img:
        return img.size  # Returns (width, height)



image_folder = Path('D:/Good DvB Datasets(O and S)/Orignal DvB dataset/val/images')  # ضع مسار مجلد الصور


for pred in predictions:
    image_id = pred['image_id']
    pred_box_pixel = pred['bbox']

    
    image_width, image_height = get_image_dimensions(image_folder, image_id)

    
    pred_box_yolo = convert_bbox_to_yolo(image_width, image_height, pred_box_pixel)

    gt_boxes = ground_truth.get(image_id, [])

    
    if not any(iou(pred_box_yolo, gt_box) > 0.6 for gt_box in gt_boxes):
        fp_images.add(image_id)


for image_id, gt_boxes in ground_truth.items():
    pred_boxes_pixel = [pred['bbox'] for pred in predictions if pred['image_id'] == image_id]

    
    pred_boxes_yolo = [convert_bbox_to_yolo(*get_image_dimensions(image_folder, image_id), bbox) for bbox in
                       pred_boxes_pixel]

    
    for gt_box in gt_boxes:
        if not any(iou(gt_box, pred_box_yolo) > 0.6 for pred_box_yolo in pred_boxes_yolo):
            fn_images.add(image_id)


#print(f"Images with false positives: {fp_images}")
print(f"Number of images with false positives: {len(fp_images)}")
#print(f"Images with false negatives: {fn_images}")
print(f"Number of images with false negatives: {len(fn_images)}")

This is the output

Number of images with false positives: 15019
Number of images with false negatives: 14388

Process finished with exit code 0

also when I set the save_hybrid=True in validation I did not get the file in output folder.

Thanks,,

[UltralyticsAssistant]

👋 Hello @AliAldubaikhi, thank you for reaching out with your question! 🚀 This is an automated response, and an Ultralytics engineer will assist you soon.

We recommend browsing the Docs for insights as it might have the information you need. You'll find extensive Python and CLI examples that could shed light on your issue.

If this is related to a 🐛 Bug Report, please share a minimum reproducible example to assist with debugging.

For custom training ❓ Questions, kindly provide detailed information, including dataset samples, training logs, and ensure you’re using our Model Training Tips.

Join our Ultralytics community in a space that suits you best. Engage in real-time discussions on Discord 🎧, join thoughtful conversations on Discourse, or participate in discussions on our Subreddit.

Upgrade

To ensure your problems are not caused by outdated software, upgrade to the latest ultralytics package along with all requirements in a Python>=3.8 environment with PyTorch>=1.8:

pip install -U ultralytics

Environments

Run YOLO in any of these verified environments, featuring crucial dependencies like CUDA, CUDNN, Python, and PyTorch:

• Notebooks with GPUs:
• Google Cloud Deep Learning VM: See GCP Quickstart Guide
• Amazon Deep Learning AMI: See AWS Quickstart Guide
• Docker Image: See Docker Quickstart Guide

Status

🌟 If this badge is green, all Ultralytics CI tests are passing. These tests confirm the correct function of the YOLO Modes and Tasks across multiple platforms.

[glenn-jocher]

@AliAldubaikhi differences in results can arise from variations in IoU thresholds, image preprocessing, or dataset handling. Ensure your IoU threshold and preprocessing steps match those used in YOLO's validation. For the save_hybrid=True issue, verify the output directory path and permissions. For more insights, visit our YOLO Common Issues Guide.