Wrong COCO format while exporting a `sv.DetectionDataset` using its `as_coco` method

[rizavelioglu]

The Issue:

I export some annotations using:

sv.DetectionDataset(
   classes=CLASSES,
   images=images,
   annotations=annotations
).as_coco(
   annotations_path=ANNOTATIONS_PATH,
   min_image_area_percentage=0,
   max_image_area_percentage=1.,
   approximation_percentage=0
)

The resulting .json is created successfully.
Then, loading the annotations using pycocotools and plotting annotations of an image:

from pycocotools.coco import COCO
coco=COCO(ANNOTATIONS_PATH)

imgId = 42  # an arbitrary image id
file_name = coco.loadImgs(imgId)[0]["file_name"]

img = plt.imread(file_name)
plt.imshow(img)
plt.axis("off")

annIds = coco.getAnnIds(imgIds=imgId, iscrowd=None)
anns = coco.loadAnns(annIds)
coco.showAnns(anns, draw_bbox=True)

which plots wrong annotations for the given imgId: the annotations (box & mask) is seen bottom-right and it seems that there is a mismatch between the image and its annotations.

Actually, for any imgId, we have the same issue: the image is shown correctly but the annotations are not. Interestingly, the annotations (annIds) are always the same for different imgId's. In my case, there is only 1 box & mask per image and for various values of imgId, the annIds are always equal to 1.

The Cause:

The annIds are not unique among all the annotations (the "id" in "annotations" in COCO format).
I presume they have to be unique among all annotation id's (see the following issue).

However, the current implementation resets the annotation id to 1 for each image:

supervision/supervision/dataset/formats/coco.py

Line 202 in 4f79d29

image_id, annotation_id = 1, 1

The (possible) Fix:

Simply changing label_id to annotation_id in the following line will assign a unique id for each annotation:

supervision/supervision/dataset/formats/coco.py

Line 218 in 4f79d29

coco_annotation, label_id = detections_to_coco_annotations(

I would be glad to open a PR.

[github-actions]

Hello there, thank you for opening an Issue ! 🙏🏻 The team was notified and they will get back to you asap.

[SkalskiP]

Hi @rizavelioglu 👋🏻

I don't think the problem originates in image_id, annotation_id = 1, 1.

Take a look at rest of the function:

image_id, annotation_id = 1, 1
    for image_name, image in images.items():
        image_height, image_width, _ = image.shape

        coco_image = {
            "id": image_id,
            "license": 1,
            "file_name": image_name,
            "height": image_height,
            "width": image_width,
            "date_captured": datetime.now().strftime("%m/%d/%Y,%H:%M:%S"),
        }

        coco_images.append(coco_image)
        detections = annotations[image_name]

        coco_annotation, label_id = detections_to_coco_annotations(
            detections=detections,
            image_id=image_id,
            annotation_id=annotation_id,
            min_image_area_percentage=min_image_area_percentage,
            max_image_area_percentage=max_image_area_percentage,
            approximation_percentage=approximation_percentage,
        )

        coco_annotations.extend(coco_annotation)
        image_id += 1

    annotation_dict = {
        "info": info,
        "licenses": licenses,
        "categories": coco_categories,
        "images": coco_images,
        "annotations": coco_annotations,
    }
    save_json_file(annotation_dict, file_path=annotation_path)

Both image_id and annotation_id are updated later in the function.

Is your sv.DetectionDataset a result of model inference?

[rizavelioglu]

Hi @SkalskiP, thanks for the reply 🤗

image_id is updated without any problem.
annotation_id is updated inside detections_to_coco_annotations:

supervision/supervision/dataset/formats/coco.py

Line 131 in 6f427b9

annotation_id += 1

however, it starts from 1 for every new image, which yields non-unique annotation_id's throughout the whole annotations, e.g.:

"annotations": [
   {
    "id": 1, 
    "image_id": 1, 
    ...,
  },
   {
    "id": 2, 
    "image_id": 1, 
    ...,
  }
   {
    "id": 1,   # --> `annotation_id` starts from 1 again
    "image_id": 2, 
    ...,
  }
   {
    "id": 1,   # --> and again
    "image_id": 3, 
    ...,
  }
]

I presume that pycocotools expects the annotation_id's to be unique. Hence, if we replace:
coco_annotation, label_id = detections_to_coco_annotations(...
with coco_annotation, annotation_id = detections_to_coco_annotations(...
the following annotation file will be created:

"annotations": [
   {
    "id": 1, 
    "image_id": 1, 
    ...,
  },
   {
    "id": 2, 
    "image_id": 1, 
    ...,
  }
   {
    "id": 3,   # --> updated
    "image_id": 2, 
    ...,
  }
   {
    "id": 4,   # --> updated
    "image_id": 3, 
    ...,
  }
]

My issue was resolved after applying this change.

"Is your sv.DetectionDataset a result of model inference?"
No. I have a set of images and the raw annotations, and want to convert them to COCO format,

[SkalskiP]

@rizavelioglu could you share some minimal dataset before and after conversion to demonstrate the issue? I’d love to gain some more understanding.

[rizavelioglu]

@SkalskiP here's a link to a Colab notebook where I demonstrate the issue.
After running all the cells, there is a sample_coco.json file, which should look like this:

As seen, for every image_id, the annotation_id starts from 1, resulting in non-unique annotation_id's throughout the whole annotations.

[SkalskiP]

Hi @rizavelioglu 👋🏻! Thanks a lot for the example. 🙏🏻 It's confirmed we have a bug. We missed that as sv.Detection.Dataset.from_coco does not care about annotation_id, so everything loaded as expected.

You mentioned that you would be keen to submit a fix. IS that still the case?

I also noticed that you asked a question in the notebook:

We store the images & annotations in the memory as shown above. But, as the number of images increase, we run out of memory. Is there any other efficient way you would recommend?

Keeping the annotations in RAM is not a problem, but keeping images can be a massive problem with larger datasets. We plan to introduce lazy loading.

[rizavelioglu]

Hi @SkalskiP 👋🏻!
Yes, sure, I will submit a fix soon!
Also, thanks for the reply, looking forward to it.

[SkalskiP]

@rizavelioglu awesome! I can't wait to see that!