How to save information about the result of instance segmentation by YOLACT?

Question

Is there any way to save the detected categories, their number, MASK area, etc. to a TXT file or CSV file when performing instance segmentation using YOLACT?

I’m using YOLACT (https://github.com/dbolya/yolact) to challenge instance segmentation. I was able to use eval.py to do an instance segmentation of my own data and save that image or video. However, what I really need is the class names and their numbers detected and classified by YOLACT's AI, and the area of ​​MASK. If we can output this information to a txt file or csv file, we can use YOLACT even more advanced.

If I can achieve that by adding an option in eval.py or modifying the code, please teach me.

Thank you.

Answer 1

You already have that information from the eval.py.

This line in the eval.py gives you the information.

# line 160
classes, scores, boxes = [x[:args.top_k].cpu().numpy() for x in t[:3]]
masks = t[3][:args.top_k]

areas = []

for mask in masks:
    # Use OpenCV's findContours() to detect and obtain the contour of the variable mask
    # Use OpenCV's contourArea() to calculate the area
    areas.append(area)

Getting the no. of detections:

# line 162
num_dets_to_consider = min(args.top_k, classes.shape[0])
    for j in range(num_dets_to_consider):
        if scores[j] < args.score_threshold:
            num_dets_to_consider = j
            break

To save the classes and scores as a csv file:

import pandas as pd

df = pd.DataFrame(columns=['class', 'score'])
c = 0
for i in reversed(range(num_dets_to_consider)):
    classname = cfg.dataset.class_names[classes[j]]
    score = scores[i]
    df.loc[c] = [classname, score]
    c += 1
df.to_csv('info.csv', index=None)

EDIT1: You can return the values (classes, scores, areas) from the prep_display() and then retrieve them in evalimage(). Note that evalimage() calls prep_display(). You can do something like this:

classes, scores, areas = prep_display()
# Here you can include the pandas code. Note that it stores the information only for one image. You can use a loop and then store the information of all the images.

EDIT2:

# This is the default line in eval.py at line ~600
# This is found at evalimage()
img_numpy = prep_display(preds, frame, None, None, undo_transform=False)

# Change the above line to this:
classes, scores, areas, img_numpy = prep_display(preds, frame, None, None, undo_transform=False)

# Also, remember to return these 4 values from the prep_display(). 
# Otherwise, it will lead to an error. 
# img_numpy is the default one that will be returned from prep_display(). 
# You're now simply returning 4 values instead of 1.

# NOTE: I have included img_numpy because I found that prep_display() is being used in many places. 
# You need to modify the returns wherever prep_display() is being called in the eval.py. I think it's being called 3 times from different regions in eval.py.