Get the polygon coordinates of predicted output mask in YOLACT/YOLACT++

Question

I am using Yolact https://github.com/dbolya/yolact ,an instance segmentation algorithm which outputs the test image with a mask on the detected object. As the input images are given with the coordinates of polygons around the input classes in the annotations.json, I want to get an output like this. But I can't figure out how to extract the coordinates of those contours/polygons.

As far as I understood from this script https://github.com/dbolya/yolact/blob/master/eval.py the output is list of tensors for detected objects. It contains classes, scores, boxes and mask for evaluated image. The eval.py script returns recognized image with all this information. Recognition is saved in 'preds' in evalimg function (line 595), and post-processing of predict result is in the "def prep_display" (line 135)

Now how do I extract those polygon coordinates and save it in .JSON file or whatever else?

I also tried to look at these but couldn't figure out sadly! https://github.com/dbolya/yolact/issues/286 and https://github.com/dbolya/yolact/issues/256

Answer 1

You need to create a complete post-processing pipeline that is specific to your task. Here's small pseudocode that could be added to the prep_disply() in eval.py

with timer.env('Copy'):
    if cfg.eval_mask_branch:

        # Add the below line to get all the predicted objects as a list
        all_objects_mask = t[3][:args.top_k]

        # Convert each object mask to binary and then
        # Use OpenCV's findContours() method to extract the contour points for each object