pyqt5 draggable rectangles on picture

Question

I am very new to pyqt5 and trying to figure out how to replicate the following idea.

1. Load an image.

2. Draw MOVEABLE rectangle(s) on top.

   -- by Moveable, I mean rectangles that I can move via click and drag after initial placement. Like if I dropped it on the top right of picture, I can later move it to middle of the picture.

   -- rectangles(s) multiple is very important. hopefully I can add/remove them dynamically too.

3. ideally show a text associated with the rectangle, like a, b, c... that is editable. (nice to have)

The following picture from LabelImg show cases the idea very well.

At this point in time, I have been able to load a picture and display it via QLabels and converting to PixMap. The self.ui, is the typical syntax for loading a UI from QT Designer.

class ImageLoader(QtWidgets.QWidget):
    def __init__(self):
        QtWidgets.QWidget.__init__(self)
        
        self.ui = Ui_Form() # from QT Designer
        self.ui.setupUi(self)
        
        self.ui.frame_image_label.setMouseTracking(True)
        self.ui.frame_image_label.setAcceptDrops(True)
       
        #.... other stuff not quite relevant #####
        
        # a special draggable Label (see below)
        # The following does NOT work.  (And I want to be able to make multiple labels on key-press...)
        self.special_label = DragLabel('hii', self.ui.frame_image_label)

    def go_to_image(self):
        '''Load the image from a video into the base QLabel to show'''
        self.current_frame_number, img = self.video_folder.get_frame(self.current_frame_number)
        if img is None:
            return
        
        img = self.video_folder.resize_image(image = img)
        
        pixmap = convertCvImage2QtImage(img)
        if pixmap.isNull():
            return
        self.ui.frame_image_label.setPixmap(pixmap)

I have also created a custom DragLabel

class DragLabel(QLabel):
    def __init__(self, button_text, parent):
        super().__init__(button_text, parent)

    def mouseMoveEvent(self, event):
        if event.buttons() == Qt.LeftButton:
            mimeData = QMimeData()
            drag = QDrag(self)
            drag.setMimeData(mimeData)
            
            pixmap = QPixmap(self.size())
            self.render(pixmap)
            drag.setPixmap(pixmap)

            drag.exec_(Qt.MoveAction)

The current issues:

1. The special_label does show up in my UI. But it doesn't move when I drag it.
2. It is not constrained to my self.ui.frame_image_label area

In general, the reason Im not using the LabelImg, which is very nice, is because I'm trying to label a VIDEO, and am mainly use this as "correction", ie finding where the Area of Interest was incorrectly identified and then hand-correcting them. Hence the need to able to move my selections. Also, I am trying to embed "play/pause", slider selection of frames for ease of use.

Any help is appreciated.

Edit: Musicamante has the correct idea in that we should swap to the Graphics View Framework and use views and scenes. QLabels with pictures is not the correct idea.