Find Corners of pool table for perspective warp with OpenCV

Question

I have an image of a pool table with slight perspective distortion:

I am trying to un-warp the image (such that the corners of the pool table correspond to the corners of my output image) and then detect the pool balls on the output image. To unwarp the table, I intend to use cv2.PerspectiveTransform() to obtain the transformation matrix that maps the corners of the pool table to the corners of my output image. However, I do not have the locations of the corners of my table; I want to find the corners programmatically.

So far, I have applied a mask (that filters out everything but the pool table). After applying the mask, I've applied a Canny edge detector to find the edges of the pool table only:

My idea was to use the Hough line transform algorithm to find the lines corresponding to the edges of the pool table (to find the corners of the pool table programmatically), but the Hough line transform algorithm seems to have a hard time detecting the edges well. How can I find the corners of the pool table from here?

EDIT

Is there any way I can just extract the sharp corners of the outline (the eight points corresponding to where the straight edges turn into the pockets)?

Answer 1

The corners can only be obtained by extrapolating the straight sides.

You can start by wrapping the outline in a minimum-area rectangle to get a rough approximation. From this approximation, you can extract sections of the outline that are fairly straight (using a predefined mask), and perform line fitting on them.