Creating a Linear Gradient Mask using opencv or

Question

I am trying to stitch images together right now to create panoramas. The approach that I have tried so far was to warp the first image and align the second image with it and repeat for n number of images. That seems to be working fine but when I try concatenating the two images together by creating a binary black and white mask using numpy slicing, there is a definite seam that differentiates the two images. I am thinking that if I could either have a feathered mask in the region where black meets white with a transition area or even just a linear gradient mask going from left side of the image to the right cross-fading from black to white, it would help make the seams blend in a little better. I tried using Gaussian Blur to blur the boundaries of my binary mask experimenting with different kernel sizes but it kinda made the situation worse since the Border of the mask started showing up in the images. I just can't seem to figure out a way using numpy and openCV to create such a mask and blend the images. I would even be happy if I can create a mask as shown below so I can use that to blend in the images to improve the results. Any suggestions would be appreciated

Answer 1

I can think of two ways to approach this. The main issue is that Python/OpenCV/Numpy requires the images to be the same shape when blending.

The first approach would be to pad out the two images with zeros to the final stitched size. Then create linear ramps over the desired overlap and pad them on the left and right appropriately with ones and/or zeros to the same final stitched size. Then blend. But this is a lot of work to create all the padding.

So the simpler approach would be just to crop the images into two parts each: the overlap area and the left of the left image and overlap area and the right of the right image. Then blend the overlap areas. Then concatenate the cropped images appropriately. This is what I do below. I use the largest mountain peak as the right side of the overlap and the region to its left in the right image to define the left side of the overlap. I also first increase the brightness of the right image just to be able to see that the blending works.

import cv2
import numpy as np


# read left and right images
# images from https://medium.com/pylessons/image-stitching-with-opencv-and-python-1ebd9e0a6d78
left = cv2.imread('left.jpg')
right = cv2.imread('right.jpg')

# increase brightness of right image so that the blend difference can be seen after stitching
rightx = 1.5*right
rightx = np.clip((rightx), 0, 255)
rightx = np.uint8(rightx)

# get dimensions
hl, wl, cl = left.shape
hr, wr, cr = right.shape

print("left",hl,wl)
print("right",hr,wr)
#left 710 818
#right 709 816

# note that the two images have different dimensions
# compute min height
hm = min(hl, hr)

# measure mtn peak x location to compute overlap region x end point 
xpl = 603
xpr = 141

# note that everything from the mt peak to the left side of the right image overlaps in the left image
# So use xpr as the ramp width
ramp_width = xpr

# compute start x position of ramp in each image
xrampl = xpl-ramp_width
xrampr = 0

# crop left image into 2 parts horizontally
# start of image to ramp start and ramp start to ramp end 
left1 = left[0:hm, 0:xpl-ramp_width]
left2 = left[0:hm, xpl-ramp_width:xpl]

# crop right image into 2 parts horizontally
# ramp start to ramp end and ramp end to end of image
rightx1 = rightx[0:hm, 0:ramp_width]
rightx2 = rightx[0:hm, ramp_width:wr-ramp_width+1]

# create horizontal ramp down from 1 to 0 over the ramp width for the left image
# convert from one channel to three channels
rampl = np.linspace(1, 0, ramp_width)
rampl = np.tile(np.transpose(rampl), (hm,1))
rampl = cv2.merge([rampl,rampl,rampl])

# create horizontal ramp up from 0 to 1 over the ramp width for the right image
# convert from one channel to three channels
rampr = np.linspace(0, 1, ramp_width)
rampr = np.tile(np.transpose(rampr), (hm,1))
rampr = cv2.merge([rampr,rampr,rampr])

# blend the overlap regions, clip and make into int
blend = left2 * rampl + rightx1 * rampr
blend = np.clip((blend), 0, 255)
blend = np.uint8(blend)

# concatenate the images for the stitched result
stitched = np.concatenate((left1,blend,rightx2), axis=1)

cv2.imshow("left", left)
cv2.imshow("right", right)
cv2.imshow("rightx", rightx)
cv2.imshow("rampl", rampl)
cv2.imshow("rampr", rampr)
cv2.imshow("blend", blend)
cv2.imshow("stitched", stitched)
cv2.waitKey(0)
cv2.destroyAllWindows()

# write result to disk
cv2.imwrite("rightx.jpg", right)
cv2.imwrite("rampl.jpg", np.uint8(255*rampl))
cv2.imwrite("rampr.jpg", np.uint8(255*rampr))
cv2.imwrite("blend.jpg", blend)
cv2.imwrite("left_right_stitch.jpg", stitched)

Original Left Image:

Original Right Image:

Brightened Right Image:

Ramp Down For Left Image:

Ramp Up for Right Image:

Blended Image For Overlap Region:

Stitched Result:

Answer 2

So, I have/had quite the same ideas as fmw42 mentions in the comments, but instead of alpha blending I was thinking of plain linear blending using appropriate "blend masks" (which are the inverted masks you would use for alpha blending).

For the sake of simplicity, I assume two images with identical image sizes here. As fmw42 mentioned, you should use the "interesting" image parts here, for example obtained by cropping. Let's have a look at the code:

import cv2
import numpy as np

# Some input images
img1 = cv2.resize(cv2.imread('path/to/your/image1.png'), (400, 300))
img2 = cv2.resize(cv2.imread('path/to/your/image2.png'), (400, 300))

# Generate blend masks, here: linear, horizontal fading from 1 to 0 and from 0 to 1
mask1 = np.repeat(np.tile(np.linspace(1, 0, img1.shape[1]), (img1.shape[0], 1))[:, :, np.newaxis], 3, axis=2)
mask2 = np.repeat(np.tile(np.linspace(0, 1, img2.shape[1]), (img2.shape[0], 1))[:, :, np.newaxis], 3, axis=2)

# Generate output by linear blending
final = np.uint8(img1 * mask1 + img2 * mask2)

# Outputs
cv2.imshow('img1', img1)
cv2.imshow('img2', img2)
cv2.imshow('mask1', mask1)
cv2.imshow('mask2', mask2)
cv2.imshow('final', final)
cv2.waitKey(0)
cv2.destroyAllWindows()

These are the inputs and masks:

This would be the output:

The linear "blend masks" are created by NumPy's linspace method, and some repeating of the vector by NumPy's tile and repeat methods. Maybe, that part can be further optimized.

Caveat: At least for the presented linear blending, ensure for every pixel you generate by

mask1[y, x] * img1[y, x] + mask2[y, x] * img2[y, x] 

that

mask1[y, x] + mask2[y, x] <= 1

or you might get some "over-exposure" for these pixels.

Hope that helps!