Is there any way to patchify the array with overlap with numpy methods?

Question

For example, there's an array like below.

li = np.array([[1,2,3,4,5],
               [4,5,6,7,8],
               [1,2,3,4,5],
               [4,5,6,7,8],
               [1,2,3,4,5]])

Now I understand that I can patchify it using

np.lib.stride_tricks.as_strided(li, (3,3,3,3), (20,4,20,4))

Then, how can I make the sliding window move more than a single stride? Say, 2 strides each along x, y axis like below.

[[[[1,2,3],
   [4,5,6],
   [1,2,3]]
  [[3,4,5],
   [6,7,8],
   [3,4,5]]],
 [[[1,2,3],
   [4,5,6],
   [1,2,3]]
  [[3,4,5],
   [6,7,8],
   [3,4,5]]]]

It can be easily implemented if I use for loop, but I want to make it more faster.

Answer 1

there is a very nice library called patchify. you can install it via

pip install patchify

Here is how you can use patchify to achieve what you want to do:

• specify the size of the patch, and specify the step size (by how much amount should we move - if the patch size is 256 and we want an overlap of 56, the step size would be 200.)

• you can also merge back patches with overlap

example from the patchify package documentation:

# 2D image patchify and merge
import numpy as np
from patchify import patchify, unpatchify

image = np.array([[1,2,3,4], [5,6,7,8], [9,10,11,12]])

patches = patchify(image, (2,2), step=1) # split image into 2*3 small 2*2 patches.
# patches
array([[[[ 1,  2],
         [ 5,  6]],
        [[ 2,  3],
         [ 6,  7]],
        [[ 3,  4],
         [ 7,  8]]],
       [[[ 5,  6],
         [ 9, 10]],
        [[ 6,  7],
         [10, 11]],
        [[ 7,  8],
         [11, 12]]]])

assert patches.shape == (2, 3, 2, 2)

reconstructed_image = unpatchify(patches, image.shape)

# reconstructed_image
array([[ 1,  2,  3,  4],
       [ 5,  6,  7,  8],
       [ 9, 10, 11, 12]])
assert (reconstructed_image == image).all()

it also works for 3d numpy arrays

# 3D image patchify and merge
import numpy as np
from patchify import patchify, unpatchify

image = np.random.rand(512,512,3)

patches = patchify(image, (2,2,3), step=1) # patch shape [2,2,3]
print(patches.shape) # (511, 511, 1, 2, 2, 3). Total patches created: 511x511x1

assert patches.shape == (511, 511, 1, 2, 2, 3)
reconstructed_image = unpatchify(patches, image.shape)
print(reconstructed_image.shape) # (512, 512, 3)

assert (reconstructed_image == image).all()