(Live) viewing of large images using pyvips

Question

I recently discovered the awesome pyvips package and would like to use it to analyze data that was taken on a homebuilt slide scanner (not built by me). I scan about 4000 tiles of 1024x1024 pixels each along the edges of a square-shaped sample (the center part of the sample is not recorded). All tiles are saved as a single binary file. I have written a python class that returns a desired tile as a numpy array from the binary file and which also gives the (x, y) coordinates of the specific tile. Unfortunately, the tiles are not arranged on a grid.

I first determine the total width and height of the full image and initialize a black image of the correct size and subsequently place the tiles at the correct locations using the insert function. The composite image is about 120k x 120k pixels, but most of the image is empty. Finally, I plot the resulting image using matplotlib.

import pyvips
import numpy as np
import matplotlib.pyplot as plt

# class to read tiles from data file
sr = TileReader("path_to_scan_file")

# some stuff to determine the width and height of the total image...

# create empty image for inserting tiles
im = pyvips.Image.black(width, height)

# loop over all tiles and place tile at correct position
for i in range(sr.num_tiles()):
    frame, coord = sr.ReadFrame(i)
    tile = pyvips.Image.new_from_array(frame)
    im = im.insert(tile, coord[0], coord[1])

# plot result
plt.imshow(im.numpy())
plt.show()
# save file
im.write_to_file('full_image.tiff')

Generating the full image in the loop seems to be very fast. However, plotting or saving the data is not. (Obviously,) the plotting only works for a small number of tiles (~10). I also tried saving the data to a pyramidal tiff. However, writing the image took several hours and the generated file seems to be corrupted or too large to be opened. Unfortunately I could not get nip2 installed without admin rights.

I would like to be able to manually select regions of interest of the composite image that I can use for further processing. What is the best/fastest way to interact with the generated image to enable this?

Answer 1

You can use crop to cut out a chunk of the image and pass that on to something else. It won't make the whole thing, it'll just render the bit you need, so it'll be quick.

Something like:

# loop over all tiles and place at correct position
# do this once on startup
for i in range(sr.num_tiles()):
    frame, coord = sr.ReadFrame(i)
    tile = pyvips.Image.new_from_array(frame)
    im = im.insert(tile, coord[0], coord[1])

# left, top, width, height
# hook these numbers up to eg. a scrollbar
# do the crop again for each scrollbar movement
tile = im.crop(0, 0, 1000, 1000)

# plot result
plt.imshow(tile.numpy())
plt.show()

If you want to get fancy, the best solution is probably vips_sink_screen():

https://www.libvips.org/API/current/libvips-generate.html#vips-sink-screen

That'll let you generate pixels from any pipeline asynchronously as you pan and zoom, but it needs C, sadly. There's an example image viewer using this API here:

https://github.com/jcupitt/vipsdisp

That's running vips_sink_screen() in the background to generate GPU textures at various scales, then using that set of textures to paint the screen at 60 fps (ish) as you pan and zoom around. It can display huge dynamically computed images very quickly.