how to read from h5py in multiprocessing without errors

Question

I have code like:

def get_df(path, key):
    with h5py.File(path) as hdf:
        df = pd.DataFrame(np.array(hdf[key]))
    return df

def f(key):
    df = get_df(path, key)
    ...transform df...
    return df

with multiprocessing.Pool(n_cpus) as pool:
    rvals = pool.map(f, keys)

And I'm getting this error many times:

terminate called after throwing an instance of 'std::system_error'
  what():  Resource temporarily unavailable

and one copy of this error:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/home/lenail/.conda/envs/py38/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/home/lenail/.conda/envs/py38/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/lenail/.conda/envs/py38/lib/python3.8/multiprocessing/pool.py", line 513, in _handle_workers
    cls._maintain_pool(ctx, Process, processes, pool, inqueue,
  File "/home/lenail/.conda/envs/py38/lib/python3.8/multiprocessing/pool.py", line 337, in _maintain_pool
    Pool._repopulate_pool_static(ctx, Process, processes, pool,
  File "/home/lenail/.conda/envs/py38/lib/python3.8/multiprocessing/pool.py", line 326, in _repopulate_pool_static
    w.start()
  File "/home/lenail/.conda/envs/py38/lib/python3.8/multiprocessing/process.py", line 121, in start
    self._popen = self._Popen(self)
  File "/home/lenail/.conda/envs/py38/lib/python3.8/multiprocessing/context.py", line 277, in _Popen
    return Popen(process_obj)
  File "/home/lenail/.conda/envs/py38/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/home/lenail/.conda/envs/py38/lib/python3.8/multiprocessing/popen_fork.py", line 70, in _launch
    self.pid = os.fork()
BlockingIOError: [Errno 11] Resource temporarily unavailable

any ideas why this might be or how to fix it? I had read that h5py supports multiprocessing.

Python 3.8.13 | packaged by conda-forge
>>> h5py.__version__
'3.6.0'