HDF5 does not play well with multiprocessing.
Opening the same file in separate process is not reliable.
I coded a small ItemList subclass that loads two tables (one for items and one for labels), and, pretty often, the items are NOT aligned with the labels.
Check line 20 on this file: https://github.com/h5py/h5py/blob/29605d258a93b1ff603047ad4ccb68d9314319a9/examples/multiprocessing_example.py#L20
For more experiments, check this thread: Get meaningful error messages out of data_bunch loaders
[spoiler: it works-ish with numpy memory mapped files]
LE: It works well with memory mapped files! Out of core data block ItemList backed up by memmap files