Train/valid split a 160GB bcolz file

My data is in a bcolz file which is about 15GB compressed and 160GB uncompressed.

I attempted to adapt the fastai framework by adding two datasets that share the same bcolz carray but with two different indexes - one for training and one for validation. This worked all the way through precomputing a resnet but then failed at save_fc1 with an assertion error.

Is there a way of using a bcolz file as input without splitting it into training and validation first. If not then what is fastest way to split a bcolz file of this size - I am thinking I have to read it all into memory, create the index, and then read/write one row at a time.