Bcolz error: "fatal error during Blosc decompression"

Part 1 (2017)
1

I’m going over the dogs/cats linear model in lesson 2, and I’m seeing this error when using lm.fit:

Exception RuntimeError: RuntimeError('fatal error during Blosc decompression: -1',) in 'bcolz.carray_ext.chunk._getitem' ignored

It only happens when I’ve saved and then loaded the training features using bcolz (not the validation features - I’ve checked). It happens no matter how many epochs. The exception repeats 55 times (which is a strange number - batch size is 64, training sample is 200, validation sample is 40). I’ve tried restarting the kernel - no change.

At the end of the day it’s not that big a deal - there’s no error if I skip using bcolz (and just hold everything in memory). But I’m not doing anything weird - it’s a p2 instance with the standard AMI you gave us for this class.

Here’s what it looks like:

blob.jpg1078×299 113 KB

2

If you use my load_array() it loads the whole bcolz array into memory, so you shouldn’t have this problem.

3

Ha! You’re right. I didn’t realize that the [:] at the end of your load_array forced the full array into memory.

It’s odd that it only seemed to be an issue for some of the arrays, but I want to focus my attention on the course content again!