The kernel appears to have died. Runnig notebook on large dataset of images

(Said Aspen) #1

I am trying to run the notebook from Lesson 1 and 2 on a very big data set of images. The notebook seems to die without any error logging when using ImageClassifierData.from_csv

This is what I run.

This part works fine:

tfms = tfms_from_model(resnet50, 199, aug_tfms=transforms_side_on, max_zoom=1.1)

But this part does not:

train_csv = PATH + "train.csv"
data = ImageClassifierData.from_csv(PATH, 'train', train_csv, test_name='test', num_workers=1, 
                  val_idxs=val_idxs, suffix='.jpg', tfms=tfms, bs=16)

After a minute or two I get this:

Nothing is ever written to the console.

Does anyone know whats going on? I believe it might be the fact that it is a very very large data-set of images (~150 Gb big).

(Susant Bisoi) #2

I am getting often the same issues.


I was able to resolve this issue by re-installing PyTorch from source as described here


(Said Aspen) #4

Tried that and updated the nvidia drivers as well as Cuda. Let’s see how it works

(Ankit) #5

Even i am getting the same issue, were you able to resolve the issue of “Kernels appears to have died.” ?

I am on AWS p2.xlarge and using the below code but even with small bs of 16 and sz of 65, it goes out of memory. Any suggestions on how to handle this?


Hello @Ankit89, I have the same situation. Have you been able to solve it?