Platform: Kaggle Kernels

You’re welcome :slight_smile:

I have copied and translated your FAQ and updated and tweak your kernels a little for the Chinese version of fastai course notes. Your works are very helpful, and looking forward to reading more of your posts.

1 Like

Thanks!
Do you mind linking the translated post in the top?
You can edit this wiki and link to the translation for our Chinese speaking friends.

1 Like

@init_27 Thank you for providing that FAQ. While I think that is super useful, it still doesn’t address permanently saving and/or exporting files created during the session. i.e. weights learned (.pth files) and like in Lesson 2 labels.csv created when you clean-up your data. If this functionally exists, where is it? How is it used? If not, I think being able to permanently save and export are great capabilities that the community will benefit from having – thoughts?

1 Like

If you need the files created an “interactive session”, you can look up how to create a downloadable link for jupyter notebook. Then you can manually download it.

Otherwise, you’ll need to “commit” your notebook to have these produced as outputs.

With Lesson 2 you cannot “commit” the nb as it creates/downloades images > 500. Kaggle kernels have a known limit of 500 output files so they crash when you try to run them.

There are 2 fixes:

Download <500 files in total.
OR
Download the .pth file by creating a downloadable link

Hope this clarifies your question?

I’ll checkout the downloadable link method – thanks!

Also, one other work-around on the commit issue is to comment out all the code in the cells – yes, it’s annoying. So, can there be a commit without code/cell execution capability added?

The issue with that is, it will cry with errors since output files will be > 500.

I’ll tell you the shortcut: Use (Edit:) Ctrl + / to uncomment, that might make it a little less irritating but apologies, I’m not sure of any other workaround.

I tried using < 500 images. That gave pretty bad accuracy values.
If you want to give it a shot, I’ll be happy to add you as a contributor (with credit) and you can see if it works better with <500 images.

@init_27 Thinking about this more, you are right. In spite of having nearly 600 image files spread almost evenly across three directories, my final confusion matrix only has a total of 81 predictions. In hindsight, I am surprised the number of predictions is that low. Perhaps ImageDataBunch() had trouble reading most of those files? I think they were mostly .jpg files.

That said, I ran a few dozen epochs and finally landed on an error rate of .148148 – not great and at times I could see the model improving even though I also saw that the surface of the problem was lumpy. In the end, I am concerned the model was over-fitting and that it wouldn’t generalize well in the real world. Sounds like I need to add more data.

1 Like

BTW – It’s CTRL+/ to toggle comment / uncomment – but thanks for the tip!!

1 Like

It might be possible that a large number of corrupt images were downloaded in that case.

Thanks for the correction!

1 Like

Thanks! it is a great idea, I will do that!

1 Like

Hi all,

I had a problem running ‘Lesson 2 Download’ notebook when calling the function:

data2 = ImageDataBunch.single_from_classes(path, classes, tfms=get_transforms(), size=224).normalize(imagenet_stats).

TypeError: transform() got multiple values for argument 'tfms'.

For anyone with the same problem, I fixed this by changing the argument of the function from tfms=get_transforms() to ds_tfms=get_transforms().

Best wishes and thanks for creating / maintaining the Fast.ai Kaggle Kernels!

Ross

1 Like

@init_27 : Just recently I got a strange error using the image item list

data = (ImageItemList.from_folder(path)
.random_split_by_pct(0.2)
.label_from_folder()
.transform(tfms, size=128)
.databunch())

6 days ago this worked fine in my code, now I get the following error message:

 NameError                                 Traceback (most recent call last)
    <ipython-input-5-b92c0520489a> in <module>()
    ----> 1 data = (ImageItemList.from_folder(path)
          2         .random_split_by_pct(0.2)
          3         .label_from_folder()
          4         .transform(tfms, size=128)
          5         .databunch())

    NameError: name 'ImageItemList' is not defined. 

I also noticed that ‘loading-in-library part’ is now shows up in blue, not green anymore - is there a problem in loading the fast ai library?

%reload_ext autoreload
%autoreload 2
%matplotlib inline

Thankful for any help <3

I believe the latest fastai now uses ImageList

3 Likes

Wow- thank you very much. I thought I tried that but apparently I didn’t… Thanks a lot <3

Thank you for maintaining those lessons! Started using Kaggle Kernels and Google Colab to do my own work using fastai and realized why this effort of having working lessons notebooks is so useful. In both environments you get some version of fastai (is it always the latest version in the repo or are those environments pointing to some internal released version of the code?) so you never know when your code will break after incompatible fastai changes

Does anyone know how to prevent the error using the learning rate? Worked well in previous versions, seems to be a problem in fastai 1.0.46 (version installed in my kernels- I guess its the recent one?)

learn.lr_find()

It throws out the following error:

OSError: [Errno 30] Read-only file system: ‘…/input/purge-tmp.pkl’

1 Like

@piaoya It looks like your learner is pointing to the input directory.
Please make sure you check the FAQ since this has been asked earlier as well.

You need to ensure that the learner points to a R/W folder, so just copy your dataset to the R/W folder in kernel. That should solve it. :slight_smile:

@shlyakh Sadly, for kernels those are always a few versions behind.
I update them each Sunday to the latest code, but you can expect them to be a few versions behind.

The workaround would be to install manually the latest version in a session but you’d have to do that everytime.

I think it has to do with the function’s expectation of the parameter names.
Thanks for sharing the fix! :slight_smile:

have u solved this problem? I met same with you :open_mouth:

@kkjusdoit: On the fastai.widget fileDeleter: I’m sorry, I don’t know how to fix this - doesn’t work for me either.

For solving the lr_find() I searched the kernel tutorials for code to copy the folder, as init27 suggested. (Thank you @init_27 for the hint - I haven’t realized that the input folder is the problem…) Just in case anyone has the same problems: I finally ended up with this code:

folder= 'working_dir'
path = Path('../input')
origin= Path('..')
dest = origin/folder

dest.mkdir(parents=True, exist_ok=True)
!cp -r ../input/* {dest}/
path.ls()
dest.ls()

So we are building up a working directory - in that case next to the input folder, copy the input to the destination folder and use this copied version for the imageList:

ImageList.from_folder(dest)

1 Like