Platform: Kaggle Kernels

piaoya · February 17, 2019, 6:41pm

I want to save a df I created. If I enter df.to_csv() I get an this message OSError: [Errno 30] Read-only file system: '../input/labels.csv'. How can I save my csv-file on kaggle? Thankful for any help <3

init_27 · February 17, 2019, 6:43pm

@piaoya
You need to set the saving path to the writeable directory where your notebook resides. It should work then.

piaoya · February 17, 2019, 6:53pm

Thank you @init_27 for your answer - I think you’re right, as it works like that for the models, too. But in case of the csv-files, I have no clue how to do that, though. I had a look at the parameters on pandas documentary, but there is no such argument, is there?
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_csv.html?highlight=csv#pandas.DataFrame.to_csv

init_27 · February 17, 2019, 7:21pm

Hi,
Path is indeed mentioned in the doc link shared by you,

path_or_buf : str or file handle, default None
File path or object, if None is provided the result is returned as a string.

Changed in version 0.24.0: Was previously named “path” for Series.

Just make sure you pass it as the parameter,m it should show on Shift+Tab completion.

jbuzza · February 18, 2019, 3:15pm

Kaggle still seems to be using fastai 1.0.39.

Does anyone know how often they update or can this be done for ourselves ?

ilovescience · February 19, 2019, 4:38am

just add !pip install fastai==1.0.45 or whatever version you want at the beginning of your kernel…

sheepish · February 19, 2019, 9:21am

I just made public my kernel that uses lesson 2 code to classify horse images by color breed:

https://www.kaggle.com/nadjetba/fastai-v3-lesson-2-horses-by-color-breed/

BTW now Kaggle uses Fastai 1.0.45, check using !pip list

SteveR · February 19, 2019, 9:57am

I tried your suggestion of running the pip install for version 1.0.45 and this seemed to go ok, but now I’m getting a version conflict for ‘fastprogress’ when doing ‘from fastai.vision import *’:

VersionConflict: (fastprogress 0.1.18 (/opt/conda/lib/python3.6/site-packages), Requirement.parse(‘fastprogress>=0.1.19’))

jbuzza · February 19, 2019, 10:03am

For some reason when doing this I get an error when using from fastai.text import *:

VersionConflict: (fastprogress 0.1.18 (/opt/conda/lib/python3.6/site-packages), Requirement.parse('fastprogress>=0.1.19'))

This is despite the pip install seemingly successfully removing fastprogress 0.1.18 and installing fastprogress 0.1.19.

SteveR · February 19, 2019, 11:01am

I think it also needs to have the latest pytorch - doing this seemed to fix it for me:

!pip install pytorch==0.1.2

jbuzza · February 19, 2019, 2:45pm

Hmm, I get an error if I try !pip install pytorch==0.1.2:

Collecting pytorch==0.1.2
  Downloading https://files.pythonhosted.org/packages/a9/41/4487bc23e3ac4d674943176f5aa309427b011e00607eb98899e9d951f67b/pytorch-0.1.2.tar.gz
Building wheels for collected packages: pytorch
  Running setup.py bdist_wheel for pytorch ... error
  Complete output from command /opt/conda/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-q71f76b8/pytorch/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-5u3fis7x --python-tag cp36:
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-q71f76b8/pytorch/setup.py", line 17, in <module>
      raise Exception(message)
  Exception: You should install pytorch from http://pytorch.org

wdhorton · February 19, 2019, 2:55pm

Pytorch 0.1.2 is definitely not right

jbuzza · February 19, 2019, 3:22pm

Yep, I guess I should have realised that since pytorch>=1.0.0 is a requirement for fastai==1.0.45.

wgpubs · February 21, 2019, 12:17am

First, thank you guys for making this happen and maintaining this kernel.

Second, as someone new to Kaggle kernels, a couple of questions:

Is there a way to specify what version of fastai you want to use? Lets say I want to use .42 instead of the latest … how would I do that?
I’ve heard some reports of folks of the fastai image classification code taking a really long time to run on Kaggle. Just wondering if you all had seen this, and if so, what are the usual suspects and resolutions?

Thanks - wg

Daniel · February 21, 2019, 1:39am

How can I download trained model weights from Kaggle kernel?

I can find the folder of dataset using console, by doing
cd /tmp/.fastai/data

but I don’t know how to find the folder for storing weights, nor how to download the weight file to my local computer.

I searched on the forum, and someone had the same question and the solution was to commit the notebook and then maybe downloaded, but the notebook takes ages to commit, I could not verify whether this solution works or not. I wonder is there an easy and quick solution to it.

Has anyone had the similar question and got it solved? Thanks

ilovescience · February 21, 2019, 2:32am

I am not sure about your second question, but for your first:

init_27 · February 21, 2019, 8:46am

@wgpubs Let me benchmark the code and see what might be the issue, Thanks.

Daniel · February 22, 2019, 12:00am

Thanks @ini_27 and @wdhorton for this wonderful work!

There seems to be a lot of tricks and efforts poured in to make them work in Kaggle kernel. Even download dataset seems very tricky.

Everytime I ran Lesson 1 pets on Kaggle, downloading works fine; but when I start a new kernel to download Pets dataset, I got the same error message below

ConnectionError: HTTPSConnectionPool(host='s3.amazonaws.com', port=443): Max retries exceeded with url: /fast-ai-imageclas/oxford-iiit-pet.tgz (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f23189d9f60>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',))

Here are my quesitons:

in your lesson 1 pets notebook, it seems every time we run the notebook we have to re-download the pet dataset. does it mean we can’t save the pet dataset somewhere on Kaggle for reuse?
what do I need to do in order to download the pet dataset in my own new kernel?
how can I download my trained model/weights on Kaggle kernel into my local computer?

Thanks!

ilovescience · February 22, 2019, 12:14am

I have saved the dataset over here

If you do learn.save() and commit your kernel, it should be under output for you to be able to download

Also, that error is probably because you have internet connection turned off

wgpubs · February 22, 2019, 12:30am

Can someone tell me where we can upload the pre-trained AWD LSTM bits (if that is possible)?

The competition I’m working in doesn’t allow submissions when “Internet Connected” = True.