Platform: Kaggle Kernels


#62

Thank you @init_27 for your answer - I think you’re right, as it works like that for the models, too. But in case of the csv-files, I have no clue how to do that, though. I had a look at the parameters on pandas documentary, but there is no such argument, is there?
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_csv.html?highlight=csv#pandas.DataFrame.to_csv


(Sanyam Bhutani) #63

Hi,
Path is indeed mentioned in the doc link shared by you,

path_or_buf : str or file handle, default None
File path or object, if None is provided the result is returned as a string.

Changed in version 0.24.0: Was previously named “path” for Series.

Just make sure you pass it as the parameter,m it should show on Shift+Tab completion.


(Jon Burrell) #64

Kaggle still seems to be using fastai 1.0.39.

Does anyone know how often they update or can this be done for ourselves ?


#65

just add !pip install fastai==1.0.45 or whatever version you want at the beginning of your kernel…


#66

I just made public my kernel that uses lesson 2 code to classify horse images by color breed:

https://www.kaggle.com/nadjetba/fastai-v3-lesson-2-horses-by-color-breed/

BTW now Kaggle uses Fastai 1.0.45, check using !pip list


(Steve Roberts) #67

I tried your suggestion of running the pip install for version 1.0.45 and this seemed to go ok, but now I’m getting a version conflict for ‘fastprogress’ when doing ‘from fastai.vision import *’:

VersionConflict: (fastprogress 0.1.18 (/opt/conda/lib/python3.6/site-packages), Requirement.parse(‘fastprogress>=0.1.19’))


(Jon Burrell) #68

For some reason when doing this I get an error when using from fastai.text import *:

VersionConflict: (fastprogress 0.1.18 (/opt/conda/lib/python3.6/site-packages), Requirement.parse('fastprogress>=0.1.19'))

This is despite the pip install seemingly successfully removing fastprogress 0.1.18 and installing fastprogress 0.1.19.


(Steve Roberts) #69

I think it also needs to have the latest pytorch - doing this seemed to fix it for me:

!pip install pytorch==0.1.2


(Jon Burrell) #70

Hmm, I get an error if I try !pip install pytorch==0.1.2:

Collecting pytorch==0.1.2
  Downloading https://files.pythonhosted.org/packages/a9/41/4487bc23e3ac4d674943176f5aa309427b011e00607eb98899e9d951f67b/pytorch-0.1.2.tar.gz
Building wheels for collected packages: pytorch
  Running setup.py bdist_wheel for pytorch ... error
  Complete output from command /opt/conda/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-q71f76b8/pytorch/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" bdist_wheel -d /tmp/pip-wheel-5u3fis7x --python-tag cp36:
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-q71f76b8/pytorch/setup.py", line 17, in <module>
      raise Exception(message)
  Exception: You should install pytorch from http://pytorch.org

(William Horton) #71

Pytorch 0.1.2 is definitely not right


(Jon Burrell) #72

Yep, I guess I should have realised that since pytorch>=1.0.0 is a requirement for fastai==1.0.45.


(WG) #73

First, thank you guys for making this happen and maintaining this kernel.

Second, as someone new to Kaggle kernels, a couple of questions:

  1. Is there a way to specify what version of fastai you want to use? Lets say I want to use .42 instead of the latest … how would I do that?

  2. I’ve heard some reports of folks of the fastai image classification code taking a really long time to run on Kaggle. Just wondering if you all had seen this, and if so, what are the usual suspects and resolutions?

Thanks - wg


(深度碎片) #74

How can I download trained model weights from Kaggle kernel?

I can find the folder of dataset using console, by doing
cd /tmp/.fastai/data

but I don’t know how to find the folder for storing weights, nor how to download the weight file to my local computer.

I searched on the forum, and someone had the same question and the solution was to commit the notebook and then maybe downloaded, but the notebook takes ages to commit, I could not verify whether this solution works or not. I wonder is there an easy and quick solution to it.

Has anyone had the similar question and got it solved? Thanks


#75

I am not sure about your second question, but for your first:


(Sanyam Bhutani) #76

@wgpubs Let me benchmark the code and see what might be the issue, Thanks.


(深度碎片) #77

Thanks @ini_27 and @wdhorton for this wonderful work!

There seems to be a lot of tricks and efforts poured in to make them work in Kaggle kernel. Even download dataset seems very tricky.

Everytime I ran Lesson 1 pets on Kaggle, downloading works fine; but when I start a new kernel to download Pets dataset, I got the same error message below

ConnectionError: HTTPSConnectionPool(host='s3.amazonaws.com', port=443): Max retries exceeded with url: /fast-ai-imageclas/oxford-iiit-pet.tgz (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f23189d9f60>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',))

Here are my quesitons:

  1. in your lesson 1 pets notebook, it seems every time we run the notebook we have to re-download the pet dataset. does it mean we can’t save the pet dataset somewhere on Kaggle for reuse?
  2. what do I need to do in order to download the pet dataset in my own new kernel?
  3. how can I download my trained model/weights on Kaggle kernel into my local computer?

Thanks!


#79

I have saved the dataset over here

If you do learn.save() and commit your kernel, it should be under output for you to be able to download

Also, that error is probably because you have internet connection turned off


(WG) #80

Can someone tell me where we can upload the pre-trained AWD LSTM bits (if that is possible)?

The competition I’m working in doesn’t allow submissions when “Internet Connected” = True.


(深度碎片) #81

Hi @ilovescience

Thanks for your very helpful response! The downloading problem and dataset reuse problem have been dealt beautifully.

Also thanks for uploading oxford pets dataset! I found it was very very slow to upload, actually I think it is not uploading at all. do you know why?

As for weights, do you know how to save weights on Kaggle and reuse it again later?

Thanks a lot!


#82

I am not sure why it was not uploading for you…

I have tried using learn.save('stage-1') and sometimes it will output under Output section of the kernel when you commit, and sometimes it does not… It depends on the weird directory structure of the Kaggle Kernels and where you have the model directory… I will try to figure out how to do that…