How to Download a Kaggle competition Dataset in Crestle

Thanks!

…however what should I do to make this work in practice with kaggle-cli :slight_smile: ?

For others potentially strugging with the same problem: I finally managed to get the data loaded to Crestle using the cURL trick showed by Jeremy in the Machine Learning course lesson 1 video (around 23:00 minutes). However, I wasn’t able to paste anything (in this case the long cURL) to Crestle terminal using Firefox. When I tried with Internet Explorer, the “paste” option became available when right-clicking…

pip install git+https://github.com/floydwch/kaggle-cli.git should fix the kaggle-cli problem.

4 Likes

Thanks, Everyone , that worked. But it will also be good @anurag can provide an upload with a zip files for the dataset then it will be great. This will allow us to prepare the dataset preprocessing and update it to crestle .

For me this had to be pip3 install, Just posting in case anybody else has that issue.

2 Likes

I am having an issue getting the Dog Breed Identification dataset to download as shown below.
Please offer suggestions. Thank you.

(fastai) ubuntu@ip-172-31-17-26:~$ kg config -g -u ‘bdekoven’ -p ‘xxxx’ -c ‘Dog Breed Identification’
(fastai) ubuntu@ip-172-31-17-26:~$ kg download
competition not found
(fastai) ubuntu@ip-172-31-17-26:~$ kg config -g -u ‘bdekoven’ -p ‘xxxx’ -c Dog-Breed-Identification
(fastai) ubuntu@ip-172-31-17-26:~$ kg download
’NoneType’ object has no attribute ‘find_all’
(fastai) ubuntu@ip-172-31-17-26:~$

Try doing a pip install kaggle-cli --upgrade and see if it resolves the issue.

Also, see similar issues here and here. Maybe these will help? :slight_smile:

Also it may(?) be case sensitive: dog-breed-identification

@anurag Hi Anurag, would it be too much to ask if dog-breed-identification can be added in kaggle/datasets of crestle ?

Here is the command line which worked: $ kg config -g -u bdekoven -p xxxx -c dog-breed–identification
Then I could download and unzip.

By the way I am running on AWS, sorry to post here since “in Crestle”

Thank you for the suggestions!

The dataset is now available on Crestle under /datasets/kaggle/dog-breed-identification.

1 Like

Thanks a lot!

1 Like

Under which directory of crestle? I can’t find the dataset… And When I try to use the kaggle cli, I get the error like "pkg_resources.DistributionNotFound: The ‘lxml<4.1,>= 4.0.0’ distribution not found and is required by kaggle-cli "

The /datasets/kaggle/dog-breed-identification directory at the root of the filesystem.

You can fix the lxml issue by installing an older version that’s needed by kaggle-cli:

pip3 install lxml==4.0.0

The problem is not with the Non-Type, I guess it is the problem with the password/username/competition, When I execute
-kg config -g -u bdekoven -p xxxx -c dog-breed-identification
it works.

It works! Thanks!

Glad you got this to work. Sorry I did not respond sooner.

It is also possible to download specific file with kaggle-cli

$ kg download -u <username> -p <password> -c <competition> -f train.zip
1 Like

hi may I ask how to get the url from web? I used right click the download button and copy url but the file download was broken. Thanks

Thanks a lot for your suggestion. Worked like a charm!

I just set up a crestle account and try to download data from kaggle. However I can’t install unzip because anconda doesn’t seem to be pre-installed.
When I am enter:

conda install unzip
bash: conda: command not found

Am I missing something? Should conda be set up already?