Kaggle API issue

[ricardocalleja]

Hi there,

I am having a little problem to make my Kaggle API to work properly, these are the steps I have follow up to now. I am working on a VM creaded with Paperspace (Free-P5000)

1. I requested the API and I acepted the terms of the competition.
   
   
2. I loaded the kaggle.json file in root/kaggle
   
3. this is the setup in the jupyter notebooks
4. But when I try to look the data I get this error:
   
   
   
   
   
   
   FileNotFoundError: [Errno 2] No such file or directory: ‘/storage/archive/bluebook/TrainAndValid.csv’

I belive that the error is in the path:

because in the video when Jeremy runs that cell it appears something more specific like this:

I would really appreciate your help guys

[joedockrill]

Some of the older datasets do get taken down. Go to kaggle and navigate around the data folders and see if the files are actually still on kaggle

[ricardocalleja]

@ricardocalleja

TrainAndValid.csv

@joedockrill yes the data is there, as shown in the image:

[ricardocalleja]

I did it manually, I just downloaded the bluebook-for-bulldozers.zip file, and loaded in the VM at storage/archive/bluebook, then from the terminal in the bluebook folder I ran unzip bluebook-for-bulldozers.zip and it worked.

But I would really like to learn how to do it using just the code shown in the jupyter notebook.

Thanks again @joedockrill

Regards

[joedockrill]

I can see your API key in the first image, you should delete that

[micstan (Michal Staniszewski)]

@ricardocalleja, are you sure you didn’t create the directory first? Because you keep everything (not only mkdir) under if not path.exists() and yet it prints you ls.

[ricardocalleja]

@micstan

re you sure you didn’t create the directory first?

sorry @micstan I don’t understand what you mean by " Because you keep everything (not only mkdir ) under if not path.exists()’

[micstan (Michal Staniszewski)]

I believe you removed the screen because of the api keys but i believe the whole section with the call was wrapped with if not path.exists(). If the path existed before, and you mentioned you also downloaded it manually, the code didn’t run, you didn’t send api request. Just try to run it without checking for path and see what error message you get.

[ricardocalleja]

@micstan

if not path.exists()

Fine, now it does not gives me any error. Thanks @micstan

[micstan (Michal Staniszewski)]

What i meant is to remove the indentation before api.competition_download_cli and file_extract. Now it only runs if there is no path already created. If you created the path before running this cell it just didn’t send api request.

[ricardocalleja]

Done, It now it gives me an error (check the image in the last comment)

[micstan (Michal Staniszewski)]

Ah, great, now i see. I see you accepted the terms so my best guess is the location of your kaggle.json. The default expected location is ~/.kaggle (mind the dot!). After downloading the file, from the location of the file run this and try again:

mkdir ~/.kaggle
mv kaggle.json ~/.kaggle/kaggle.json
chmod 600 ~/.kaggle/kaggle.json

[ricardocalleja]

I did it as you told me,

I still believe that the problem is related with the path in the line:

path = URLs.path('bluebook') path

[micstan (Michal Staniszewski)]

The error you get it is just kaggle authorization error.

1. Make sure you have your kaggle.json in ~/.kaggle (the error you show now suggests that you already have .kaggle directory). Run ls ~/.kaggle, you should see the file.
2. Just run api.competition_download_cli commend in a separate cell

[ricardocalleja]

1. 
2. 

If I run the if not path.exist(): without the indentention as you suggested it gives me the same error as before

I will just leave it like before. and keep with it. Thanks for your time @micstan

[micstan (Michal Staniszewski)]

ok, if you want to troubleshoot in the future make sure the ~/.kaggle/kaggle.json exists (you do not have to provide credentials in any other way, now i’m thinking that maybe you created environment variables as in the first screenshot you shared so running unset KAGGLE_USERNAME and unset KAGGLE_KEY could help) and you can run this to test if you still get 401 error:

from kaggle import api
api.competition_download_cli('bluebook-for-bulldozers')

[ricardocalleja]

You were right the problem was always the location of the .json file

[micstan (Michal Staniszewski)]

great! good luck with the bulldozers!

[wlw]

Hi there,
I’m having a similar problem to @ricardocalleja - getting an error “ApiException: (401) Reason: Unauthorised” when I try to download the bulldozers data.

I originally tried to pip install kaggle before reading on to rest of the notebook, and followed instructions online to add kaggle.json to .kaggle folder, which might have messed with the fastbook code somewhat.

A terminal search revealed that I do indeed have the kaggle.json file in .kaggle and I have run chmod 600 so hopefully permissions are correct.

Help would be much appreciated! Thank you!

[wlw]

Just after posting, I realised I should try manually downloading data like @ricardocalleja suggested - and that did indeed work, I navigated to ‘bluebook’ folder then ran line kaggle competitions download -c bluebook-for-bulldozers from kaggle website.
So I’m sorted, but would be interested to know what was wrong, and maybe it will help others, so thought I’d keep this up. (let me know if better to take down!)