Kaggle-cli issues

(javed.qadruddin) #1

Post here for problems and discussion of the kaggle-cli tool

Lesson 2 discussion - beginner
How to Download a Kaggle competition Dataset in Crestle
(javed.qadruddin) #2

I’m getting this error when trying to submit using the kaggle cli tool:
$ kg submit ‘output.csv’ -c ‘dogs-vs-cats-redux-kernels-edition’
Starting new HTTPS connection (1): www.kaggle.com
’NoneType’ object has no attribute ‘find’

Has anyone seen this before/know how to fix it? I tried pip upgrading kaggle-cli, but that didn’t help.


Do you have a Kaggle account, and did you set your username and password information (i.e., “kg config -g -u username -p password -c dogs-vs-cats-redux-kernels-edition”) ?

(javed.qadruddin) #4

Yes, I did the username and pass config. It was working just the minute before this error started showing up. I think I’ve figured it out though… I’ve probably hit the max number of submissions per day cap for that competition and this the error you get when that happens?

(Jeremy Howard) #5

Could be - try using the FileLink() trick I showed in the last class to download a file that you can then submit using your browser. That way you’ll be able to see any errors that occur.

(javed.qadruddin) #7

Thanks. The problem was that I hit the cap for the day. Now that the UTC day has rolled over, it is working again.

(Sam Shleifer) #8

Did anyone figure out how to use kaggle-cli if you logged into kaggle through google.

(Samuel Ekpe) #9

#just compile this python code and use it to download the files

  GNU nano 2.5.3                                  File: ../download_data.py                                                                           

import requests

# The direct link to the Kaggle data set
data_url =raw_input('Please enter the url for the data: ')
print "To download data from",data_url 

# The local path where the data set is saved.
local_filename = data_url.split('/')[-1]
print "To save as",local_filename

# Kaggle Username and Password
kaggle_info = {'UserName': "username", 'Password': "password"}

# Attempts to download the CSV file. Gets rejected because we are not logged in.
r = requests.get(data_url)

# Login to Kaggle and retrieve the data.
r = requests.post(r.url, data = kaggle_info)

# Writes the data to a local file one chunk at a time.
f = open(local_filename, 'w')
for chunk in r.iter_content(chunk_size = 512 * 1024): # Reads 512KB at a time into memory
    if chunk: # filter out keep-alive new chunks
        print chunk

(Alex) #10

I have used this trick with Facebook, expect it to work with Google, too. Ask kaggle to reset your password. You will recieve e-mail with three links, the last one allows you to setup username/password pair. Then just use this data with kaggle cli as normal.


I used w3m browser on my aws instance was much easier…install with apt-get install w3m and then use commad w3m “url” to get to kaggle and DL

(Maxim Pechyonkin) #12

I tried to to that, I was able to log in with my password and the website looked like this:

However, when I go to most of the pages, the page will load and will only display this:

I do not know how to proceed as there are no links displayed. Is there another way to get data to AWS?

(Maxim Pechyonkin) #13

As of today, Kaggle seems to have changed html of their pages so kg donwload will not load data even with the proper setup. There is a new opened issue on GitHub, which was not resolved yet.

Do you have any suggestions on how to get data to AWS from a local machine or directly from Kaggle?

(Orlando Adeyemi) #14

@maxim.pechyonkin have you tried this hack ?

(Maxim Pechyonkin) #15

The issue has been patched, it works now.

(Dmitrij) #16

Yep, just need to upgrade the kaggle-cli package:

pip install --upgrade kaggle-cli


Heads up for people who run into error messages like the following :point_down:

reduce() of empty sequence with no initial value

I did some digging around the kaggle-cli repository and found this issue where the author mentioned that he fixed this in the latest version. It looks like the issue was resolved 7 days ago, so in case anyone is running an older version of kaggle-cli like I was, you can run pip install --upgrade kaggle-cli to fix this issue as well.

(Moshe) #18

This worked for me, I was getting this error “Node Type is missing find-all” trying to donwload the competition.
Youre suggestion to upgrade fixed it.

(Sradhanjali) #19

I have used the same trick. Now I use the username and password in kg config command. But when I use kg download, it says ->

Starting new HTTPS connection (1): www.kaggle.com
There was an error logging in: The username or password provided is incorrect.

However, I can login to the kaggle website directly using the same user name and password.
Any idea what could be the issue?

(Sradhanjali) #20

Even after doing
kg config -g -u username -p password -c dogs-vs-cats-redux-kernels-edition

I had to pass username and password in kg download command again to resolve this issue
kg download -u username - p password -c dogs-vs-cats-redux-kernels-edition

(Dhairya) #21

I’m still having issues downloading cats vs dogs using the kaggle-cli. Each time I attempt to download I get:
Warning:download url for file test.zip resolves to an html documentrather than a downloadable file.

and then the download aborts. Any suggestions?