Google Colab not persisting Kaggle

Hi, I have followed a few different tutorials on how to get Google Colab to persist Kaggle API information (~/.kaggle/kaggle.json), and while I do get my json file uploaded the instance does not save it to my Google Drive therefore in order to use the Kaggle CLI I have to run the steps all over again. This is a big problem when downloading datasets such as the planet dataset (over 600mb), what am I doing wrong? Thanks!

Every time you make a request through kaggle API it always looks for the file kaggle.json stored at ‘~/.kaggle/kaggle.json’ and whenever you link your google drive to colab, it is linked at drive/ so kaggle API won’t look there. When you start a new session all the data is wiped out and you need to upload the kaggle.json file again.

If you don’t want to do that, use cliget extension if you are using Mozilla or CurlWget with Chrome. Click on the download button on kaggle dataset page and then cancel the download. Now go to the extension and it will create a download link for you for that dataset. You can use wget <link> in the colab cell to download the dataset without using kaggle API. Also, this is a temporary link so it might not work the next day and you need to create a new link again. I suggest removing these links in case you are publishing the notebook on GitHub or somewhere else publically as it might have some session info or something like that.

1 Like