How to download data for Lesson 2 from Kaggle for Planet Competition


(Stas Bekman) #41

Combining all the useful suggestions from this thread, here is what I needed to do - and hopefully you just need to copy-n-paste it to work:

### kaggle (native) tool setup ###
pip install kaggle
mkdir ~/.kaggle/
# get the API key from your account 
1. visit https://www.kaggle.com/ => login => My Account 
   e.g. https://www.kaggle.com/YourUsername/account
2. hit [Create New API Token] 
3. save the file as ~/.kaggle/kaggle.json
4. set permissions
chmod 600 ~/.kaggle/kaggle.json

### get the data for the competition ###

# 1. accept the rules here:
https://www.kaggle.com/c/planet-understanding-the-amazon-from-space/rules
you may need to verify your kaggle account/phone for this to work.

# 2. download data
kaggle competitions files -c planet-understanding-the-amazon-from-space
COMPETITION=planet-understanding-the-amazon-from-space
DATA=~/data/planet  # your path to data
mkdir -p $DATA
kaggle competitions download -c $COMPETITION -f train-jpg.tar.7z -p $DATA
kaggle competitions download -c $COMPETITION -f test-jpg.tar.7z -p $DATA
kaggle competitions download -c $COMPETITION -f test-jpg-additional.tar.7z -p $DATA
kaggle competitions download -c $COMPETITION -f train_v2.csv.zip -p $DATA
kaggle competitions download -c $COMPETITION -f test_v2_file_mapping.csv.zip -p $DATA
kaggle competitions download -c $COMPETITION -f sample_submission_v2.csv.zip -p $DATA

# 3. unpack/cleanup
sudo apt install p7zip-full
cd $DATA
7z x -so train-jpg.tar.7z | tar xf - 
7z x -so test-jpg.tar.7z | tar xf - 
7z x -so test-jpg-additional.tar.7z | tar xf - 
unzip train_v2.csv.zip
unzip test_v2_file_mapping.csv.zip
unzip sample_submission_v2.csv.zip

# optional cleanup:
# rm *zip *7z
# rm -rf __MACOSX

Thank you all who contributed code and suggestions!