How to download data for Lesson 2 from Kaggle for Planet Competition

(Stas Bekman) #41

Combining all the useful suggestions from this thread, here is what I needed to do - and hopefully you just need to copy-n-paste it to work:

### kaggle (native) tool setup ###
pip install kaggle
mkdir ~/.kaggle/
# get the API key from your account 
1. visit => login => My Account 
2. hit [Create New API Token] 
3. save the file as ~/.kaggle/kaggle.json
4. set permissions
chmod 600 ~/.kaggle/kaggle.json

### get the data for the competition ###

# 1. accept the rules here:
you may need to verify your kaggle account/phone for this to work.

# 2. download data
kaggle competitions files -c planet-understanding-the-amazon-from-space
DATA=~/data/planet  # your path to data
mkdir -p $DATA
kaggle competitions download -c $COMPETITION -f train-jpg.tar.7z -p $DATA
kaggle competitions download -c $COMPETITION -f test-jpg.tar.7z -p $DATA
kaggle competitions download -c $COMPETITION -f test-jpg-additional.tar.7z -p $DATA
kaggle competitions download -c $COMPETITION -f -p $DATA
kaggle competitions download -c $COMPETITION -f -p $DATA
kaggle competitions download -c $COMPETITION -f -p $DATA

# 3. unpack/cleanup
sudo apt install p7zip-full
cd $DATA
7z x -so train-jpg.tar.7z | tar xf - 
7z x -so test-jpg.tar.7z | tar xf - 
7z x -so test-jpg-additional.tar.7z | tar xf - 

# optional cleanup:
# rm *zip *7z
# rm -rf __MACOSX

Thank you all who contributed code and suggestions!