Helper script for downloading files from platform.ai

Part 2 (2017)
1

I got a little tired of downloading each file individually from platform.ai. This script has been useful for me, thought I’d share!!

Just use python3 syncfiles.py http://www.platform.ai/part2/lesson1/ lesson1 to download all the files into a new folder called lesson1

    from bs4 import BeautifulSoup
    import sys, requests, os

    import zipfile

    def unzip(path_to_zip_file):

        zip_ref = zipfile.ZipFile(path_to_zip_file, 'r')
        fold, fil = os.path.split(path_to_zip_file)
        zip_ref.extractall(fold)
        zip_ref.close()

    def download(url, file_name):
        with open(file_name, "wb") as file:
            response = requests.get(url)
            file.write(response.content)


    def main(download_url, folder_name):

        doc = BeautifulSoup(requests.get(download_url).text)
        fils = [item.text for item in doc.find_all('a') if '.' in item.text]

        try:
            os.makedirs(folder_name)
        except OSError:
            pass

        for fil in fils:
            file_download_url = download_url + fil
            file_name = os.path.join(folder_name, fil)
            download(file_download_url, file_name)
            if file_name.endswith('.zip'):
                unzip(file_name)

    if __name__ == '__main__':
        download_url = sys.argv[1]
        folder_name = sys.argv[2]

        main(download_url, folder_name)
4 Likes
2

Thanks @ben.bowles that’s great.

FYI you forgot to put this in the ‘part2’ category, so I’ve moved it. I also prepended all your code with 4 spaces so that markdown formats it as code.