Getting fastai and Dropbox links to play nice

Part 1 (2019)
1

Hi there,

I’m having a hard time getting the fastai library to play nice and properly download a file from a Dropbox link.

Here’s an example: https://www.dropbox.com/s/3no8bw40ccae0v5/bike_or_motorcycle.tar.gz?dl=1

I’ve tried multiple versions tweaks to the untar_data, download_data and download_url, but nothing seems to work.

I eventually wrote a super dirty workaround, as seen below:

import urllib
import os
import tarfile

url = "https://www.dropbox.com/s/3no8bw40ccae0v5/bike_or_motorcycle.tar.gz?dl=1"
u = urllib.request.urlopen(url)
data = u.read()
u.close()

if not os.path.exists('/tmp/.fastai/data/'):
    os.makedirs('/tmp/.fastai/data/')

with open('/tmp/.fastai/data/bike_or_motorcycle.tar.gz', "wb") as f :
    f.write(data)

tar = tarfile.open('/tmp/.fastai/data/bike_or_motorcycle.tar.gz')
tar.extractall(path='/tmp/.fastai/data/')
tar.close()

I’d really like to avoid having to import urllib, os and tarfile, and I’d especially like to avoid managing folder paths, mostly because I know fastai can take care of almost all of this with other downloads.

And just to be clear: I know I can download my files outside of fastai (i.e. using wget <url> through the terminal), but I’m trying to create an all-in-one solution where I can download and unzip a file from a Dropbox link only using the Jupyter notebook.

Thanks, everyone!

2

You can use commands in jupyter really easily!

For example:

for i in range(10):
    !echo $i

So you should be able to write a function or class,
that gets the file using wget and other commands without using those imports.

I am not sure this is a possible solution to your problem though… :slight_smile:

3

It would help greatly if you could say what didn’t work when you tried to download_url and untar_data the dropbox file.