Floyd - alternative to AWS P2 instance

I know exactly what you mean. It costs me about 2000 INR a month.

There is a much more affordable (almost 80% less expensive) option - using p2 Amazon Spot Instances. It takes a little extra effort to setup, but the process is well documented in the fast.ai wiki : http://wiki.fast.ai/index.php/AWS_Spot_instances

I also tend to spend a lot of time reading through the notebook and doing my research before turning my p2 machine on. :slight_smile:

There are issues uploading large data, you can use a script to download but what I found works best is preprocessing the data locally. If you resize images to the actual size you are going to use in your network (224x224, or maybe 448x448 for data augmentation zooming) then the dataset becomes much much smaller and you can easily upload it to floydhub.

What I like most is that you don’t have to worry about shutting down stuff like you do with AWS. When you’re done you’re done.

What I don’t like is the way data and job output is managed. It’s not easy to remove old jobs. It would also be nice to combine data.

I also notice that the times reported in the logs are not very reliable. According to the logs there is nothing happening at the beginning of a job, while in fact there is because doing less shortens this ‘startup lag’.

I always download the data first (and unpack) and then use the output in subsequent runs. I was not able to upload the training data on job submission even after pre-processing.
To remove the old data you can just do:

floyd data delete DATA_ID
I’d like to have a way to mount multiple outputs from other jobs as an input to the new one.

How can I access tensorboard on floydhub?
I have problem navigating to localhost:6006 (http://172.17.0.5:6006) to view tensorboard.


Thank you @kijes, I’ll try that.

I only used Tensorboard for offline viewing (after the job has finished).
I used Tensorboard callback from Keras to generate data to “/output” folder, then you can download the data and run Tensorboard locally:

tensorboard --logdir=path/to/log-directory

This is not ideal if you want real time visualization, but you can always check logs of the running job for some statistics. Maybe there is a better way.

I am new to this course and machine learning to. How can i setup my local machine with a gpu to start course? Is there any way to do it?

A bit late to the party but one of the text embedding datasets would certainly be nice, like the Glove Word vectors…

Floydhub looks like it will be the future of ML, it’s very easy to use and setup (I figured out pretty much everything immediately, and I am a ML novice). Problem for me was that the version of keras they have on their theano instances was old, and I had no end of trouble trying to make it work. This would be fine if I knew what I was doing with keras and ML, because then I could just rewrite the scripts, which I did, but then it threw some other error and I gave up (for now). I expect to be back soon though because it seems like Floydhub is very promising

Hi! We released a new set of theano images with keras 2.0.3 installed. You can try it out by passing --env theano-0.9 or --env theano-0.8 to the run command. We will be updating the documentation soon to pick up all the new images including tensorflow-1.2.

@natedl98 Please update your CLI (pip install -U floyd-cli) and try the new environments that @houqp mentioned above

1 Like

Awesome, can’t wait to try them out! Considering I just last used FloydHub like a couple weeks ago, y’all must be moving at a fast pace- very nice work

FloydHub is YC, correct?

Thanks! Yes, we were in the last (W17) YC batch :slight_smile:

3 Likes

Hi Sai

I’m using this guide but i have some errors, any idea?

import utils; reload(utils)
from utils import plots

WARNING (theano.sandbox.cuda): CUDA is installed, but device gpu is not available  (error: Unable to get the number of gpus available: no CUDA-capable device is detected). Using Theano backend.

Hi Sai
Do you know why this error appears?

import utils; reload(utils)
from utils import plots

WARNING (theano.sandbox.cuda): CUDA is installed, but device gpu is not available (error: Unable to get the number of gpus available: no CUDA-capable device is detected). Using Theano backend.

There is no error :slight_smile:

What you see is the previously saved notebook. When you actually run the cell, you should see:

Using cuDNN version 5110 on context None
Preallocating 10867/11439 Mb (0.950000) on cuda
Mapped name None to device cuda: Tesla K80 (0000:00:1E.0)
Using Theano backend.
1 Like

Hi, I just signed up, but when I login, my browser did not open so I did not get an authentication token. Can you please help? Thanks

Susan

Never mind, I was able to figure it out. But now I am having problem uploading data, is it because my trial 3600 seconds ran out? My jupyter notebook has become “404”.

Susan

is there any chance that you’ll offer the free 100h trial again (or perhaps something similar)?

I am having the same issues with Floyd on uploading data. What did you end up using? I’m having “Segmentation fault (core dumped)” problem with AWS when downloading dogscats data. So frustrated.

If you don’t have access to a browser for floyd login, please use the --token flag. See http://docs.floydhub.com/commands/login/