Deep learning actual workflow

Fellow deep learners,

It has been a few years now that I play with machine learning/deep learning.

Up to now, I was able to run models on my own machine (provided by the office, Windows) & google colab. However, I am faced today with the desire to use a library that is only available in linux.

  • I therefore can’t use my office machine (Windows only).
  • I did not suceed installing the package on Colab. Docker seems to not be available under colab even following these advices (if someone has a colab notebook with docker working & associated commands to install, I would gladly take it)

Therefore, I am left with AWS, Google Cloud, Paperspace & similar options. I am wondering how some of you guys go about this ?

  • On AWS, up to now, I was only able to get a CLI interface (a graphical interface using linux was too slow). But on my machine, I am using VScode for debugging, … using the CLI for debugging with e.g. VI seems so odd to me. Is this a mandatory step in order to work with deep learning on the cloud ?
  • I did not test paperspace yet but I am wondering if I will not face the same issues as on Google colab regarding docker
  • I guess I could use jupyter notebook to stream from the AWS machine to an IP which I can access from a web browser. But then again, debugging capabilities in jupyter notebook are not so good as in VScode. Is then nbdev the way to go ? And also, wouldn’t I run in the same docker issues as in google colab ?

Any advice would be appreciated. I am trying to get up to speed in managing deep learning models in a more professional way at the moment and this bugs me like nothing that I am stuck because of hardware things :slight_smile:

Thanks in advance for your help and time !

Personally, I use the CLI and Jupyter notebooks to do all of my work. Knowing how to use the CLI is useful anyway, especially if you’re using computers over the network.

If you want to use VSCode, you’ll need to run a graphical interface. No way around that. If the graphical interface is too slow, then it may be worth finding out why.

2 Likes

The more I look at it, the more I believe I will have to learn to enjoy CLI, Vi, …
Life is hard :slight_smile:

You can use any CLI text editor, I personally use nano on Ubuntu as I am not a good on VIM.