Thanks for the guide, @FourMoBro. A few updates for the Ubuntu installation. I’m running 18.04 LTS.
pip install azure - no longer works, but doesn’t seem necessary to run the nbs
pip install "ipywidgets >= 7.5.1" - requirement already satisfied
pip install scikit_learn - requirement already satisfied
Otherwise, I haven’t had any issues thus far.
I installed fastai2 according to your guide in the top post. Thanks for the clear instructions!
How do I keep the fastai2 that my notebooks use up-to-date? I have been using pip install fastai2 fastcore --upgrade.
The reason I may be confused is that the above command replies
Requirement already satisfied: fastai2 in /home/malcolm/anaconda3/envs/fastai2/lib/python3.7/site-packages (0.0.16)
Yet when in the fastai2 directory, git pull updated many files. Also, I saw in another user’s notebook that they are using fastai2 0.0.17.
To be clear, I do not want to contribute yet to fastai2. I only want to be using the latest released version.
Thanks for your help!
@Pomo, When you run:
pip install fastai2 fastcore --upgrade
You are getting your fastai2 from pypi. That version is 0.0.16, it was released on March 30th 20220.
The reason why your are seeing the following message:
is because, you are trying to install the fastai2 last version found at pypi, and you already have it on your local machine. Therefore, the Requirement already satisfied
As for the other notebook:
In that notebook, they most likely installed the latest version of fastai2 (0.0.17 as of today) using one the 2 following options:
Option 1: From Github - Non editable version
you have to install directly from the fastai2 master branch like this:
pip install git+https://github.com/fastai/fastai2.git
As a good practice, you have to install fastcore at the same time, like this:
pip install git+https://github.com/fastai/fastcore.git
Option 2: From Github - Editable version
fastai2 editable versiongit clone https://github.com/fastai/fastai2
cd fastai2
pip install -e .
fastcore editable versiongit clone https://github.com/fastai/fastcore
cd fastcore
pip install -e .
Every time, you want to upgrade to latest version of both fastai2 and fastcore, you run (from the corresponding folder):
git pull
This is already a long answer but if you would like to learn more about this subject, I wrote this blog post: 3 ways to pip install a package: fastai2 use-case where you will find a more detailed answer.
Thanks so much! Your blog post explained exactly how to proceed. Now updated to v 0.0.17.
What was confusing was the need to clone the fastai2 repo before creating the new conda environment, while the fastai2 in actual use is coming from pypi. Is that fastai2 clone now irrelevant?
Again, thanks for your willing response. It saved me a lot of frustration.
@Pomo, You are very welcome!
The cloning will copy the environment.yml file on your local machine. Then, it’s used when creating the conda virtual environment like this:
conda env create -f environment.yml
If you are not interested in cloning the fastai2 repo, you can create an empty environment.yml text file on your local machine, and copy the content found in the repo and paste in your newly created file, and run the command above. Afterwards, you pip install fastai2.
Even better, I think you can just add fastai2 as a dependency in the environment.yml file, and just run the same command ( without the pip install fastai2 needed here above):
conda env create -f environment.yml
I just tried the option (with the strike-through) mentioned here above, and it doesn’t work because fastai doesn’t have yet a fastai2 package in the Anaconda Repository. Until then, one have to create a virtual environment using the the environment.yml file found in the fastai2 repo, and then pip install fastai2.
Since it has been more than a month, and the post is not a wiki, I cannot edit the first post, but here is a 4/24/20 Update:
With the new release of Ubuntu 20.04 LTS in the past 24 hrs, I decided to rebuild my VM using this new LTS image. I then proceeded to follow my original instructions to see if they still work. Everything works, for the most part. Here is what I found, similar to what others have since posted, for step 9:
Replace pip install graphviz with conda install graphviz or as @bsalita suggested, conda install python-graphviz
DO NOT run pip install azure You will get errors, and it appears that it may not be needed. (credit @neuradai )
Otherwise, the install steps still work as originally written, even if some of the last few commands can be skipped or changed slightly. Happy learning!!!
Any particular reasons for going to 20.04? I purposely went 18.04 hoping all the bugs were ironed out.
20.04 is an LTS just like 18.04. It just came out, so there may be interest by others to try it out. For me it is in a virtual machine running as a headless server that I ssh into, and this VM only runs fastai. If it breaks, so what, I have the 18.04 server in a VM to fall back on. So no real reason other than, “why not?”. I will try out the desktop build in another VM maybe tomorrow.
For those with an older CUDA or Nvidia driver (in my case CUDA 10.1 and driver 418.67), a workaround is to force pytorch 1.4 and CUDA 10.1
You can do this by editing a copy of environment.yml (https://github.com/fastai/fastai2/blob/master/environment.yml) to replace the line “pytorch>=1.3” with the lines:
cudatoolkit=10.1
pytorch=1.4
Otherwise follow the instructions by FourMoBro above
Just a heads up for part 2 and Swift4Tensorflow much of the early release were on LTS 18.04.
Perhaps not sure if that’s included in V4 Part2 this year. The latest from the toolchain suggest 18.04 but for 20.04 would probably need a local compile from source.
I had no problem upgrading 18.04 to 20.04 with these instructions:
Fast.ai environment setup with local GPU - Ubuntu 20.04
I’ve a laptop with Intel Core i9, 8 GB RAM (to be upgraded to 16GB after the crisis), Nvidia 1650 with 4GB vRAM. Though there are multiple cloud options, running your notebooks locally feels great, at least for me. So I setup dual boot with Ubuntu 20.04 on my laptop. I ran the first 3 notebooks without any issues (except for the first lesson, the pet dataset takes longer than a while to download). All the information you’re seeing below is already present on the forums and FastAI Github page and course page. I had to do this twice since I messed up my OS setup and I had to search in the forums again to complete this. So I thought of putting up a note for myself and share it with you people. Thanks to @FourMoBro and all my other classmates who pitched in the forums.
Github Links
git clone https://github.com/fastai/fastai2
cd fastai2
conda env create -f environment.yml
source activate fastai2
git clone https://github.com/fastai/fastai2
cd fastai2
pip install -e ".[dev]"
I made a silly mistake in forgetting to install after creating the environment above. So, when I tried running the course notebooks,it was not able to find the fasti2 library
conda install pyarrow
pip install pydicom kornia opencv-python scikit-image
fastai2 installation is now done.
For the course/fastbook notebooks to run, there are some extra dependenices that you might want to install
pip install graphviz
pip install azure-cognitiveservices-vision-computervision
pip install azure-cognitiveservices-search-websearch
pip install azure-cognitiveservices-search-imagesearch
pip install "ipywidgets>=7.5.1"
pip install sentencepiece
pip install scikit_learn
Link to Google Docs version of the same
I’ll try and update the notes with how I installed the Nvidia GPU driver as well.
I have just setup a DL workstation using an RTX 2070 Super and have used Unigene Heaved and Superposition to compare my setup and they seem just about there.
Are there any such standardized deep learning benchmarks I can use? I have been trying to replicate notebooks in Fastbook which I have previously run on Colab Pro and GCP using a Tesla P100, but I don’t think that gives an accurate representation…
Not to the best of my knowledge: Here’s what I did, bench marked individual parts and compared their scores online (after averaging over 5).
Any specific tests you used to benchmark the individual parts? I used the Unigene suite to benchmark the GPU, specifically Heaven and Superposition and compared my results to the leaderboard scores with the similar configs.
pip install graphviz
Thank you very much, worked beautiful (ubuntu 18.04, GeForce 2080Ti, driver version 440.100, CUDA v 10.2)
Hi everyone,
I’m totally new to this topic and i’ve been struggling with the set up part.
I have a GeForce MX150 on an Asus Laptop.
I am unable to install properly the Nvidia driver (tried with 390 and 440 version) and obtain the first picture of this post. I have followed the instructions but get the error :
“NVIDIA-SMI has failed because it couldn’t communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.”
I’ve looked at different forums but most of them seems to reinstall the driver and things work which is not my case.
Does someone encounter the same issue ?
If you need more info on my config or previous installation let me know.
Hi everyone, I’ve recently gotten into the fastai course and decided to make my local setup. Since I was successful (largely due to this guide), I wanted to share my two cents on how I achieved it as of 17/02/2023. My hardware is a desktop PC with an Intel Core i9-12900K and an Nvidia GeForce RTX 3060.
Fastai_setup_Ubuntu_server_2004.pdf (1.3 MB)
Ubuntu_Server_2004_Setup.pdf (574.8 KB)
Conda_Environment_Dependencies.pdf (523.5 KB)
Unfortunately, can’t get it to work using these instructions in October 2023. Nvidia installs later versions (525 and Cuda12) even when instructed to install specific earlier versions - I suspect this is the main problem causing training to revert to CPU. Also, certain conda channels no longer exist so some commands don’t work. If you experienced with ML and Linux no doubt you could get this working but after 3 days of trying for WSL and full Ubuntu no joy.