try
conda env list
That will show the environments on your system.
Looks kind of weird that your environment is named fastai/
try
conda env list
That will show the environments on your system.
Looks kind of weird that your environment is named fastai/
Thanks a lot @rob @radek and Jeremy. Finally, it works.
Alternatively, pip install fastai
is available - thread
Glad to hear it. What was the solution?
@radek, probably you already know but just in case, if I see GPU utilization is low and GPU memory is under say 50% I will increase batch size and usually means much more optimized training.
About CPU, I agree that even if not the number one bottleneck in DL it will pay to have a good enough one, I had for some months this year a dedicated server with Ryzen(8cores, 16 threads) and really happy about its performance, good benchmarkings as far as I know.
And, lastly, I think CPU RAM to be the bottleneck many times because data wrangling requires a lot (and inversely correlated with your coding efficiency). I have 32 GB laptop locally, minimum 64GB on rented servers⌠and I always find myself in situations where I will need more. For a personal DL box, that I dont have, I wouldnât have less than 128GB if possible,
Kind of beefy specs but, well Christmas is near already!
The nice thing about RAM is you can upgrade it after the fact. So as long as you are mindful of wanting 128GB at some point, you can build a system with that in mind and actually just put like 32GB in with the intention to add the rest later. Just make sure not to use like 8X4GB sticks if you want to bump it up later.
I remove all things and re-install it again and again. Once, it was working, I stopped.
Thats, obviously, an explanation of a day !!!
Currently:
imho, its worth getting the fastest ram supported by your CPU (within cost reason). Keep track of the model and its timings - you may need to manually change settings in the bios to ensure it is running at full speed and you want any new ram you buy to match the speed and timings for optimal performance (try to order the same model to keep it simple)
Has anybody tried to activate conda virtualenv
and run jupyter notebook
from within crontab
job? source activate
does not work for me, source bin/activate
throws me to root user and does not activate anything. Nothing useful in google forest so far.
I havenât, but maybe you can try running a script as a login shell,
#!/bin/bash -l
cd fastai
source activate fastai 2>/dev/null &
nohup jupyter-notebook 2>/dev/null &
That has usually solved my âthis isnât working in cronâ woes in the past
Thanks @rob, this is what worked for me link
UPDT: no it did not work as well. Cant activate conda env from cron.
I tried setting up Deep Learning Machine on Azure for fast.ai and itâs working fine.
During the setup, I faced an issue which seems to be an issue with Jupyter.
The issue is while following the steps in readme, even after creating the environment and activating the fastai environment I am not able to find the actual kernel for fastai.
I have browsed and many people faced similar issues with jupyter and conda .
I have resolved by manually installing kernel after activating fast ai environment.
python -m ipykernel install --user --name fastai --display-name âPython (fastai)â
Please let me know if anyone else faced the same issue.
Hello, I have used paperspace script with a fresh install ubuntu 16.04 with a 1070 everything was ok untill I tried to run learning cell âresnet34â. I monitored my system and it uses all the RAM until the point where kernel shutdowns itself (8G RAM) and 1G Swap.
But it doesnt use VRAM at all I checked it with nvidia-smi tool. Iâm guessing it doesnt use GPU at all? But still shouldnt 8G ram should be enough? I also added the kernel with the command posted and switched kernel to that stil no luck.
So what can be the problem?
Adding the notebook and some screenshots before and after might help other forum members to respond otherwise itâs like throwing an arrow in darkâŚ
Thanks âŚ
@lymitshn I wouldnât suggest using your own box for learning this course - better to use the fast.ai AMI on AWS, Paperspace, or Crestle. Once youâre comfortable with the basic techniques then you can come back to getting your own machine working. Youâll know enough at that point to understand how to debug your issues and ask for help in a way that we can be useful to you.
Thank you for suggestion but I really want to run my local machine.
I created a new env and verified and torch uses GPU also added 6G swap and it seemed to work this time. Ran until hitting %12 (slowlyâŚ) but it was using only 800 MB VRAM and %16 GPU Power at peak and after consumed all DRAM and swap, kernel restarted itself.
It can clearly access GPU but still tries to use high DRAM is this how the model supposed to work? Or is something wrong with my setup?
Hi! I ran into the same issue. solution was just given on Wiki: Lesson 1. It was not a setup issue (at least in my case), but reducing the number of workers was necessary for loading/transforming the data as this part is done on CPU/main RAM:
data = ImageClassifierData.from_paths(......, num_workers=1)