Personal DL box

Still no luck.

sorry I’m out of ideas

Maybe add the --force flag to conda install bcolz?

I did the same gpu load check as you yesterday (just bought a 1080 ti after reviewing all the notes on this thread again). I’m still really happy with my 6 core Ryzen, but I’d pay up for a Threadripper on my next build (supports way more ram, NVMe ssds, gpus).

IMHO you built in a good way. Having a solid GPU and not having to pay the hourly cloud rates for that - esp given how long it takes to train models as we progress, is awesome. Its comparatively cheap spin up a bunch of CPUs with reasonable ram to throw at shorter running ml models (and the aws credits will go a long way).

1 Like

try
conda env list

That will show the environments on your system.

Looks kind of weird that your environment is named fastai/

1 Like

Thanks a lot @rob @radek and Jeremy. Finally, it works.

Alternatively, pip install fastai is available - thread

6 Likes

Glad to hear it. What was the solution?

@radek, probably you already know but just in case, if I see GPU utilization is low and GPU memory is under say 50% I will increase batch size and usually means much more optimized training.

About CPU, I agree that even if not the number one bottleneck in DL it will pay to have a good enough one, I had for some months this year a dedicated server with Ryzen(8cores, 16 threads) and really happy about its performance, good benchmarkings as far as I know.

And, lastly, I think CPU RAM to be the bottleneck many times because data wrangling requires a lot (and inversely correlated with your coding efficiency). I have 32 GB laptop locally, minimum 64GB on rented servers… and I always find myself in situations where I will need more. For a personal DL box, that I dont have, I wouldn’t have less than 128GB if possible,

Kind of beefy specs but, well Christmas is near already! :grinning:

The nice thing about RAM is you can upgrade it after the fact. So as long as you are mindful of wanting 128GB at some point, you can build a system with that in mind and actually just put like 32GB in with the intention to add the rest later. Just make sure not to use like 8X4GB sticks if you want to bump it up later.

1 Like

I remove all things and re-install it again and again. Once, it was working, I stopped. :sweat_smile:

5 Likes

Thats, obviously, an explanation of a day !!! :rose: :joy:

2 Likes

@beacrett What is your full DL box setup?

Currently:

  • Ryzen 5 1600 (6 core, can over clock if desired, comes with cooler)
  • AM4 B350 chipset motherboard (B350 is the middle tier of their chipsets - it is worth it for the small price bump)
  • 16GB DDR4 3200 RAM (fastest supported by my processor, going to get another 16GB)
  • 250GB NVMe M.2 SSD
  • 2TB HDD
  • 1080 ti (EVGA GeForce GTX 1080 Ti SC Black Edition - very happy with this so far - great cooling)
  • 750 watt modular power supply (would need to get a bigger one if adding a second GPU)
  • Dual boot Windows 10 / Ubuntu 16.04 LTS

imho, its worth getting the fastest ram supported by your CPU (within cost reason). Keep track of the model and its timings - you may need to manually change settings in the bios to ensure it is running at full speed and you want any new ram you buy to match the speed and timings for optimal performance (try to order the same model to keep it simple)

5 Likes

There is no turning back :slight_smile: Delivered today, “feeling like a little kid” :nerd_face:

17 Likes

Has anybody tried to activate conda virtualenv and run jupyter notebook from within crontab job? source activate does not work for me, source bin/activate throws me to root user and does not activate anything. Nothing useful in google forest so far.

I haven’t, but maybe you can try running a script as a login shell,

#!/bin/bash -l
cd fastai
source activate fastai 2>/dev/null &
nohup jupyter-notebook 2>/dev/null &

That has usually solved my “this isn’t working in cron” woes in the past

1 Like

Thanks @rob, this is what worked for me link

UPDT: no it did not work as well. Cant activate conda env from cron.

1 Like

I tried setting up Deep Learning Machine on Azure for fast.ai and it’s working fine.

During the setup, I faced an issue which seems to be an issue with Jupyter.

The issue is while following the steps in readme, even after creating the environment and activating the fastai environment I am not able to find the actual kernel for fastai.

I have browsed and many people faced similar issues with jupyter and conda .

I have resolved by manually installing kernel after activating fast ai environment.

python -m ipykernel install --user --name fastai --display-name “Python (fastai)”

Please let me know if anyone else faced the same issue.

1 Like

Hello, I have used paperspace script with a fresh install ubuntu 16.04 with a 1070 everything was ok untill I tried to run learning cell “resnet34”. I monitored my system and it uses all the RAM until the point where kernel shutdowns itself (8G RAM) and 1G Swap.

But it doesnt use VRAM at all I checked it with nvidia-smi tool. I’m guessing it doesnt use GPU at all? But still shouldnt 8G ram should be enough? I also added the kernel with the command posted and switched kernel to that stil no luck.

So what can be the problem?

Adding the notebook and some screenshots before and after might help other forum members to respond otherwise it’s like throwing an arrow in dark…
Thanks …