Deep Learning Server at home

[parthi2929 (Parthiban)]

Hi

Both me and my wife are working on Software, and I want to learn AI via fast ai. I learnt online we have players like vast.ai. I initially thought of buying a medium deep learning desktop computer, but now thinking if we could set up one as server at home, me and my wife both could use that powerful rig for our high demand jobs, from our thin client laptops.

Any one has tried similar setup at home? If so please share your setup details both Hw and Sw. I could not find these online.

[meanpenguin (Edward)]

Hi,

Search for “Rig” on this forum for various different builds

This is a long running thread on building a ig

Recommendations on new 2 x RTX 3090 setup

If you want to use two 3090, you basically have the following options: Get a motherboard with four full-length pcie 16x (2-slot spacing). In such a way, you can install 2 3090FE with one slot separation between them, without interfering with the front-panel connections. Of course the upper card will suckle very hot air from the lower one. Don’t know if you’ll incur in thermal issues. Get any motherboard you like, and a case with 3-slot vertical mounting (e.g. the phanteks P600S), and a PCI…

A very detailed analysis of GPUs

The Best GPUs for Deep Learning in 2020 — An In-depth Analysis

Hope this helps.

[parthi2929 (Parthiban)]

Thank you, yes I read it almost fully.

From HW perspective, its clear which GPU to target for initially (RTX 30xx). What I am confused about, is CPU. It looks like AMD does not support Ubuntu officially for some CPUs. I was thinking of APUs initially like Ryzen 4x50G with integrated graphics for a local server as a test setup, but then I read a lot of complaints about highly unstable Ubuntu on these machines. If ubuntu, we are better off with intel chips?

For SW, I will have to figure out how to host jupyter notebook in the server, and then access from any laptop in same LAN

[parthi2929 (Parthiban)]

The more I dig, the more bizarre things are getting.

• Considering Performance/cost it was no brainer to choose an AMD Ryzen Zen3 CPU over any Intel CPUs for now (GPU being a RTX 30XX)
• I learn AMD APUs 4x50G and X300/A300 chipset officially support Win 10. Check 1, 2. Do not know what issues this setup (Ubuntu + AMD APU) would bring home.
• Then I find, the ML stacks may be discriminative on CPU. For example, Anaconda using MKL optimized for Intel being one of many cases, I do not know how much of a rabbit hole is awaiting, going with AMD CPUs.

I almost made up my mind for AMD CPUs for any of my rigs but I think I slowly start to see why they are not widespread despite strong VFM. How many of you are successfully using your own rig made up of a AMD CPU + Nvidia GPUs + Ubuntu? And AMD APU + Ubuntu for any of your local server/mini pc solutions?

[florianl (Florian)]

Hi … i had to make the same decision and here are my 2 cents:

1. 1-2 GPUs and limited budget -> AMD Ryzen Zen3
2. 2+ GPUs and limited budget -> Intel i9
3. 2+ GPUs no budget limit -> AMD Threadripper

You are totally right … that’s a rabbit hole (could add motherboard, RAM channels, PICe lanes, Temp sensor support on linux, etc …). But generally the CPU will not be the performance bottleneck. The only hard constraint imho for Ryzen Zen3 are the mainboards and PCIe lanes which limit them to 2 GPUs in my opinion.

Florian