Building my own Deep Learning Box: Intel vs AMD

I am considering building my own Deep Learning box - is there a recommended guide by Fast.ai? Does it matter if I get an Intel CPU or an AMD?

Thanks
Ali

1 Like

some of the python libraries around linear algebra and others are tailored to take advantage of intel processors. I would do a search to see if any have been optimized in the past few years for AMD. if so, I wouldn’t mind saving a few bucks going the AMD route. I currently use Intel, but would switch if the situation warranted and I planned on using the box for other server type activity.

2 Likes

Hi Ali,

Found this interesting article hope this helps --> https://medium.com/the-mission/how-to-build-the-perfect-deep-learning-computer-and-save-thousands-of-dollars-9ec3b2eb4ce2

" As of December 2019, AMD offers more performance for less money. AMD’s 1920X with 12 Cores/24 Threads/60 PCIe lanes is only $199! Intel’s 7900X with 10 Cores/20 Threads/44 PCIe lanes is $1000. Benchmarks show comparable performance, so AMD seems like a no-brainer.14 Jan 2019"

Building Deep Learning Box ? are you planning to build Production level setup or Developer level setup ?

For Production Level setup:
Motherboard: ASUS WS X299 Sage
Memory: 64GB
CPU: Intel Core i9-9820X (10 Cores at 4.1 GHz)
GPUs: 4x NVIDIA GeForce GTX 2080Ti “Turing” 11GB (44GB total)
Storage: 3x 3TB Raid5 (6TB available), 1x 1TB SSD and 1x 1TB M.2
Power supply: 1600W

For Developer Level setup:
Motherboard: Gigabyte X470 Aorus Ultra
Memory: 32GB
CPU: AMD Ryzen 7 3700X (8 Cores at 3.6 GHz)
GPUs: 2x NVIDIA GeForce GTX 2080Ti “Turing” 11GB (22GB total)
Storage: 2x 3TB Raid1 (3TB available), 1x 500GB SSD and 1x 500GB M.2
Power supply: 750W

Software:

  • OS: Ubuntu 18.04 LTS
  • CUDA Toolkit: 10.1
  • CuDNN: 7.6
  • NCCL: 2.6
  • TensorRT: 7.0
  • Anaconda: 2020.x

You can also Overclocking - Most gaming motherboards used for DL workstations make it easy to overclock. Intel MKL is fairly well-supported in DL with the major frameworks (Caffe2, CNTK, MATLAB, MXNet, TensorFlow, and even Theano) CUDA/CuDNN is by far the major framework for hardware accelerated deep learning.

Nvidia reveals why it chose rival AMD over Intel for its deep learning system

PCIe 4.0 support had a lot to do with it and Zen 2-based Epyc 7742 CPUs (https://www.techspot.com/news/81350-amd-unveils-2nd-gen-epyc-cpus-world-fastest.html) .

Epyc 7742 CPUs

For CPU you can choose AMD but for GPU Nvidia.

Resource:

  1. https://rocmdocs.amd.com/en/latest/
4 Likes

Hi Ali, I have always liked Tim Dettmers’ pages on hardware and GPU selection, and he even mentions Fastai in the latter article:


Like one poster above, I have also heard that the optimizations are better for Intel, not sure if that still holds or not. Also, there is a wide range of budgets. For example, I was able to build a system with an i5-8400, 16GB of RAM and a 6GB 2060 card for about $1000. It is nowhere near as speedy as a $3000+ setup but it is fine for exploring.

Caveat: It seems like the recent trend is to go for 8+ GB on the GPU. I’m not particularly far into my deep learning journey, so I’m just sticking with my current card until I encounter something that I really want to run, but FP16 training, smaller batch sizes, etc. absolutely will not help.

1 Like

Unless you plan on building a server farm, you probably can’t notice the difference. The bulk of the workhorse will be in the GPU. I went with AMD a year ago because it was the best bang for the buck (which would probably only be a ~100$ difference). Someone mentioned a review above.

2 Likes

I just built a DL workstation on AMD Ryzen 7 3700x and an RTX 2070 Super and can safely say that all DL libraries work flawlessly.

The direct alternate is an Intel i7 9700k which is considerably more expensive for lesser threads. Intel chips are maybe optimized better for gaming, but going for an AMD CPU gives you great performance and value for money.

3 Likes

Hi
I recently bought a PC to do deep learning. After several hours, I found those articles the most useful.
They are perfect for 4 GPU pc. If you want 2 GPU, then buy a Ryzen 3300X or 3600 + a motherboard that supports PCIe 4 for both GPUs as PCIe 4 x 8 == PCI 3 x 16 :smiley: . If not, buy Ryzen 3300X or Ryzen 3100 or some intel xxxxF core (those that haven’t gpu). Others things, you can figure out reading the articles above.

Here, my two configs (in spanish): 2200€ (for dual GPU, high budget) and 870€ (for single GPU + 2060 super, low budget). For 1000€, you replace 2060 super for 2070 super and maybe, tweak the PSU (power supply). Finally, some budget configurations in English. I didn’t follow it.

3 Likes

Thanks for a fantastic answer and the links! I am looking to build a beginner box to take part in Kaggle competitions before upgrading to a Developer level so the links will definitely help towards the research. Any links/advice for beginner box?

1 Like

Thanks for the links! I was thinking of using an NVDIA 2070 to get started and then to upgrade to a higher card once i am comfortable with Kaggle. Would you recommend a Ubuntu as the base operating system or Windows with Ubuntu running in Virtual Box?

1 Like

2070 should be solid, and matches Tim’s value recommendation (although he says later in the comments that the 2060 Super is similar). I have only run Ubuntu by itself, not virtually in Windows. Hopefully someone who has experience with the latter can comment on the pros and cons?

Also, it looks like we’re getting some thumbs up on the Ryzen chips, which is encouraging to hear. I wonder if we’ll soon see the day when AMD GPU’s also become a viable option.

It really depends on what you want your hard earned $$$ invested in hardware doing. In the past 3 years I have done it all with respect to trying to get the machine to be a DL rig. The problem with most virtualization options is that you cannot get the GPU to pass thru correctly for CUDA to run, in particular on Nvidia “gaming/consumer” cards.

In the early days of fastai, I ran exclusively on windows, but it sure was not perfect. I then went to dual boot on the machine with Ubuntu installed next to win10. But i ended up with issues with the machine going to sleep, waking up, random reboots and I just got tired of it. I then went to Ubuntu exclusively and ditched windows as fastai matured, but the windows libraries did not. I then tried the virtualization route, and had some success with GPU passthru in UNRAID, but I could only get one card at a time in the VM. Never could get docker to work quite right in UNRAID for DL either. So I went back to Ubuntu alone. Problem with that was, I still liked windows as my daily driver, and lots of $$$ just sitting idle if I wasn’t doing DL. Now, in the past few days, you can get GPU passthru within windows using WSL2. See my post here:


Again, it is not perfect, but it is a step in the right direction for the windows loving folk. Maybe by the time you get all your hardware it will even be more mature and stable.

1 Like

Hi Gilaniali,

I highly recommend using Linux systems stable version for any AI Projects in my opinion windows is slow eats up lot of memory by itself. I have a dual boot laptop with windows 10 (2004ver) with WSL2 and Ubuntu 20 both configured with AI setup. I use both windows and Linux but Linux makes your life easy for AI projects.

For Beginner (Basic) Level Setup. (Laptop)
Hardware:
a) 1 TB SAS for operating system
b) 2 TB SATA for DATA
c) Dedicated for AI Nvidia Graphics card of your budget choice / There are plug-able external Graphics cards as-well
d) Intel Graphics card for operating system (any)
Try to keep sync RAM and GPU memory (16gb ram / 16 gb gpu) preferably.

Software:
a) Ubuntu 18 if you are looking for stable version or Ubuntu 20 also good currently I am using It.
b) Install CUDA 10.1 with latest Nvidia drivers
c) Install Cudnn Library specific to cuda 10.1
d) Pytorch choose cuda 10.1
e) Fast ai ( I am in love with this library as no other library I have seen which utilities GPU fully . It always runs at 100% utilization which is amazing.) thanks to @jeremy for this wonderful library.

For Beginner (Advance):
Above setup as per budget you can upgrade high end configuration desktop.

You can look at NVIDIA DGX Station this will be good start.
or
Configure your own with Ubuntu 18 desktop with Nvidia V100 GPU

If you are looking for Ryzen chips then choose Ryzen Threadripper 1950X

Refer this link for pricing and further details
[25][9]

Hope this helps.

2 Likes

I use Ubuntu as main OS system and use Windows for games. I just installed Ubuntu 20.04 and it’s beautiful. I don’t tried WSL2. So, my personal recommendation is to use Ubuntu 20.04 (you’ll have soport for 5 years from now) for DL and your favorite OS for anything else. Also, you should use conda where you can install cuda there. So, you only need close nvidia driver (not cuda toolkit installed on ubuntu) to run.

Hii 2 gilaniali

If you’re looking to build your own Deep Learning box, Fast.ai does have a recommended guide that you can follow. The guide is available on their website and provides detailed instructions on how to build a high-performance Deep Learning workstation that can handle a wide range of machine learning tasks.

Regarding the choice between an Intel CPU or an AMD, both have their advantages and disadvantages. Intel CPUs generally offer higher clock speeds and better single-core performance, which can be beneficial for some Deep Learning workloads. On the other hand, AMD CPUs offer more cores and threads, which can be advantageous for tasks that can be parallelized across multiple cores.

Ultimately, the choice between Intel and AMD will depend on your specific needs and budget. It’s important to do your research and compare the performance of different CPUs before making a decision.

One thing to keep in mind is that Deep Learning workloads are often GPU-bound, which means that the choice of GPU may be more important than the CPU. NVIDIA GPUs are generally the preferred choice for Deep Learning workloads, as they are well-supported by popular Deep Learning frameworks like TensorFlow and PyTorch.

Or other if you want to go with intel, you can go through with this Intel Evo Laptops: What is Intel Evo Platform? | Lenovo India

In summary, if you’re looking to build a Deep Learning box, Fast.ai’s guide is a great resource to get started. When it comes to choosing between an Intel CPU or an AMD, it’s important to consider your specific needs and budget. However, for Deep Learning workloads, the choice of GPU may be more important than the CPU, with NVIDIA GPUs being the preferred choice.

How’s the situation in 2023? Is Intel still the safer choice?

I think in practice it does matter. The GPU is much more important than the CPU with Deep Learning