Importance of system CPU for DL workstation?

Deep Learning
1

Hi.

At my new job I can configure a deep learning box for on-site model training (max. price ca. 8k Euro). I will start out with a RTX Titan (but will likely extend this build to a second titan in 1-2 years). Other components will be 128GB RAM and NVM SSDs (280GB system, 1.2TBdata).

Now, I’m undecided about the best CPU setup for the machine. I will also do ‘traditional’ data crunching and general compute on this machine…

Would one CPU like the Intel E5 2650 v4 be good enough for this system? Or should I go for a lower end Xeon SP (possibly in a 2xCPU config, silver range)? I’m not sure if I’d benefit of the fancy new AVX512 instructions in any way? Also, the lower (say silver) SP range has pretty low clock speeds… not sure if it’s better to go 8-core with higher clocks?
The Xeon SP range gets really expensive real fast (esp. in 2xCPU) config…

Cheers,
Christian

2

I can’t comment regarding these specific cpu models, but from my experience in some tasks cpu might be a bottleneck - activity detection is one of them. Personally, I’m waiting for the release of Zen2 based Ryzens and Threadreapers from AMD - they should be announced on 27th of May.

3

Thanks for the input. I was wondering about AMD systems, too. The commercial builders here do not offer AND + Nvidia GPUs in their systems for some reason?

4

As intel released i9 version of it’s cpu’s their performance is much better, than the xeon one’s.

I agree that AMD cpu’s are good as they bring more PCIe lanes, but tech enthusiasts suggest using intel cpu’s as intel still hold the position for the most powerful ones in the market.

Ideally, more cores can work for you but seeing that i9 cpu’s are already near 18 core (plus hyperthreading), I think going for i9-79xx series or i9-99xx series is better as the clock speeds of these cpu’s is way more than the zeon one’s and they can also be overclocked. In most of the cases, more clock speed helps and adding more cores may not be much beneficial as the time to communicate between the processors increases.

5

@kushaj Thanks, this was my gut feeling too… Won’t go into overclocking and will buy from a system dealer. Was wondering of the new instruction sets AVX2 and esp. AVX512 would benefit me in any way. I think those are reserved for the high end lines (Xeon SP/ Xeon SP2)

6

We don’t have to deal with the instructions our self, pytorch has to handle it. Look at this link. It shows that pytorch can use AVX2 instructions.

7

Thanks again. Will check out the link. I also learned that AVX512 is included in the Xeon-W and latest i7 and i9 cpus…
I’m currently looking to get the Xeon W 2145 8-core CPU for my machine

8

I have corei9-9900k which has 8 cores (x2 HT = 16 logicl cores) with 3x1080ti connected
and I can say a lot of the time the CPU is the bottleneck… That is usually with heavy augmentation… So definitely CPU is an important part to consider…

I have looked into AVX512, and whether it is useful or not… It seems not so important to consider, since using it will increase the CPU temp. and will lead to thermal throttle…

So I have ignored whether AVX512 available or not when I built my system…

Here are the articles that turned me off about considering AVX512:
By how much does AVX-512 slow down your CPU? A first experiment.

Table 15-16 in Intel’s optimization manual describes the impact of the various instructions you use on “Turbo Boost” (one of Intel’s frequency scaling technology). The type of instructions you use determines the “license” you are in. If you avoid AVX-512 and heavy AVX2 instructions (floating-point instructions and multiplications), you get the best boost. If you use light AVX-512 instructions or heavy AVX2 instructions, you get less of a boost… and you get the worst results with heavy AVX-512 instructions

AVX-512: when and how to use these new instructions
However, some of these instructions use a lot of power and generate a lot of heat. To keep power usage within bounds, Intel reduces the frequency of the cores dynamically. This frequency reduction (throttling) happens in any case when the processor uses too much power or becomes too hot. However, there are also deterministic frequency reductions based specifically on which instructions you use and on how many cores are active (downclocking). Indeed, when any 512-bit instruction is used, there is a moderate reduction in speed, and if a core uses the heaviest of these instructions in a sustained way, the core may run much slower. Furthermore, the slowdown is usually worse when more cores use these new instructions. In the worst case, you might be running at half the advertised frequency and thus your whole application could run slower. On this basis, some engineers have recommended that we disable AVX-512 instructions default on our servers .