For those that have made it this far, we recommend building your own box, if you can afford to. For a bit over US$500 you can build something that should be faster than an AWS P2 or Google Colab server. For $1500-$2000 you can build something that’s about as good as a high-end Paperspace server.
This is a wiki thread. Please add resources that are reasonably up to date, and that you have personally read and found helpful. (If you see resources listed here that have incorrect advice, please add a note next to them mentioning those issues.)
Which GPU(s) to get for Deep Learning: Tim Dettmers published a nice cost-benefit analysis of different kinds of GPU appropriate for your goals and budget (from GTX 1050 for when you’re learning, all the way to the GTX Titan/1080Ti for research). Although it hasn’t been updated for nearly a year, there’s been no major changes to the Nvidia consumer cards since then (and no known changes on the immediate horizon)
I built a dedicated box earlier this year with a Titan XP I ordered from nvidia (uk) which was a good deal but a 1080 would have been just fine. The last one I built was in the late 90’s. Pc Part Finder was a great site to detect incompatibilities. I built it in an afternoon, but it took a couple of sessions to decide on parts. Part availability was a significant factor in choice.
The various blogs give good info, generally. What can I add beyond these blogs? It’s not difficult and you don’t need to be overly technical, it is essentially fiddly assembly. A bigger case is easier to work inside, if you have the room to put it, and may stay cooler. Decide how much you care about noise and understand the quietness of your case and any fans/coolers you buy. I ended up with BeQuiet parts. Think about buying a power supply with enough wattage and motherboard that can take another GPU later. You can add these on pc part picker to check future capacity. Consider keeping your OS on a separate disk to your data. Watch some videos on YouTube/Vimeo for your case/mobo if you can find them. I found watching the videos invaluable for fiddly bits - eg I reversed the side of my mobo and would never have worked that out on my own or with a mere pdf manual.
The most difficult part for me was having to find/buy a couple of adapters to hook up an old screen to do the initial config, and then working out on Ubuntu forums how to configure WiFi, and how to put my user directory on a 2nd disk. All plain sailing from there. Since this is a wiki I’ll see if I can find helpful links for those.
CPU: i7 7820X (LGA2066)
CPU cooler: NZXT Kraken X52
Mboard: MSI X299 TOMAHAWK AC ATX LGA2066 (ASRock X299 Taichi gives 2x16 1x8 for 44 lane cpu, was a tough couce between these…)
RAM: Corsair - Vengeance LPX 32GB (2 x 16GB) C16 DDR4-2666 Memory
PSU: Corsair RM1000i
HD: WD Gold WD1005FBYZ 3.5” 1 TB
SSD: Samsung 1TB
GPU: Gtx 1080 ti
Case: Dark Base Pro 900
I tried to put a gtx1070 (from another machine) in with the 1080ti. Right next to each other the 1070 was hitting 90 degrees and 1080ti hitting 85 degrees before I shut down and pulled out the 1070.
Plan now is to water cool the 2xGPU’s in one box, the main worry being that I get a water leak and destroy some very expensive equipment. (reservoir in 5.25" drive bay looks like best option to conserve space)
I found that there isn’t that much info on multi GPU deep learning builds - how to keep cool with just airflow, which are best motherboards, cases, how to watercool in ATI mini towers when you don’t want some fancy gaming box with lights everywhere and giant coloured reservoirs.
After inspiration from @tensoralex below I looked into cooling options with submersible pumps and found the following from AquaTuning a good value option:
NexXxoS ST30 420mm radiator
Alphacool Double Power Pumpstation
2x Eheim 1000 pumps (run in parallel with one at say 50% for redundancy).
I use my own box with GTX 970 (built it last year). I am wondering if it is sufficient (say for Part 2 Fast AI for now) or I should upgrade to GTX 1080 ti. (luckily my power supply is work if I do decide to upgrade )
Does anyone have good advice on how to set up two disks (one fast SSD, and a bigger, slower HDD) in Ubuntu? I put off the issue when I first built my machine, but I’m getting to the point where all the datasets for part 1 and part 2 don’t fit on just the SSD anymore.