Here’s the results of the poll:
https://andrestorrubia.typeform.com/report/iw1Rhl/kxJG0FnuRRfkC8Xr
4 Likes
I think what you’re wanting is:
- Solve cooling problem / transfer the “guts” of your current computer to a new case to fix cooling
- Add a second GPU
- Can you do this yourself
Let me know if this is not the case.
Your current case certainly isn’t the best one I’ve seen for airflow but I’d be a little surprised if you couldn’t get it working to acceptable levels. How many case fans do you have and which way are each of them blowing? Linus Tech Tips on youtube has several videos on case fan testing. I would start with watching those. I would have expected a PC builder to set this up right but maybe they didn’t. Are there more case fan mounting slots that you could add more fans? I couldn’t find what was included by default on your current case.
As for mounting multiple GPU’s, your motherboard can handle multiple GPU’s. Ideally you want space by where your GPU fans are to allow for proper airflow. I have seen plenty of builds where they have blocked the GPU fans but I when I did that with my GPU’s, i was not happy with the temperature results. Which 1080ti do you have? There are many variants from different manufacturers. I need more information to help answer this.
As for the question: Can you do this upgrade yourself? There are a ton of great online resources about building your own machine. Youtube would be a good resource. I personally like Linus Tech Tips on youtube and he has several building a pc guides. I believe you already have your PC built so it should be much easier to know how it goes together as you’ll have to take it apart first. I found it pretty straight forward to build my own PC after watching Youtube videos. If you’re not comfortable with doing it yourself than I would expect there is probably a local PC shop that could do the work for you at a reasonable cost. A lot of PC gamers build their own PC. If you have a gamer friend that has built their own computer you could ask them to help you out. I probably dedicated about 20-30 hours of my time building my box from research on what parts to buy, learning how to do the build and then actually building it and getting all of the software set up. Building a PC is not something you’ll be able to do in an afternoon if you’ve never done it before. I started off with a basic understanding of what all of the different components function was and how they worked together. My total experience with building PC’s prior to building my own rig for fast.ai was watching a friend build one over 10 years ago. I believe it is certainly something you can do yourself as long as you are committed to spending the time learning how to do it properly.
A few notes if you decide on doing this yourself:
- Pay attention to cable management. It does not need to be perfect by any means but if you put some thought and effort into it, it will help with cooling and maintainability.
- Pay attention to mounting your CPU to your motherboard and cooler to your CPU if you end up having to take that apart. I don’t think you should have to take that apart to get your motherboard out of your case, but if you do that is probably the most delicate thing you’ll have to do. Once you’ve watched enough guides on how to do it, it’s pretty straight forward. If you don’t do it right and don’t handle the parts correctly it wouldn’t be hard to permanently damage them.
- Sometimes you have to push a lot harder than you’d expect to get connectors, ram and video cards plugged in. If it doesn’t go in easily, triple check that you’re doing it right and then just push harder. Most things are designed so they can only be installed one way. For example, there is a notch in your ram that won’t let you install it backwards. That’s what I’m talking about when I say “triple check” you’re doing it right.
- Note the order in which the guides tell you to install things. This will make your life easier.
- Read the instructions. You probably don’t have the instructions readily available as your PC was built by someone else, but you should be able to find them online pretty easily from the manufacturer with a little bit of Googling. Some people prefer to jump in to things without reading instructions, but I would not recommend that with building a computer for your first time.
Don’t worry about the fact that your GPU’s are different and are not SLI compatible. This is only needed for gaming. If you had 2x2080ti’s you could set up NVLink between them which I read helps with training on multiple GPU’s simultaneously, but you will be just fine without that.
Here are some helpful links that I would suggest you read:
http://www.fast.ai/2017/11/16/what-you-need/
Hopefully that helps!
2 Likes
My organization is planning to build a deep learning rig as its more cost efficient as compared to already available builds in market like 1 and 2 i have suggested them the Gold/Silver/Bronze buckets. Need To know all of your views.After looking at the comments i am thinking to add 2080ti in my gold plan.Just little bit skeptical about the compatibility issues with the components. @init_27, @antorsae , @matdmiller need to know all of your views.
If anyone can post the pcpartpicker link of a multi GPU setup that they have build out then it would be helpful. I am bounded with a budget constraint of 5000$-6000$
This is the Pcpartpicker link : https:/pcpartpicker.com/list/fJkTtg
Some comments on your lists:
- Jeremy has suggested using Intel CPU’s because of deep learning library optimizations for Intel CPU’s.
- 1,600W power supply is overkill for 2 GPU’s. 1,000W is enough
- I’ve talked a lot about GPU airflow in other posts and your build does not address the issues of blocking the fans. Please read through those posts. FE cards get a lot hotter if the fans are blocked.
- I would get a full tower case as it will be easier to work in and probably no more expensive. Make sure it has plenty of fans that come with or at least has the ability for you to add more fans if it doesn’t come with many.
- Make sure if you are using water cooling / radiators that your case can fit them.
Here is a link to my build: https://pcpartpicker.com/list/N6mM6s
This is what I would do differently:
- I would have gone for an i9 CPU and corresponding i9 compatible motherboard as I am bottlenecked at times with my current CPU when working with image data on multiple GPU’s.
- The case I picked functionally works well but it is gigantic and I’m sure there are other more compact cases that would be better. It works well, it just takes up a lot of room.
- 3 GPU’s instead of 4. I haven’t run into a ton of cases, with the exception of some kaggle competitions where I use my 3rd 1080ti. If you are going to use this with a monitor I would get a low powered GPU to drive it. If you are going to use this as a server, then it’s not necessary.
- Bigger NVME drive. I have hard drives for my cold storage, but it takes some maintenance to move data around and is something else for me to do. If I had a bigger NVME drive i wouldn’t do this as much. If you aren’t working with big image datasets than your drive size will likely not matter nearly as much.
- Only 1 HDD. I was originally running my machine in Unraid and I decided to move away from that and I don’t really need the second HDD now.
- I would definitely get a 2080ti, but it was not available when I built my machine.
4 Likes
Can you point me where Jeremy recommended Intel CPUs? This may be the case when training on CPU, but if you’re using GPU for training I see it was a moot point.
From personal experience I can vouch for the Threadripper (I have the 16-core/32-threads one) and it becomes very handy when doing a lot of on-the-fly augmentations, second that with 64 PCIe lanes and 128 Gb ram max.
If you’re not going the Threadripper route many intel mobos as well as AMD ones (provided you have one processor ending in -G) have support for on-board graphics in which case you can use the onboard card for display; although personally I run mine w/ 2 x 2080 Tis and when Im not using it I do sudo service lightdm stop which will kill the graphic desktop and free all GPU memory for deeplearning; but even if you don’t do this I can still watch 4K movies with that computer while it’s training on both GPUs. 
1 Like
2 Likes
Thanks. Only comment is that if your workload is multi-thread friendly you’d likely find multiple core/threads provide much gains vs. what optimization would do (for DL or scientific programming I’d go with more cores rather than X% faster single-core performance).
1 Like
We’ve often found CPU performance the restriction on model training time, due to data augmentation and jpeg decoding. With tensor cores this is quite a big issue.
3 Likes
Hi @matdmiller - thank you again for a great reply, much appreciated.
FYI - I have the Zotac Geforce GTX 1080 ti, like this with two fans on it. Looking inside my case I only appear to have two case fans (one front which is intake I think and one exhaust at the back) but there are attachment points for two at the side, two at the top and one on the bottom. However it’s already looking a little crowded in there so I’m not sure how much difference more fans would make?
To try to keep this simple (and not take much more of your time) I think we can boil it to the following:
- Do you think adding a second GPU and more case fans could work for this case/set-up? I know it gets a little complicated in terms of wanting positive pressure etc… when trying to get the best airflow. This is something I know little about.
Depending on your response to this I will go away and do as much research as I can before making a call on doing it myself or taking it to a pc shop.
Thanks again and thank you for the links you provided,
Mark
I agree that multi-core would in many cases out perform library optimizations alone, however with a high core count i9 you can have both. I suggested an i9 vs another intel CPU for being able to drive 2x1080ti’s + a 2080ti like he said he wanted in his “Gold” option. I’ve personally run into bottlenecking with my current i7 CPU and I would have gone with an i9 if i knew then what I know now. My i7 is far from top of the line as far as i7’s go, but it was the best price to performance one with 40 lanes that I could find at the time. CPU performance was not emphasized nearly as much in the fastai forums when I was building my PC, but it is very important as Jeremy said for fp16 which would come into play if he got the 2080ti plus driving the other 1080ti’s. Threadripper is certainly less expensive than i9 so it’s a balance on what you want to do. I don’t have benchmarks on which is better, or by how much in deep learning workloads, i’m just going off of what Jeremy recommended.
Turning off your display manager is certainly an option to free up GPU ram. I find the display manager in Ubuntu to be finicky if I mess with it so I try and leave it alone as much as possible. This is partly due to my GPU i use for my display not being in slot 1. My GPU I use for my display is sitting at 1GB of ram used at the moment driving my 4k display with some browser windows open. That is a substantial amount of memory to be taken up if it is needed for training. You can almost always decrease batch size and it will work but if you have another means of driving your display it is 1 less thing you have to worry about (cuda out of memory errors). In addition, I mentioned that it would only benefit him if he was planning on using the display while training i.e. not as a server. I don’t think turning off the display is an option if you wanted to use jupyter notebooks on that display. If you were only running training via scripts it wouldn’t matter really because you could launch them from the terminal.
Integrated GPU is an option but both your CPU and motherboard need to have the capability. None of the high core count i9’s have integrated graphics. I didn’t think that threadripper had integrated GPU’s either, but I have not really kept up to speed with Threadripper.
2 Likes
Mark,
Adding case fans, as long as they physically fit and are installed in a direction that makes sense, will almost certainly help. They are cheap and pretty easy to install so may be a worth-while experiment to run even though it may not solve your issue. It is cheaper and easier than your alternative. It’s not possible for me to know for sure if it will definitely solve the issue or not.
As for your current GPU - It has a different style of fans than mine. I know these have different airflow characteristics than my GPUs. That GPU will stay cooler if you don’t have a card immediately against it, but because its airflow characteristics are different than mine, I can’t tell you if you will be OK adding another GPU directly against it or not. I just don’t have any experience with that style of GPU cooling. If I had to make an educated guess, I would think that your current GPU will be too hot w/ another GPU directly against it while running deep learning workloads. That is probably the most common style of GPU fans though so maybe someone else could answer that question who has a similar style GPU.
Getting a new case with enough properly installed fans and a new motherboard with larger spacing between GPU’s will solve your heat issue. Your setup is definitely not optimal for multi-gpus with respect to thermal management, but I really don’t know if it will be good enough once you add case fans and your additional GPU or not.
Thanks,
Mat
Anyone using water cooled GPUs? my 1080 ti runs at 82 Celsius. Any ideas on bringing down temp
Thanks @matdmiller.
Looking a bit more inside your earlier comment strikes me - the first GPU does indeed occupy two slots so I’d have no option but to put another GPU into slot 3 which has a bit of a gap it seems (I don’t want to buy a new motherboard) - so it looks like I might need some research as to whether skipping slot 2 is an issue or not.
Thanks for all your help, I’ll report back with what I decide.
Mark
1 Like
I would try adjusting the fan curve first
2 Likes
Hi @matdmiller,
Quick further question if I may (just realising how much info you’ve actually given me now I’ve started googling 
): what does PCI Express 3.0 x 16 slot (x4 link) mean? I think this is my expansion slot 3 (where I’d hope to put the second GPU) but I’m a little confused what the ‘(x4 link)’ bit means.
Many thanks again,
Mark
Blower type gpu would be better for multi-gpu setup if water cooling is not available.
1 Like
It means that the slot is sized for x16 devices to be able to plug in, such as a graphics card, however behind the scenes it only has the bandwidth of an x4 link. There are x2, x4 x8 and x16 slot sizes and link sizes. You can run a x2, x4, x8 or x16 link over an x16 slot size, but you cannot run a x16 link over an x4 slot size. Just because your slot size is x16 doesn’t mean it has x16 of bandwidth available. From everything I’ve read you need a minimum of x8 bandwidth to be able to run a GPU for deep learning meaning x4 link bandwidth is not going to work. On some motherboards, some of the slots have a fixed maximum bandwidth and on others the bandwidth varies per PCIE slot based on what else you have plugged in. Most motherboard manuals show you this information. If you install PCIE cards as shown in your motherboard manual, you can generally expect it to work as described in the manual. I am sure there are some motherboards that will work with configurations not explicitly specified in the manual but the only way to know for sure is testing, experience or having a deeper understanding of the architecture than the average PC builder. Also, just because it “works” does not mean it will perform optimally or to your expectations.
2 Likes
Thanks @matdmiller - that basically kills the 2 GPU idea for now then. 
I suppose a quick win would be to use a second GPU to drive my monitor (I have a crappy 2GB GTX 650 ti from my old PC that ought to be okay I think) as well as trying a few extra case fans. You mentioned that you drive your monitor from a GPU not in slot one - is this straightforward enough to do?
Cross-posting my reply from another thread since I think this table might be relevant to this discussion.
Recently, I was researching nVidia GPU cards and synthesized my leanings of the cards in a table here.
I always found nVidia’s naming super confusing (GTXs, Kepler, Tesla, Pascal, etc). So I made this table. I hope others find this summary in table format useful as well.
References
https://web.archive.org/web/20180821173206/http://timdettmers.com/2018/08/21/which-gpu-for-deep-learning/
https://thewiredshopper.com/gpu-hierarchy-graphics-card-list/
https://blog.slavv.com/picking-a-gpu-for-deep-learning-3d4795c273b9
https://en.wikipedia.org/wiki/Volta_(microarchitecture)
https://en.wikipedia.org/wiki/Nvidia_Tesla
https://en.wikipedia.org/wiki/Nvidia_Quadro
https://en.wikipedia.org/wiki/GeForce_10_series
1 Like
Hello everyone,
I’m currently in the final stages of building my own DL rig. I plan to have a 2080ti for now, and I’ll add a second video card in a few months. Any comment and suggestion on my build would be greatly appreciated !
PCPartPicker part list / Price breakdown by merchant
| Type | Item | Price |
|---|---|---|
| CPU | Intel - Xeon E5-2620 V4 2.1GHz 8-Core Processor | €415.00 @ Amazon France |
| CPU Cooler | Cooler Master - MasterLiquid ML240L RGB 66.7 CFM Liquid CPU Cooler | €68.47 @ Amazon France |
| Motherboard | MSI - X99A GODLIKE GAMING EATX LGA2011-3 Motherboard | €485.09 @ Amazon France |
| Memory | Corsair - Vengeance LPX 32GB (2 x 16GB) DDR4-3200 Memory | €326.99 @ Amazon France |
| Storage | Samsung - 970 Evo 500GB M.2-2280 Solid State Drive | €140.89 @ Alternate |
| Storage | Seagate - Barracuda 2TB 3.5" 7200RPM Internal Hard Drive | €65.90 @ Amazon France |
| Video Card | MSI - GeForce RTX 2080 Ti 11GB GAMING X TRIO Video Card | €1360.91 @ LDLC |
| Case | Rosewill - RISE Glow ATX Full Tower Case | |
| Power Supply | Corsair - 1000W 80+ Platinum Certified Fully-Modular ATX Power Supply | €216.99 @ Amazon France |
| Prices include shipping, taxes, rebates, and discounts | ||
| Total | €3080.24 | |
| Generated by PCPartPicker 2018-10-25 17:56 CEST+0200 |
1 Like