How to use Multiple GPUs?

That is exactly what was happening with me. One gpu worked, two failed. I now use it with one. It seemed too complicated to change the PSU.

Anyone here who has compared performance and practical use of of 2x gtx1070 vs 1x gtx1080 ti?
On paper you get
8(x2) vs 11gb RAM
1920(x2) vs 3584 processors
256(x2) vs 484 GB(s) bandwidth.

For a single model, a c. 50% speedup scaling to 2x gpus would put 2x1070 on par with 1x1080ti (using T. Dettmers rough performace metrics)

@avn3r just to be clear if you increase batch_size * num_gpus, then youd be able to combine the memory of both GPU’s (ie for 2x1070 for 16GB)?

Does multi gpu in pytorch work better for some nn’s (eg convnet) vs say rnn’s (eg see Tim Dettmers posts)?

If you had room in your box you can just use a second smaller PS for the second GPU only, may be cheaper to do that than buy a 1000W+ PS.

May have been mentioned somewhere here already but pcpartpicker will allow you to calculate your power use for current parts, ball of park Ive read is to add 100W buffer to this for safety.

According to PCPartPicker the power requirements are 626W and I’ve got at 750W PS. Guess an extra 124W isn’t enough.

Personally ive gone with a gold+ rated psu with a some wattage to spare in case i add some more parts.

Just got a reply back from ASUS:

After reviewing your email I understand that whenever you try to run both of your graphics card at the same time your PC crashes. I’m sorry to hear you are experiencing this issue and thank you for letting us know about this. It will be my pleasure to assist you in resolving it.
I appreciate the details that you have sent out. As I check you are using a MSI motherboard, the best recommendation I can give you is to contact MSI for the power requirement of the power supply that you must use for your system to be stable.
Because as I check with our ASUS motherboard if you are a using a high-end PCIe devices like the TURBO-GTX1080-8G you must use a 1000w power supply or higher to assure the stability of your computer. Also I am very sorry to tell you but you cannot throttle back the power requirements using NVIDIA-SMI because the cards need power for it to work properly and to assure its stability.

What temperatures do you get per card when both were running?(or has the PSU issue prevented running both for any significant time?). I was running a 1080ti and 1070 in same box and was getting 80-90 degrees C on the gpus (a couple of mm apart, nb these were ‘open’ style with 3 fans not fe turbo shrouded). Alone the 1080ti sits at around 70 degrees. Now i am in process watercooling-which is a pretty time consuming process

Use MSI Afterburner if you are using windows to control the temperature of the NVIDIA cards
Don’t let them go above 70 C or your cards will fry.

In Ubuntu you can use the nvdia tools to set the fan parameters.

NVidia-smi is a great tool to see the numbers. (Win 10 and Ubuntu)

Working at last with 2 GPUs, 1000 Watts is the anwer

Sorry for the later reply, but I’ve been busy ripping out my 750W Platinum PS and replacing it with a 1000W gold PS. As for the temps, the GPUs run as hot as 81C and draw as much as 187W of power each. The Enhance.ipynb notebook is now running fine with both GTX 1080’s engaged.

Interesting. But how do you make it work? I devise a couple of problems:

You got to have the second PSU starting as the system starts, but the signal comes from the mainboard, which is connected just to the main PSU.

If you run a ac power cable through a hole in the back of the box the second internal psu would be powered on just like you power on the main one vie your powerpoint/ups. Could use add2psu if dont want to use jumpers. For the messing about and extra room this takes, unless you have a really large case and spare psu lying around, easier and safer to use one.
Something im looking at right atm is to mount a psu on a bracket outside the case and use dc power cable extensions-will allow me to fit more fans in the case

My problem is that I have a Fujitsu M730 workstation. It is a very nice, sturdy workstation, with plenty of ecc ram and a 40-lanes Xeon e5 CPU.
On the other hand, the power supply and the mainboard are coupled with a 12V (non-ATX) proprietary connector, which makes it impossible to replace the PSU.

It wuould be a shame to trash such a reliable and powerful machine, so I’m interested in adding a second psu. Indeed the problem of bootstrapping both PSUs is solved with that add2psu contraption (thank you, I didn’t know it), but I wonder how much additional power consumption the second psu would add, particularly while the machine stays in idle… Has someone made experiments about that?

The watts on the PSU are the power the PSU is rated to on demand. Without load power use should be minimal - especially if you get a 80+ gold or higher efficiency. You should be able to find a good efficient one with a bit of research (I recall seeing some graphs but cant recall where). Out of curiosity which Xeon E5 did the machine come with?

Xeon e5-1650v2, six cores, Ivy Bridge-E generation. The main handicap is the power supply. Good quality, 80+ gold, the system consumes less than a mainstream quad-core, particularly in idle (it idles at ~40W). But it is 500W and has just a single 6-pin connector.

On the other hand, I had the occasion of purchasing an XFX 650W (also 80+ gold) with four 6+2 connectors, for just 65 euros. Without the burden of powering the system, I think it can easily cope with two titan-class GPU or three 1070.

My main concern is that the whole contraption will be incredibly untidy. One option would be rebuilding in a dual psu-case, which are extremely difficult to find though.

I have a coolermaster HAF 932 which supports 2x psu mount locations, havent seen any other cases that support 2 may have to get a full tower and build a bracket and just rivet in. An old machine i have has a xfx 650, only thing id note was that the output cables warent quite as long as newer psus but were still ok. Ive found dell component sizes can be non standard (mb and psu), may want to check the fujitsu ones will fit without too many mods to a new case

1 Like

Yes, indeed. The cables by fujitsu are pretty short… They do it on purpose, to prevent you from making custom mods…
I have found a dual psu case which is quite cheap: the Anidees AI8.

In case you are curious, I did it as per your advice. Two PSUs with add2psu, and a large case specifically engineered for supporting them. Also posting it here for someone else who seeks to build a multi-gpu rig with appropriate lanes and power.

The good thing is that all this contraption was not expensive (100 euros the case, 67 euros the second psu, 6 euros the add2psu).

The xeon-e5v2 workstation has been bought crap cheap from ebay.

The only expensive things were the GPUs (1080ti + 1070), for a total of almost 1200 euros. I’ll add an NVMe drive soon, and a little one slot video card to leave the two GPUs alone for DL.

2 Likes

Thats a great looking build, and probably couldnt do much better for terraflops per dollar.

Id be interested in gpu temps your getting, especially the 1080ti fe. (A blog on build specs could be useful for others looking for a relatively low cost high performance build if you have the time)

Something i noticed is that it takes a fair bit of time to plan the experiments to run and qc results to keep a couple of gpu’s busy 24/7 (i work full time in a non ml/dl job, so have to stay focussed).

My motherboard only allows me to use a specific slot for primary video, which i am using a pcie riser in so i cant have a simple video only gpu :frowning:️.

I bought an nvme last weekend 240gb are at a good cost with sufficient capacity for now for me. I havent noticed any difference over an ssd yet but havent done any benchmarks.

84C in FurMark Burn-In mode. Fan speed never exceeded 52%.

How true :slight_smile: But I think we’ll improve as we get more experienced.

It would be interesting to build an external enclosure. However, you can upgrade grabbing used parts on ebay!

If you benchmark it, let me know. For example, a pair of epochs over a big dataset with precompute=false would be a good baseline.

I ran a few tests on NVME vs SSD, and put results in part 2 of my medium post. For day to day notebook use, probably wont notice any difference, for heavy read-write definitely will notice. None of the tests I ran in notebook showed much difference, but on unzipping multiple zipped folders on the ssd vs nvme, nvme was 2-3x faster. Interestingly, at my local computer shop Samsung 250GB nvme dives are slightly cheaper than 250GB ssd.