That’s another point… The hardware used in computer liquid cooling is basically recycled stuff from acquariums.
Even the best things (d5 pumps, etc…) come from that industry.
And they think that’s fine, for people buying them are just a bunch of gaming-addicted kids with flashing led strips.
The only example of liquid cooled professional rig is the DGX station, for which Nvidia had stuff specifically custom-built, and offers rock solid on-site warranty.
How much performance degradation (%) have you noticed after the card starts throttling?
Some good 30% on average, but it changed depending on how much throttling it required (winter… summer…) the power level, and the task at hand. Also remember that operating vrams at throttling temp should be done for limited timespans. For gddr6x, Micron declares they start taking damage at ~120C. Throttling temp is 110C.
I was able to test out one of my 3090’s on another PC. I used nicehash, not sure if that’s the best for testing or not. It looks like my memory junction temps were bouncing between 102-104 after ~30 min of mining. I know that temp isn’t good, but it doesn’t seem to be at throttling temps. This is a stock EVGA 3090 hybrid 3988. Not sure if I’m interpreting this correctly but it does seem like it’s not throttling. I might try adding another fan on top of the backplate to see if that lowers it even more. I’m happy to run another test if you have any suggestions.
They are not throttling, and that’s good, but note that you are operating them at a maximum of 106C, which is very close. Note also that vram occupation is modest, some 5 Gb. Try to fiddle with nicehash settings to see if you can achieve near-maximum vram occupation. Another way could be installing fastai/pytorch on windows and train a big transformer.
An active fan on the backplate could help. Try to put some 10C between throttling temp and your long-time operating temp. That is, less or equal than 100C.
Also reducing the operating power (which is very high, >400W) can help a lot.
Anyway, given that the power level is so high, the card is behaving better than your typical 3090… Probably evga employs good quality thermal pads, and a backplate with good thermal capacity.
But please test with near maximum vram occupation. Vram chips are ugly beasts. If a chip is not addressed by the controller, it doesn’t produce heat.
I thought I’d post here about my 2x 3090 rig, since i’ve seen very few builds with 2x 3090 documented.
Case: Lian-Li O11 XL
MB: ASUS X299 Sage
CPU: i9 10940
Memory
GPUS: 2x3090-FE
PSU: Corsair AX1600i
CPU cooler: LianLi AIO
I selected this motherboard so that i could get 4-slot spacing for the 3090s, allowing a larger air gap between them, per Tim Dettmer’s blog.
The case, i guess is a splurge but also was great to have the space for cabling and circulation.
I had started this build using a 1200W Corsair. This did NOT work for 2 GPU training workloads. As soon as i would start certain types of training workloads, the machine would immediately power cycle. I could prevent power cycling by locking gpu clocks (nvidia-smi -lgc 1600) to something below peak clocks. 1600mhz worked well for me, YMMV. I didn’t have success with wattage limiting using nvidia-smi and i think the reason is because 3090 has power spikes even if you wattage limit, but by limiting the peak frequency, the spikes are reduced. On many training pipelines, this would result in a pretty minimal reduction in overall speed, but i wanted to be able to run the 3090’s flat out, so i migrated to the 1600W PSU which has solved the power problems.
After resolving the power problems, the system’s been a dream.
At flat out, the lower GPU maintains around 63C and upper GPU around 75C. I will probably invest in one more set of fans to the right of the GPUs to provide more cool intake. The lower fan set is pull and the upper AIO is push, the rear fan is push.
This is great. How much did it cost? Would love to see some performance benchmarks if you get some time to share. Cheers!
If you have time and will, check the vram’s temp on the upper one.
I was not able to successfully get fastai working on Windows. When I installed it following the fastai instructions (with some tweaks) I was getting a warning that the cuda version that is installed does not work with the 3090. I have made several different attempts at this with different techniques with no luck. Below is the steps i followed in the latest attempt and I recorded everything I did this time.
If you have any other suggestions for benchmarking I’m happy to give them a shot. I was not able to find any settings in NiceHash to increase the Ram utilization from the last temperature test I ran. I have not tried installing WSL2 and going down that rabbit hole, and I’m trying to avoid that to keep my windows install lightweight as it’s not what I use for real AI work anyways.
Install Steps:
conda create -n fastai8 python=3.9
conda activate fastai8
conda install -c pytorch -c nvidia -c fastai -c conda-forge -c anaconda fastai anaconda gh fastbook fastcore
--------
RESULTS:
/ DEBUG menuinst_win32:__init__(201): Menu: name: 'Anaconda${PY_VER} ${PLATFORM}', prefix: 'C:\Users\mathewmiller\.conda\envs\fastai7\envs\fastai8', env_name: 'fastai8', mode: 'system', used_mode: 'system', root_prefix: 'C:\ProgramData\Anaconda3'
DEBUG menuinst_win32:create(328): Shortcut cmd is %windir%\System32\cmd.exe, args are ['"/K"', 'C:\\ProgramData\\Anaconda3\\Scripts\\activate.bat', 'C:\\Users\\mathewmiller\\.conda\\envs\\fastai7\\envs\\fastai8']
DEBUG menuinst_win32:__init__(201): Menu: name: 'Anaconda${PY_VER} ${PLATFORM}', prefix: 'C:\Users\mathewmiller\.conda\envs\fastai7\envs\fastai8', env_name: 'fastai8', mode: 'system', used_mode: 'system', root_prefix: 'C:\ProgramData\Anaconda3'
DEBUG menuinst_win32:create(328): Shortcut cmd is %windir%\System32\WindowsPowerShell\v1.0\powershell.exe, args are ['-ExecutionPolicy', 'ByPass', '-NoExit', '-Command', '"& \'C:\\ProgramData\\Anaconda3\\shell\\condabin\\conda-hook.ps1\' ; conda activate \'C:\\Users\\mathewmiller\\.conda\\envs\\fastai7\\envs\\fastai8\' "']
DEBUG menuinst_win32:__init__(201): Menu: name: '${DISTRIBUTION_NAME}', prefix: 'C:\Users\mathewmiller\.conda\envs\fastai7\envs\fastai8', env_name: 'fastai8', mode: 'system', used_mode: 'system', root_prefix: 'C:\ProgramData\Anaconda3'
DEBUG menuinst_win32:create(328): Shortcut cmd is C:\ProgramData\Anaconda3\pythonw.exe, args are ['C:\\ProgramData\\Anaconda3\\cwp.py', 'C:\\Users\\mathewmiller\\.conda\\envs\\fastai7\\envs\\fastai8', 'C:\\Users\\mathewmiller\\.conda\\envs\\fastai7\\envs\\fastai8\\pythonw.exe', 'C:\\Users\\mathewmiller\\.conda\\envs\\fastai7\\envs\\fastai8\\Scripts\\spyder-script.py']
DEBUG menuinst_win32:create(328): Shortcut cmd is C:\ProgramData\Anaconda3\python.exe, args are ['C:\\ProgramData\\Anaconda3\\cwp.py', 'C:\\Users\\mathewmiller\\.conda\\envs\\fastai7\\envs\\fastai8', 'C:\\Users\\mathewmiller\\.conda\\envs\\fastai7\\envs\\fastai8\\python.exe', 'C:\\Users\\mathewmiller\\.conda\\envs\\fastai7\\envs\\fastai8\\Scripts\\spyder-script.py', '--reset']
done
---------
conda install ipykernel
python -m ipykernel install --name=fastai8
Sorry for the late response, I didn’t get any notification for this post.
Trust me, WSL2 won’t mess with your windows installation, apart from occupying some disk space. I’d encourage you to install it, as it’s quite straightfoward to work with (no rabbit hole should happen).
When you don’t need it, just leave it shut off.
I just wanted to drop in here and let people know that tons of miners are either adding Copper Plates or Re-Pasting/Re-Padding their GPUs (or both).
The reason being is that there are tons of cards with crap thermal paste/pads. coolmygpu.com is a solid choice for copper plates (which can give a significant reduction in heat) (up to 34% heat reduction on VRAM)
As for creating a system that can handle multiple GPUs. I have to say that it is definitely a challenge. One that I myself run into. (With 4x 3080TI FEs, its hard to find something that is a turnkey solution)
Also, i see a lot of people mentioning the performance difference between the 3080 and the 3090. And you might want to dig into some nvidia spec sheets. NVIDIA RTX 30-series
The 3080 10gb model is only 320-bit memory interface/bandwidth w/ 272 Tensor cores. Meaning it can only handle about 720GB/s in memory bandwidth.
Vs the 3080 12gb and 3080TI 12GB model which both have 384-bit memory interface/bandwidth (900GB/s).
For reference the
3080 has 8704/8960 CUDA + 272 Tensor cores and 68 SMs
3080Ti has 10240 CUDA + 320 Tensor cores and 80 SMs
3090 has 10496 CUDA 328 Tensor Cores and 82 SMs
3090Ti has 10752 CUDA + 336 Tensor Cores and 84 SMs
I finally got around to installing WSL2 and tested out my EVGA 3090 Hybrid Kingpin. I’m having pump issues with my other 3090. The kingpin seems to have excellent cooling. Probably overkill, but the memory temps (all temps really) when training the imdb nlp model from fastbook with fp16 and bs 384 are shockingly low. I was not planning on buying the kingpin as it’s quite a bit more expensive than the other options, but it was all I could get my hands on, though GPU availability seems to be much better now. They seem to have solved the backplate memory temp issue with this card variant.
I think that’s a good price. Particularly the bundle 3090+1600w psu.
Which kind of 3090 did you get?
Look, we can’t chase nvidia and its prices forever. I’m a bit sick & tired of this game. My A6000 has to last quite a few years, and I’m surely gonna skip this gen, maybe even 5000 gen. Consider that people is still doing DL rather gracefully on Pascal cards, and they are quite OK except for the memory amount. That was the point of getting 48gb VRAM.
Interestingly, an M1 Ultra chip is only 3 times slower than a 3090 with Pytorch.
These Apple silicons are on par with Ampere in terms of perf-per-watt, if not a tad better, and of course a Mac Studio with the M1 Ultra is much more silent than a 3090 and outputs less heat.
And, while costing as much as an A6000 (M1U, 128gb, 1Tb) it’s a whole computer, and a very good one.
Maybe the Nvidia monopoly, with its absurd prices, will end sooner than we thought, and not by the hand of AMD.
Honestly, if I had 3000 , I’d spend 2400 on a MBP 14" with 32GB RAM, and invest 600 in Jarvislabs or Paperspace for occasional A6000/A100 runs. Obviously, to get the most mileage for the money, I’d do the initial experiment on a smaller dataset on my local 1070ti or a free Kaggle GPU (which btw is equivalent to a local 1070ti with double the ram.) Once I get my ducks in order I’d fire up a beefier GPU on Jarvislabs or Paperspace, do the runs, save the results via wandb or something and stop the instances. This is more work on my part, but this also means I’d have a really nice laptop which is almost a 3050 equivalent (GPU performance wise) since pytorch performance will only get better from here on forward.
If I find that I’m really getting into this whole DL thing where having a local 3090 or equivalent will actually make a difference , then another 3000 - 4000 would be money well spent as an investment in a career where the ROI will be rather good in the larger scheme of things.
Just my thoughts on this.
My personal perspective (but based upon experience): do not try to run anything serious on a laptop, even a good one. You’ll get mediocre performance, generate a lot of heat & noise and overcook the battery. And that’s kind of a waste on an expensive laptop.
Rather, rig a rubust desktop workstation and use that laptop to remotely log in into it (or into the cloud). For the same price, you’ll get better performance, less heat and noise, and you won’t cook the laptop.
Totally agree! I think I’m just rearing to get a laptop 
I don’t do anything serious with DL atm so using Paperspace/Jarvislabs and/or my local workstation with a GPU works fine for me. But I agree, for any type of serious stuff, laptop may not be the ideal platform (unless its less intensive exploratory stuff.)
Macbook pros are much more silent (at the expense of internal temps, they deem cpu temps >100C to be ok), but you’ll f*ck up the battery pretty soon all the same. Or even sooner. Two things, especially, kill li-ion batteries: fast discharge and heat.
I definitely agree.
Good machine. I have one. Quick, light, and awesome battery life. But the M2 MBA is about to launch.
Not to go OT but I agree, I really like the new M2 MBA design, but I don’t know about that price and with the extra power, it’ll probably throttle more. Though the prospects of holding more data in memory (upto 24GB ram) is attractive proposition.
My ideal MBA would’ve been this M2 MBA design but in 15" form factor. I love the MBA for it being so light but find that screen is too small, so I spend most of my time on my 2015 MBP i7 16GB … though performance wise a base M1 MBA blows it out of the water
Not real OTs, since this thread went for ‘general DL/ML-oriented hardware’ long ago 
I don’t know guys. Maybe, or maybe not. I mean:
As you may see, the random performance of these M1-integrated NANDs are pure and sheer crap. Like… Slower than a SATA ssd.
3090s at 1400$. Very good price. Still, I was very, very unhappy with the VRAM temps when I had the 3090s (granted, thermally speaking, managing 2 cards is quite a different business).
