Recommendations on new 2 x RTX 3090 setup

Deep Learning
121

  1. Good job :wink:

  2. Since you mentioned windows, be advised that windows 10 pro doesn’t bode well with more than 64 logical processors. Use windows 10 workstation edition.

  3. Does the bottom 3090 cover the front panel connections?

  4. Do they show any sign of sagging?

  5. Why do you use the cards on such inefficient overclocked settings?

  6. Would you share the schematics?

1 Like
122

2: Yea I am using workstation, but whenever I do machine learning I jump over to ubuntu anyway. I am currently trying to figure out docker so I can start using it with fastai v2

3: Yes it does so I have to remove the card to get to those connections. Also it really smooshes them in there in an almost uncomfortable way I feel.

4: Nope but I want to move to watercooling anyway because I find these things to be way too massive.

5: I was just messing around with msi afterburn. I honestly have no idea what I am doing.

6: Schematics of the final build? Sure! Whenever I figure everything out. I am thinking of getting a TT WP200 just to start looking at it and planning out how I will do it.

123

This was the main reason my vendor advised me to move away from Zenith so I switched to team blue (I was earlier going to get 3960x + Zenith II Mobo myself)…

1 Like
124

  1. If I may: how do you start the machine with the second card installed?

  2. Nonetheless, it’s good to know that the heatsink is rigid enough to prevent sagging.

  3. Look at post #23. I seek a similar configuration (horizontal gpu mounting, 2nd story), but closed and dust-proofed. If you live in north america, mountain mods does something like that. Nothing in EU, I’m afraid.

1 Like
125

Not to appear too smug, but I told you that before your vendor :stuck_out_tongue_closed_eyes:

Seriously, it’s quite upsetting that you buy a 5000$ processor and then there is not even one single professional motherboard upon which to install it.

2 Likes
126

Apologies and correction: My Vendor had confirmed what Andrea had warned me about :slight_smile: :pray:

1 Like
127

:muscle: :muscle: :stuck_out_tongue_winking_eye:

2 Likes
128

@balnazzar the motherboards you suggested look interesting. I quite liked the ROMED8-2T Mobo. Didn’t see the zeon ones yet. What type of cases do these mobos need?
The rome Mobo seems to be ATX factor which suggests normal full-tower PC cases should work? Although, the EPYC CPUs seem to be priced quite high, $700-1200 for a 16-core chip. So many epyc models (pun!), I got confused what to even look for.

@init_27 Which vendor are you consulting? I’m in India too. Feeling the need for a workstation right now for extended experiments and contemplating a 4x 2080ti setup vs waiting for 30xx series.
And have you come across good websites for used stuff here? I found some, but no one is selling this type of high end stuff.

1 Like
129

Ant-PC, I’m happy to connect you with them, they do build DL systems similar to your req.
Based on my market research, they are offering me GREAT Pricing + They have an awesome service, 10/10 would recommend-when my 2080Ti broke down and they didn’t have one, they sent me a spare 2080 while they got the replacement figured out, great service.

1 Like
130

The ROMED8 is just a standard ATX mb. Also, look for the Supermicro X12 series for epyc, also ATX. The X11SPA is an SSi-EEB motherboard (30x33cm). Note that such format will allow you to use all the slots without covering critical connectors, and it also has a certain number of m.2 sockets away from the video cards (e.g. they won’t overheat). On the other hand, a standard case won’t support it (but the Ethoo Pros are good, inexpensive ones that support it, along with a lot of gpu-oriented rackmount cases). No EEB mobos for Epyc, since AMD doesn’t want people to use them for workstations. Also, no Epyc motherboard supports sleep/hibernation (ACPI S3 & S4 states). Last but not least, any cooler for TR4 will support SP3 (epyc).

The best Epyc Rome in terms of bang4bucks is the 7282, followed by the 7302P. Both are 16c, respectively 600$ and 800$, with the former having four memory channels and a 120W tdp (vs. 155W). Around 1200$ you’ll find the 7402P (24c).

Note that a 3960X is, essentially, a de-featured 7402P (no RDIMMs, half memory bandwidth, half pcie lanes, crappy mobos, but increased clock and, consequently, insane tdp & heat generation). Indeed, all Zen2 TRs have been developed from the corresponding Epyc Rome.

1 Like
131

Hi there, I just got a 3090 and it took me a while to get it working on Ubuntu 18.

I installed these display drivers:
https://us.download.nvidia.com/XFree86/Linux-x86_64/455.28/NVIDIA-Linux-x86_64-455.28.run

And this version of CUDA drivers:
https://developer.nvidia.com/compute/cuda/10.1/Prod/local_installers/cuda_10.1.105_418.39_linux.run

First remove all NVIDIA drivers you might have installed before installing the above.

Hope that helps.

3 Likes
132

The ROMED8 documentation at https://download.asrock.com/Manual/ROMED8-2T.pdf mentions S4 (pg. 26) as well as some S3 related debug BIOS codes. Does anybody have experience with this MB and can confirm if it supports sleep/hibernation?

1 Like
133

I have experience with its close sibling, the EPYCD8-2T. Tried every trick on Earth to get it in S3/S4 (Linux). No success. Note that S4 should be much simpler. Still, it was not capable of restoring the system after a S4 shutdown. Blank screen.
People here and there all over the internet report the same for the ROMED8-2T and Supermicro X11 series for SP3 socket, despite the fact that manuals typically do mention at least S4.
That’s not to say I’m 100% sure that one cannot get it (them) in S4. It’s just that myself and some people over the internet did not succeed.

134

Hey man! Did you run some benchmark upon your dual 3090 rig? Keep us posted! :slight_smile:

1 Like
135

Hi all.
Quad GeForce RTX 3090 in a desktop - Does it work?
Here are some results from Puget Systems.

1 Like
136

As expected, lowering the power limit barely impacts the performance, if at all (although it could be said that if one forgets to set the limit upon startup or something goes awry with the boot scripts, you risk to blow a capacitor).

What surprised me is that the 2-slot turbo cooler seems to be perfectly sufficient in cooling the cards, even in the 4-gpu setup and even running them at full power (80C absolute max). Note that the airflow from the front fans is actually blocked by the drive cages. Removing them will probably improve the temps a bit.

Now the only issue consists in getting our hands upon those Gigabyte cards, which, I’m afraid, seems to be near impossible right now.

137

Unfortunately: https://sea.pcmag.com/graphics-cards/39831/report-nvidia-cancels-rtx-3080-20gb-and-rtx-3070-16gb-graphics-cards

138

I just got my new 3090 :smile:
I will start benchmarking after its 5 days of isolation are complete. :sweat_smile:

Meanwhile, If anyone has any benchmarks that they’d like me to run or would want to help out, please lmk.
I have a 3090, a Titan and a 2080Ti so it should be a great benchmark build. (All in a single build, air-cooled)

I will share how this is air-cooled after testing-I need to make sure nothing is bottle-necking/thermal-throttling before sharing my setup

3 Likes
139

Since this is a fastai forum, I would just benchmark the existing course notebooks as they are written to get a sense of how the 3090 stacks up to previous gen processing. Then I would try to max them out with different batch sizes to see how much time can be saved, and the resulting accuracy. Then, I would try other datasets. Again, that is just my jealous suggestion.

140

Agreed, but it won’t hurt to add a big transformer to the test suite :wink:

For example: https://huggingface.co/transformers/benchmarks.html

Congrats pal :slight_smile:

1 Like