Recommendations on new 2 x RTX 3090 setup

Hello wise people of Multi-GPU kingdom!

Tomorrow my system completes 5 days of isolation so I can now get started.

I have a x299-Sage (ASUS) Mobo which has 7 PCIE x16 (YES, 7!!!) Slots, so my plan is:

  • 3090 in top slot
  • Titan in 5th Slot
  • PCIE Extendor coming out of 7th slot for my 2080Ti
  • Wifi Card in 3rd Slot

I still might have to swap the Extendor and Wifi card-we’ll see.

I’ve a 10940x which means the 2080Ti will be on x8 lanes- which is still amazing! I’m not sure if I’d want to plug in a second M.2 since it might lead to the 2080Ti being on x4 lanes? (I haven’t done my math)

I’ll keep you all posted!

1 Like

mmhh, you should see the mb’s specifications. Lane distribution topology given a specific processor varies. Some slot will run x16, some x8 and some others could even drop to x4

1 Like

Thanks for the suggestion!

Looks like Titan and 2080Ti will be on x8 according to this (Not sure if I’m reading it correctly)

You’ll go at 8x on that slot no matter what, and that’s not by the virtue of the processor, which just supports 48 lanes. The mainboard got two Broadcom PLX switches, as you may see on the diagram.
The m.2 and the u.2 will always go at maximum speed. As for the slots, You can have 16x/16x/16x/16x if you use just the grey slots, or 16x/8x/8x/8x/8x/8x/8x if you use all the slots.
All of the above with a 48 lanes processor, like yours (they become 96 lanes).

1 Like

Sorry for stupid questions, I had a few more Qs for the diagram. Please point me to a better resource where I can read these if the Qs are too stupid:

  1. The bottom PCH (X299) box, do those 4 PCH lanes count from Mobo or are those supplied by CPU?
  2. All of the LAN/USB/Audio parts-do they eat more lanes from CPU? What do the x2 and x1 for different sockets signify?

I understood the 2 PLX switch part-so that means the PLX switch supplies the lanes further to the PCIE sockets?

I’m not sure about 2 things still:

  • Where should I plug my Wifi Card (it eats x1 slot) to avoid a GPU falling to x4
  • Should I plug in a second M.2? (I have a 970 Pro 512 GB and a WD Black 1 TB from my older build)

Edit:

Also to add:

My current M.2 is plugged into M2_2 (Bottom one) and not M2_1 (The one that gets a lane from CPU), and I don’t have any U.2 drives

Edit another diagram:

1 Like

These are not stupid questions.

Look at the specs: https://www.asus.com/us/Motherboards/WS-X299-SAGE/specifications/

They guarantee your slots will run at no less than 8x no matter what, and the same goes for the m.2/u.2 ports. That’s thanks to the PLX switches, that effectively double the lane amount provided by the processor. The lanes by the PCH stay the same.
If you still are unsure, take a x299/c422 motherboard whatsoever without the pcie switches, and look at the specs. They will mention that lane bandwidth is shared between the slots and/or with m.2 ports, due to lane scarcity.

Wherever you want. No GPU will fall at 4x.

As you wish. It will not eat up slot bandwidth, nor will it drop at 2x.

In this topology, they are supplied by the mb, but even if they were supplied by the cpu, there are plenty of them thanks to the PLX.

They eat up lanes provided by the chipset. Then, it communicates with the CPU by DMI.

Bottom line: do not worry. Plug in every piece of hardware you want into this board. No GPU will drop below 8x.

1 Like

Thank you Andrea!

It makes sense now! The PLX Switch act as if PCIE lanes are supplied in parallel and not in series :smiley:

I was kind of hoping to run the second GPU at x16 (for no reason) but I don’t think my CPU has that many lanes :sweat_smile:

Now it makes sense-I was getting confused w my Z270 mobo-I was assuming that if I plug it into a slot that would cause the single lane for wifi card to come out of the 8 lanes, but the PLX switches are life-saver :+1:

Noted!
Now I’m just waiting for my 20 CM riser cable order-to be chained with another 25 cm riser cable and then the build is complete :smiley:

1 Like

Sorry for more stupid questions:

  • Has anyone tried working with Mixed RAM Sizes?
  • I have 4x32 Sticks and 4x16 Sticks from my older build (Same brand and frequency)

Currently the 32 Sticks are populated in A1,B1,C1,D1 slots-I cannot understand which slots to populate with the 16GB Sticks :sweat_smile:

Maybe I should just sell the 16GB sticks and get 32 ones instead.

Here are the recommended settings but no mention of mixed sizes :frowning:

Recommended setup:

1 Like

Since yours are non-ecc, unregistered dimms (x299 does not support ecc rdimms), you can safely mix them, at least if they have the same number of ranks (but they could work even with different ranks, if you place them correctly). Check it, the label should report something like “2Rx4”.
Put the 32gb dimms in the grey or black slots. Put the other dimms in the slots marked with the other color. Try to respect that ordering, particularly if the memory modules do have different number of ranks.

1 Like

Don’t wanna work towards a unified fastai bench based on fastai’s training scripts?

Thanks Andrea!
Please excuse (again) my lack of knowledge:

Where should I check for labels and I’m not sure how may I check the ranks.

The Memory I have installed is in gray slots: it’s this one and the 4x16 ones I’d like to add are these ones.

Sorry for more stupid questions that somehow keep appearing in my pursuit :sweat_smile:

1 Like

Your observations are not stupid. On the contrary, I’m accustomed to server/workstation memory, which have a label upon their chips, and I took it for granted about yours too.

These damn gaming memory manufacturers do not report their rank distribution… Look:


That’s why I usually stay away from gaming hardware.

Note that they have different SPD speeds (the SPD informs the mb about the memory specs).

I’m still pretty convinced that they’ll work, however. Just try. All the memory will be probably set to the lowest speed. Worst case scenario, the system won’t boot, or it will boot, but you shall experiment moderate to severe instability.
Nothing will be broken, though. In such case, just remove the second set (black slots), and reboot.

1 Like

Thanks! In all my time spent reading subreddits and blogs upon blogs I was worried if I missed all these details :sweat_smile:

I’ll try and share what happens, the issue/reason I’m measuring twice before cutting is-the CPU radiator (360mm) is mounted on top and is partially blocking the RAM Slots, I’m sure I can manage my way around and insert the RAM but taking it out might require unmounting the rad-which would be a lot of work :sweat_smile:

Time to shutup and plug-unplug :slight_smile:

1 Like

Don’t want to give unrequested advice but… In the future try and stay away from gaming stuff. It’s not intended for serious applications. Reliability should be preferred over speed, and in any case you can have both by choosing with care. Good luck with your experiments :wink:

1 Like

All your advices are very welcomed, duly noted! :pray:

1 Like

After 3 hours of trying and giving up and checking my bank balance to see any multiples of 32 GB Ram, I decided to try one last time :smiley:

After quite a few CMOS resets and plugs and unplugs, it worked :pray:

New issue now: due to some reason my SATA SSD is not being recognised :open_mouth: I tried investigating-all of the SATA Ports are enabled, not sure what happened :man_facepalming:

3 Likes

Is the SATA controller in AHCI mode? Maybe the cmos resets defaulted to some other value.

Show us the screenshot relative to SATA configuration.

Thank you Andrea!

More updates from the Chai PC:

I actually tried the BIOS options and forgot the most important trick of Engineering : Plugging and Unplugging. That made it work :joy:

So now I have The 3090 and Titan and 192GB RAM in XMP (3200Mhz) Setup. I’m still need to do some testing for temps, they do have a PCIe slot space in between.

Just the final riser cable remains for the 2080Ti addition :+1:

1 Like

I think it will suffice. Mind that the Titan is the worst of the lot in terms of overheating attitude. Place it accordingly (I’d say into the lowest slot, but then the other cards will suckle its heat… Do some testing).

1 Like

Thank you!

Currently:

  • 3090 in Top slot (Completely blocks 1-3 slots)
  • Titan in 5th (Blocks 5th and 6th)
  • PCI Extender (Extremely bent and almost touching the Titan cooler) in 7th slot

If I move the Titan to 6th, I’ll have to unplug the front I/O so power limiting might be the best case solution

2 Likes