Having at least a full slot free between cards is ideal.
My 3090’s are not too loud, I do think the hybrid cards are quieter then air cooled cards to some degree. Having good airflow in your case really helps reduce overall noise because the fans don’t have to run as fast. This is quite important. These do kick off a lot of heat when running full tilt. Even when idle I still notice a difference when the machine is on vs off. I have mine running in a bedroom that I converted to an office. When I’m running full tilt that room gets very warm (85+ degF) and I often have to open the door to cool it back down.
Generally the specs look fine for a single card machine. Intel has an edge over AMD for some AI workloads. As far as the components, it’s hard to tell if they are good or not as they don’t actually tell you what is included. There is a good chance that because they don’t tell you, it’s probably the cheapest versions available which probably aren’t the best. Ex: there is a huge range of NVME SSD’s out there. Some are great, some are really crappy. They don’t tell you which ones are actually included. That’s just 1 example. Also 1,000W PSU is not going to cut it for multiple GPU’s. I think if you selected parts for a machine you would probably spend less money and get higher quality, but it does require up front research and work putting it together.
Hi @matdmiller ,
Do you (@matdmiller ) or anyone having any experience of running multiple models on a single gpu (RTX 3090 or RTX 3090 TI on PyTorch + windows11 ). I tried running two models to train and it takes double time when i run two separate models in separate power shell on windows (is there any limitations on rtxt 3090 or rtx 3090 TI) ?
When training models, there are many possible hardware bottlenecks which constrain the overall speed. If training a single model maxes out your current hardware bottleneck then training a second identical model simultaneously on the same hardware would cut the speed of each in half. Ex: if training model A was using 100% of the processing power of the GPU, then when training 2 identical models (A&B) at the same time on the same GPU, each will be allocated 50% of the GPUs processing power and will take 2x as long to complete. (training A&B simultaneously takes the same amount of time as training A then B sequentially). A counter example might be if you had 2 slow hard drives and your hard drive speed was the bottleneck, then if you had 2 hard drives and model A dataset was one 1 hard drive and model B dataset was on the other hard drive then you are alleviating your bottleneck by using a second hard drive and training both simultaneously would not take 2x as long.
@matdmiller Hi Again,
Do you think it makes sense to go for rtx a4500 or rtx a5000 instead of rtx 3090 (because of power and price and one can fir 4 a4500 in a machine but not 4 rtx 3090 in a machine ) ?
Do you think if this build makes sense (its air cooled of 4 rtx a4500 and idea is to use nvlink to increase memory to 40 gb) ? or should go for dual rtx 3090 ti ?
Don’t expect the 2xA4500’s to appear as 1 big card with 40gb of memory after you have installed the nvlink bridge. That’s not how nvlink works for deep learning. You would potentially have better performance for model parallel workloads, but it’s not necessarily trivial to set that up and get it working optimally. If you truly need > 24gb of ram for your models then you’re probably best of stepping up to the A6000 but it’s an expensive card so you’d want to be sure.
Previously you mentioned the purpose of this machine is for personal use for learning and experimenting. You have spec’d out a > $10,000 machine. If you’re just getting started I would personally start off with something a lot cheaper with 1 or 2 cards (3080 or better). If you’re not worried about the money, then what you’ve spec’ed is fine but probably overkill. If you’re planning on running this constantly for business use then I’d go with the pro A series cards, if it’s just for learning for you then I’d go with the consumer grade cards and go with a consumer grade cpu and motherboard combo or used workstation grade gear.
But NVIDIA website says otherwise (it says it appears as a single card of 40 gb, so if a model is 40 gb big, I can fit on two gpu and there will be no replication, but what i guess is that the total memory bandwidth will be limited by the nvlink bandwidth of 112 ?) :–
NVLink enables professional applications to quickly and easily scale memory and performance with multi-GPU configurations. NVIDIA NVLink Bridges allow you to connect two RTX A4500s. Delivering up to 112 gigabytes per second (GB/s) of bandwidth and a combined 40GB of GDDR6 memory to tackle memory-intensive workloads.
The application has to support Nvlink’d cards. I suggest reading more about PyTorch distributed training- Model Parallel and Data Parallel. It’s possible to use resources from multiple cards when doing training (or inference) but you have to set the model up to do that, it’s not the standard behavior and is not the same as having 1 larger card.
The CPU and Motherboard you selected are not compatible. Apparently Micro Center’s pc builder doesn’t check that.
I suspect the PSU you selected is not sufficient or at best barely sufficient for 2x3090ti, I don’t know which 3090ti’s you have so I wouldn’t know if there was sufficient slot spacing for them on the motherboard, you should confirm your motherboard will allocate 8x (or 16x) pcie lanes to each card.
I would stick with NVME storage for your ‘fast’ storage and a spinning hard drive for bulk cold storage. A sata SSD is probably not fast enough to feed your cards, at least in some situations. You could probably get away with a sata ssd for your boot drive if you wanted. Make sure if you populate both nvme slots on your motherboard that it doesn’t disable the second pcie slot or that the pcie card doesn’t disable the second nvme slot (if you want multiple nvme drives). To know this you’ll have to read the motherboard manual.
I have 3090 TI Founder edition (its almost similar to 3090 Founder edition). Micrcocenter somehow changed the PSU to lower value, but I will keep the PSU demand in mind. I plan to run these cards with undervolt or with less power like 300 w. Do you have any motherboard in mind which make sense ?
These are 3 slot cards so ideally you want 4 slot spacing.
I have not looked at motherboards recently so nothing in particular to recommend. I do like Asus boards, but there are certainly other good brands. Ideally you want something that enables your required slot spacing and delivers at least 8x PCIE Gen4 lanes to the slots where you’re planning to put your GPU’s as well as supports those speeds with the NVME slots populated. You will have to read the users manual to figure this out.
I did a quick google search for Intel 13th Generation Dual GPU and this board popped up in a forum in one of the first results - Gigabyte Z690 Aero D. This board looks like it has 4 slot spacing between the GPU pcie slots and it allows the bandwidth to be shared between the top 2 slots if they are both populated at x8/x8 speed. This is ‘generally’ what you’re looking for. Some of the other boards had the second x16 slot as a 4 lane link going through the chipset - this is not what you want. As with anything do your own research, but hopefully this gives you a good starting point.
Yes but then you have to figure out how to mount the GPU.
ECC Ram for a research PC is not necessary.
You’re probably going to need a full tower case, not mid-tower, that supports eatx. Make sure your bottom card will fit. I have personally used the Lian Li XL case which I like and should be big enough, but I’m sure there are other good options too.
What are you confused about? I would suggest reading up on pcie lanes and cpus/motherboards. If you haven’t built a computer before then you will have to put in the time to researching things. When I built my first deep learning computer I probably spent 20-30 hours doing research and learning about things that others suggested were important.
x299 would probably be fine, but it’s EOL so not something you’re going to be upgrading in the future.
I have a question, the below mother board does not have ssd, what are the option in that scenario, if we use adaptor for ssd, will it be slow or any other issue ?
ASUS Z9PA-D8 (Ver 1.02) Motherboard, 2x Intel Xeon E5-2620 v2
Also, does it make sense to go for it if i work in computer vision with two rtx 3090 ti or four rtx a4500 (due to MKL libraries optimization, any suggestions ?)?
