Hi all,
I have decided to build a new PC.
I was planning to buy Nvidia RTX 5090, but Nvidia has also announced Project Digits marketed as “personal AI supercomputer”.
I need to decide which one to buy before 30th January as the 5090 Founders Edition will be immediately sold out, probably never to be seen again.
My main interests are:
- General Deep learning training (primary requirement)
- Would love to try training generative AI (both images & text)
- Would love to be able to train/fine-tune/run small/large LLMs locally as much as possible
- Reinforcement learning in the future
The tradeoff seems to be:
- RTX 5090 will give training speed but won’t be able to deal with medium/large LLMs(from what I think).
- Project Digits(PD) can run LLMs up to 200B params at the cost of some training speed.
My question is, how slower will Project Digit be as compared to 5090?
And what existing GPU is the Project Digits equivalent to, in terms of speed(apart from it’s memory)?
If it’s slightly slower for training, I would love to be able to run 200B models. But if it’s too much slower for training, I’ll go with the 5090.
RTX 5090 specs:
- AI TOPS: 3352
- Tensor cores: 5th gen
- VRAM: 32 GB DDR7
- Memory bandwidth: 1792 GB/sec
- Memory bus: 512 bit
Project Digits specs:
- Nvidia GB10 Grace Blackwell Superchip with 5th gen tensor cores
- 1 PetaFLOPS of AI performance
- 128 GB unified memory (low powered DDR5x)
- Up to 4 TB NVME storage
- Plus, two of these can be combined to run 405B params models.
Unfortunately, we don’t seem to know the memory bandwidth/bus on the Project Digits.
But here are few things to notice:
The Project Digits is the size of Mac mini which includes everything (storage etc.). No special cooling and no big PSU required.
Whereas the 5090 the GPU alone with fans is bigger than this, plus it requires a big PSU!
So, 5090 must definitely be faster, but how much faster than the Project Digits is what will help decide which one to buy.
While we are at it, also wondering how the Project Digits will compare to the Macbooks with similar unified memory (and price) although most probably I won’t be buying one.
Dear experts, please help me understand the difference/tradeoffs which will help me decide which one to buy. _ /\ _
