I have decided to build a new PC.
I was planning to buy Nvidia RTX 5090, but Nvidia has also announced Project Digits marketed as “personal AI supercomputer”.
I need to decide which one to buy before 30th January as the 5090 Founders Edition will be immediately sold out, probably never to be seen again.
My main interests are:
General Deep learning training (primary requirement)
Would love to try training generative AI (both images & text)
Would love to be able to train/fine-tune/run small/large LLMs locally as much as possible
Reinforcement learning in the future
The tradeoff seems to be:
RTX 5090 will give training speed but won’t be able to deal with medium/large LLMs(from what I think).
Project Digits(PD) can run LLMs up to 200B params at the cost of some training speed.
My question is, how slower will Project Digit be as compared to 5090? And what existing GPU is the Project Digits equivalent to, in terms of speed(apart from it’s memory)?
If it’s slightly slower for training, I would love to be able to run 200B models. But if it’s too much slower for training, I’ll go with the 5090.
RTX 5090 specs:
AI TOPS: 3352
Tensor cores: 5th gen
VRAM: 32 GB DDR7
Memory bandwidth: 1792 GB/sec
Memory bus: 512 bit
Project Digits specs:
Nvidia GB10 Grace Blackwell Superchip with 5th gen tensor cores
1 PetaFLOPS of AI performance
128 GB unified memory (low powered DDR5x)
Up to 4 TB NVME storage
Plus, two of these can be combined to run 405B params models.
Unfortunately, we don’t seem to know the memory bandwidth/bus on the Project Digits.
But here are few things to notice:
The Project Digits is the size of Mac mini which includes everything (storage etc.). No special cooling and no big PSU required.
Whereas the 5090 the GPU alone with fans is bigger than this, plus it requires a big PSU!
So, 5090 must definitely be faster, but how much faster than the Project Digits is what will help decide which one to buy.
While we are at it, also wondering how the Project Digits will compare to the Macbooks with similar unified memory (and price) although most probably I won’t be buying one.
Dear experts, please help me understand the difference/tradeoffs which will help me decide which one to buy. _ /\ _
That’s an exciting project! Let’s break down the trade-offs between the RTX 5090 and Project Digits to help you make an informed decision.
RTX 5090
AI TOPS: 3352
Tensor Cores: 5th gen
VRAM: 32 GB DDR7
Memory Bandwidth: 1792 GB/sec
Memory Bus: 512 bit
Project Digits
Superchip: Nvidia GB10 Grace Blackwell
AI Performance: 1 PetaFLOPS
Unified Memory: 128 GB DDR5x
Storage: Up to 4 TB NVMe
Model Capacity: Up to 200B parameters (2 units: 405B parameters)
Key Differences
Memory and Storage: Project Digits offers significantly more unified memory (128 GB vs. 32 GB) and storage (up to 4 TB NVMe vs. none specified for RTX 5090). This is crucial for handling large models and datasets locally.
Model Capacity: Project Digits can handle models up to 200B parameters, and two units can manage up to 405B parameters. RTX 5090, while faster, is limited to smaller models due to its lower memory capacity.
Size and Power: Project Digits is compact and doesn’t require special cooling or a big PSU, making it more convenient for home setups. RTX 5090 is bulkier and needs more power.
Performance: The RTX 5090 is likely faster in raw training performance due to its higher memory bandwidth and specialized tensor cores. However, Project Digits is optimized for inference and running large models locally.
Training Speed Comparison
While exact training speed comparisons aren’t available, Project Digits is expected to be slower in training due to its lower memory bandwidth and focus on inference. However, the difference might not be drastic enough to outweigh the benefits of running larger models locally.
Macbook Comparison
Macbooks with similar unified memory (like the M4 Max with 196GB) are also capable of running large models for inference. However, Project Digits offers more storage and is specifically optimized for AI tasks, making it a better choice for your needs.
RTX 5090 = $2000 while GB10 = $3000 (so for $1000 more u can have 2x RTX 5090)
You will maybe be able to train models but they will have to be significantly smaller than the available VRAM, let’s assume 32GB worth of parameters. At this stage, you might as well get an RTX 5090.
If you want to run inference the GB10 can still fit a model with 50B parameters @ fp16.
Regarding training… I’d say it’s best to manage your hopes but nvidia has proven me wrong before so it’s probably best to wait for the release date, just be careful you don’t get caught up in their misleading marketing
Powered by the NVIDIA GB10 Grace Blackwell Superchip,Project DIGITS delivers a petaflop of AI performance in a power efficient, compact form factor. Inference presumably will run faster on. If the 5x memory ROSE WINE bandwidth figure holds, then token generation would run 5 times faster. The main difference between Giga Texel Shader eXtreme which stands for Ray Tracing Texel eXtreme, is that the latter is better developed to perform the real-time ray-trace technique. As a result, it works according to the real space and provides better imagery and texture in video gaming.
