Nvidia Project Digits vs RTX 5090 dilemma

Deep Learning
1

Hi all,

I have decided to build a new PC.
I was planning to buy Nvidia RTX 5090, but Nvidia has also announced Project Digits marketed as “personal AI supercomputer”.
I need to decide which one to buy before 30th January as the 5090 Founders Edition will be immediately sold out, probably never to be seen again.

My main interests are:

  1. General Deep learning training (primary requirement)
  2. Would love to try training generative AI (both images & text)
  3. Would love to be able to train/fine-tune/run small/large LLMs locally as much as possible
  4. Reinforcement learning in the future

The tradeoff seems to be:

  1. RTX 5090 will give training speed but won’t be able to deal with medium/large LLMs(from what I think).
  2. Project Digits(PD) can run LLMs up to 200B params at the cost of some training speed.

My question is, how slower will Project Digit be as compared to 5090?
And what existing GPU is the Project Digits equivalent to, in terms of speed(apart from it’s memory)?

If it’s slightly slower for training, I would love to be able to run 200B models. But if it’s too much slower for training, I’ll go with the 5090.

RTX 5090 specs:

  • AI TOPS: 3352
  • Tensor cores: 5th gen
  • VRAM: 32 GB DDR7
  • Memory bandwidth: 1792 GB/sec
  • Memory bus: 512 bit

Project Digits specs:

  • Nvidia GB10 Grace Blackwell Superchip with 5th gen tensor cores
  • 1 PetaFLOPS of AI performance
  • 128 GB unified memory (low powered DDR5x)
  • Up to 4 TB NVME storage
  • Plus, two of these can be combined to run 405B params models.

Unfortunately, we don’t seem to know the memory bandwidth/bus on the Project Digits.

But here are few things to notice:
The Project Digits is the size of Mac mini which includes everything (storage etc.). No special cooling and no big PSU required.
Whereas the 5090 the GPU alone with fans is bigger than this, plus it requires a big PSU!
So, 5090 must definitely be faster, but how much faster than the Project Digits is what will help decide which one to buy.

While we are at it, also wondering how the Project Digits will compare to the Macbooks with similar unified memory (and price) although most probably I won’t be buying one.

Dear experts, please help me understand the difference/tradeoffs which will help me decide which one to buy. _ /\ _

1 Like
2

That’s an exciting project! Let’s break down the trade-offs between the RTX 5090 and Project Digits to help you make an informed decision.

RTX 5090

  • AI TOPS: 3352
  • Tensor Cores: 5th gen
  • VRAM: 32 GB DDR7
  • Memory Bandwidth: 1792 GB/sec
  • Memory Bus: 512 bit

Project Digits

  • Superchip: Nvidia GB10 Grace Blackwell
  • AI Performance: 1 PetaFLOPS
  • Unified Memory: 128 GB DDR5x
  • Storage: Up to 4 TB NVMe
  • Model Capacity: Up to 200B parameters (2 units: 405B parameters)

Key Differences

  1. Memory and Storage: Project Digits offers significantly more unified memory (128 GB vs. 32 GB) and storage (up to 4 TB NVMe vs. none specified for RTX 5090). This is crucial for handling large models and datasets locally.
  2. Model Capacity: Project Digits can handle models up to 200B parameters, and two units can manage up to 405B parameters. RTX 5090, while faster, is limited to smaller models due to its lower memory capacity.
  3. Size and Power: Project Digits is compact and doesn’t require special cooling or a big PSU, making it more convenient for home setups. RTX 5090 is bulkier and needs more power.
  4. Performance: The RTX 5090 is likely faster in raw training performance due to its higher memory bandwidth and specialized tensor cores. However, Project Digits is optimized for inference and running large models locally.

Training Speed Comparison

While exact training speed comparisons aren’t available, Project Digits is expected to be slower in training due to its lower memory bandwidth and focus on inference. However, the difference might not be drastic enough to outweigh the benefits of running larger models locally.

Macbook Comparison

Macbooks with similar unified memory (like the M4 Max with 196GB) are also capable of running large models for inference. However, Project Digits offers more storage and is specifically optimized for AI tasks, making it a better choice for your needs.

Best regards,
Monica
BOGO Subway Code