• 5 Posts
  • 42 Comments
Joined 3 months ago
cake
Cake day: March 22nd, 2024

help-circle
  • The last part is absolutely false. The Nvidia H100 TDP is like 700W, though ostensibly configurable. The B200 is 1000W. The AMD MI300X is 750W.

    They also skimp on RAM with many SKUs so you have to buy the higher clocked ones.

    They run in insane power bands just to eek out a tiny bit more performance. If they ran at like a third of their power, I bet they would be at least twice as power efficient, and power use scales over nonlinearly with voltage/clock speed.

    But no, just pedal to the metal. Run the silicon as hard as it can, and screw power consumption.

    Other AI companies like Cerebras are much better, running at quite sane voltages. Ironically (or perhaps smartly), the Saudis invested in them.