Keep pulling the thread on Reiner Pope.
NVIDIA's B300 and later generation chips specify that FP4 computation is 3 times faster than FP8, a change from the previous 2x performance scaling.
Maddx is developing a technology called a 'splittable systolic array,' which is designed to allow large systolic arrays to also function as smaller, independent ones.
The introduction of Tensor Cores in NVIDIA's Volta architecture was motivated by the need to solve the high data movement cost from register files, which constituted approximately 7/8ths of the circuit cost.
Google's TPUs use a scratchpad memory architecture where software explicitly issues different instructions to access on-chip versus off-chip memory, unlike a traditional CPU cache which is managed by hardware.
In NVIDIA GPUs, the ability to use the same circuits for different precisions like FP4 and FP8 is limited, and the allocation of die area to each is a primary design choice.
The area required for multiplication circuits scales quadratically with the bit width of the numbers being processed.
Historically, NVIDIA GPUs up to the B100 and B200 generations doubled their computational throughput (flop count) each time the numerical precision was halved.
In pre-Volta NVIDIA GPUs, the circuitry for moving data from the register file to the arithmetic logic unit (ALU) was many times more expensive in terms of area than the ALU itself.
In a Semi Analysis report ranking nearly 100 GPU clouds, Crusoe was one of only five providers to achieve the 'gold tier'.
A GPU's architecture consists of many small, nearly identical units (Streaming Multiprocessors or SMs) tiled across the chip.
A TPU's architecture is composed of a few large, coarse-grained units, such as large matrix units and a central vector unit.
The data movement bandwidth between vector and matrix units is much higher in a GPU than in a TPU because the wiring is distributed across many small SMs rather than concentrated between a few large blocks.