What If We Stopped Using GPUs? | YC Paper Club
GPU-and-backprop computing is plateauing on energy efficiency, and talks on zero-order sharded training, optical compute, neuromorphics and living neurons explore substrates co-designed with their algorithms.
Watch on YouTube ↗Charlie's summary
- ▶ 1:10
Hardware and optimizers co-adapted: GPUs shifted from FLOPs per joule to memory bandwidth, and efficiency gains have stalled
The host says CNNs were not memory-bound, so Nvidia chased compute per joule. After GPT-2/3, transformers demanded memory capacity and bandwidth because “attention is n squ,” and compute efficiency stopped mattering. Gigaflops per joule has barely improved in two years. The brain runs on about 20 watts, so he argues cost per gigaflop must fall a lot to reach human level.
- ▶ 8:35
Sharded zero-order training (SOMA) scales along a new axis if forward passes become nearly free
The host found SPSA, a finite-difference method needing only two forward passes per perturbation, best among zero-order methods for a 1B LSTM. Gradient-estimate error grows with model size, so big models are infeasible. Sharding data into clusters with a small expert each caps the noise, so about 64 perturbations suffice. He says more shards keep improving results. Paired with cheap forward passes in light, neurons or analog hardware, this could replace backprop.
- ▶ 15:55
Optics wins on physics, but conversion and memory costs eat the advantage
Ilker says photons have about 10,000 times lower loss and bandwidth advantage than electrons, and beams don't interact, so compute is massively parallel. Matrix-vector energy can scale with n rather than n squared. But DAC/ADC conversion, calibration, missing cheap optical memory and hard nonlinear activations dominate the energy budget. Optical computing therefore needs carefully chosen, application-specific uses.
- ▶ 21:48
Co-design for optics: a passive diffractive diffusion model reuses fixed weights to generate images
Ilker's EPFL/Google work makes light propagation take the denoising steps of a diffusion model. Because rewriting optical weights is hard, he split the steps into subunits, each with fixed parameters reused about 100 times. On small datasets like MNIST it showed power-law scaling like digital nets and roughly a seven-times energy advantage, with training still done by backprop on a digital twin. The next milestone is a billion-parameter system.
Give your email to unlock the rest of the key points — for every episode.
Do more with Charlie
Free — sign in with your email, no password.