Lilian Weng spent seven years as OpenAI's VP of Research and Safety. She co-authored the safety papers that shaped how we think about alignment. Now she runs Thinking Machines Lab. Last week she posted the most honest technical analysis of scaling laws I've read in years, and the AI community is still arguing about it.

The post is called "Scaling Laws, Carefully," and it does something deceptively simple: it traces the entire history of scaling law research from Hestness et al. through Kaplan to Chinchilla to the present day, showing where the models disagree, why they disagree, and what that means for everyone betting billions on "just make it bigger."

Kaplan scaling law chart showing power-law relationship between compute, parameters, and loss

Here's the core tension. Kaplan et al. at OpenAI in 2020 concluded that for optimal performance, you should grow model size faster than data. Their formula suggested N_opt proportional to C^0.73. Two years later, DeepMind's Chinchilla paper proved the opposite: you should scale model size and training data equally, with N_opt proportional to C^0.5. Both teams used rigorous methodology. Both got clean power-law curves. Both can't be right at the same time.

Weng's explanation is elegant. The discrepancy comes from two things: scale and embedding parameters. Kaplan's experiments ran on smaller models where embedding layers made up a significant chunk of total parameters. When you account for non-embedding parameters only, the local power-law exponent gradually converges toward Chinchilla's estimate as compute increases. In other words, Kaplan wasn't wrong. He was measuring from a regime where the math looks different.

This matters because the entire industry has been building on these curves. Every frontier lab uses scaling law extrapolation to plan training runs. If the exponent is wrong, the cost estimates are wrong, the timeline predictions are wrong, and the "we need X billion dollars for the next model" pitches are wrong.

Chinchilla scaling law results showing compute-optimal training frontier

The post also tackles data scarcity head-on. We're running out of unique high-quality text. When you repeat training data, returns diminish exponentially. Muennighoff et al. proposed a decay model for token value, and Lovelace et al. added an explicit overfitting penalty: larger models are more sensitive to data repetition. Strong weight decay helps, but it doesn't eliminate the problem. This is the real "data wall" and it's not about token count. It's about quality and diversity.

What the HN discussion reveals is a deeper philosophical split. Some commenters argue that language has an irreducible entropy floor, a hard cap on how well any model can predict the next token. Others counter that the interesting intelligence lives in the final, hardest tokens, the ones near the floor where prediction requires genuine reasoning rather than pattern matching. The argument about whether coding languages have "WAY less entropy" than natural language is particularly interesting, because it suggests that LLM capabilities might plateau at different rates for different domains.

One comment that stuck with me: "even if you're a fraction of a nat above the floor, you could have enormous headroom in performance left because there are still rare tokens amongst the irreducible noise that require so much capability to predict." This reframes the entire debate. The question isn't whether we'll hit the entropy floor. It's whether the last few nats of loss contain enough signal to keep scaling worthwhile.

Weng's practical warning is the most important part. Scaling law fitting is fragile. Small variations in loss precision, rounding, or optimizer choice produce wildly different extrapolations. The "data wall" isn't just about running out of tokens. It's about what counts as good data. Deduplication, filtering, and reweighting all change the curve. And the laws assume everything else stays constant: architecture, optimizer, learning rate schedule. If any of those shift during training, the scaling law you fitted on small runs becomes unreliable.

This is not a "scaling is dead" argument. It's a "scaling is harder than you think" argument. The curves hold, but the parameters you need to measure them correctly keep changing. Every lab extrapolating from small runs to billion-dollar training budgets should be reading this post carefully.

Sources