data - Distributed Thoughts

Distributed Thoughts

Sign in Subscribe

data

A collection of 4 posts

The Loop Is Only as Good as the Metric

The Loop Is Only as Good as the Metric

On Thursday I wrote about Karpathy's autoresearch, the 630-line training loop that runs 100 ML experiments overnight on a single GPU while you sleep. The post generated a lot of conversation, and most of it centered on the automation: agents doing research, models training themselves, the future of

The Model Is the Byproduct

The Model Is the Byproduct

Last Friday, Andrej Karpathy open-sourced a 630-line Python script and went to bed. By morning, an AI agent running on a single GPU had completed roughly 100 complete LLM training runs, each lasting exactly five minutes, autonomously modifying the neural network architecture, the optimizer, the hyperparameters, evaluating the results, keeping

Will I Make It To The Restaurant Before The Soup Dumplings Get Cold? (And Other Problems In Machine Learning)

Will I Make It To The Restaurant Before The Soup Dumplings Get Cold? (And Other Problems In Machine Learning)

I'm chronically late. Not because I want to be rude - I feel terrible about it every single time - but because I'm catastrophically bad at predicting how long it takes to get anywhere. Turns out machine learning algorithms have the exact same problem. Here'

WTF Do You Spend $300B On?: The Oracle-OpenAI Deal Decoded

data infrastructure

WTF Do You Spend $300B On?: The Oracle-OpenAI Deal Decoded

Discover the challenges and expenses behind OpenAI's $300 billion Oracle deal, revealing a true AI cost breakdown: 80% data prep, 20% training