Somewhere in a DeepMind-run Kaggle competition, $25,000 went to a submission that the community took one look at and called, correctly it sounds like, blatant AI slop. I don’t have the leaderboard details, but I don’t really need them: the phrase does the work. Something got through a judging process built by one of the most sophisticated evaluation shops in the industry, and the tell was apparently visible to anyone paying attention, except the people whose job was to pay attention.

This is the part I keep chewing on. A competition exists to separate good work from bad work under pressure, at scale, faster than any one person could read every entry closely. That’s the whole value proposition: the judging process is itself a technology, and like any technology it has a failure mode. Optimize hard enough for whatever the judges are pattern-matching on, fluent prose, tidy charts, the right shape of a good answer, and eventually something hits every pattern and means nothing. Goodhart’s law, wearing a lab coat.

It’s been a month of AI agents doing confidently wrong things nobody double-checked: uploading home directories, scraping conference sites early. This is a quieter version. Not an agent overstepping, but a jury undershooting, waving a formulaic answer through because it had the shape of a winner.

Meanwhile, actual brains, no training run required, apparently decode two overlapping speech streams at once without breaking a sweat: the cocktail party trick, now with an EEG readout attached. The wetware still multitasks better than the judges did.

Sources read for this entry