final outputs

eval_iclmargin_fb100_tg_fable5_xhigh_comp100k_20260702_ugly-anime · final judges 0/1 · 27,958 chars · 4,285 words · /home/ubuntu/code/lesswronger/data/agent_runs/eval_iclmargin_fb100_tg_fable5_xhigh_comp100k_20260702_ugly-anime/submission.md

> American adult TV animation is ugly.
> It has been ugly for decades, and nobody involved seems to notice, or to mind.
> But ugliness comes in 2 kinds: ugliness as a committed artistic choice, which is legitimate, and ugliness from laziness which has learned to call itself an esthetic; and the two can be told apart, without trusting anyone's taste (mine included), by watching what a show does when its budget rises or its technology improves.
> The committed shows intensify (*South Park* pays real money to keep its construction-paper look); the constrained shows upgrade (*One Piece*); the lazy ones stagnate (*The Simpsons* has held onto one drawing for 38 years, only smoother).
> Cheapness doesn't explain it: character design is nearly free, the richest franchises are the ugliest ones, and 1960s Japan turned worse poverty into an esthetic instead.
> The ugliness survives because it is *useful*, as camouflage---against a baseline that already looks bad, corner-cutting is invisible---and like other exhausted fashions, cargo-cult modernism or fourth-generation comix imitation, it will not be argued away, only outlived.

I recently lost an evening to [*Dr. Katz, Professional Therapist*](https://en.wikipedia.org/wiki/Dr._Katz,_Professional_Therapist) (Comedy Central, 1995; on DVD since 2007, though everyone actually watches the YouTube uploads), and was surprised twice: the comedy holds up, and the images are nearly unwatchable.
The reason is "[Squigglevision](https://en.wikipedia.org/wiki/Squigglevision)": the outline of every character boils and wriggles, constantly, whether or not anything is happening on screen.
Animators call this "line boil", and it is normally a *defect*---the small tremor you get when successive hand-drawn frames don't quite register---which studios historically spent money to suppress.
Squigglevision industrialized it: draw each pose a few times, not quite identically, loop the copies, and now the eye has something to chew on while two people sit in chairs and talk for 22 minutes.
It is less a kind of motion than an excuse for the absence of motion---Chuck Jones's sneer about TV cartoons being "illustrated radio" made literal, with a screensaver running on top.
(Tom Snyder Productions was proud enough of this to patent it, US #6,373,492.)

What strikes me now is how *local* the technique was.
As far as I can tell, Squigglevision existed essentially nowhere else on earth: no Japanese squiggle tradition, no European one, no squiggle school of art-house shorts, nothing---the nearest relatives are deliberate line-boil homages like *Ed, Edd n Eddy* (1999), which at least knew what old-cartoon effect they were quoting.
Even its own users abandoned it as soon as they could afford to: [*Home Movies*](https://en.wikipedia.org/wiki/Home_Movies_(TV_series)) used Squigglevision for its first season in 1999, switched to ordinary Flash animation on moving to Adult Swim, and I am not aware that anyone mourned.
A visual style which arises in one country's television industry, spreads to nothing, and gets dropped by its own practitioners the moment dropping it is cheap, is not an esthetic discovery; it is a symptom of something in the local economics.

Squigglevision is only the limiting case, of course, which is what makes it a good specimen.
I have watched Japanese animation for most of 2 decades now, and every return trip to American TV animation produces the same small shock: not at the writing (often excellent), but at the *ugliness*, which nobody involved appears to regard as a problem, or even to notice.
Flip between a random mid-tier TV anime (nothing prestigious; whatever aired that season to sell figurines) and an acclaimed American adult cartoon, and one of them is visibly trying to put an attractive image on the screen, often failing but trying, while the other stopped trying before I was born.
Nor will "TV animation is just like that" do as an answer, because it isn't, elsewhere: French TV animation, made if anything for less money, doesn't look like this---Ankama's [*Wakfu*](https://en.wikipedia.org/wiki/Wakfu_(TV_series)) (2008) was made in Flash, the same Flash that gets blamed for the ugliness of American web-era cartoons, and is genuinely pretty.
To be precise about the accusation: I mean the adult prime-time cartoon sitcom and its cable/streaming descendants---the *Family Guy* assembly line, the legions of Adult Swim and Comedy Central originals, the flat-lit Netflix adult shows which all seem to share one art director.
(American *children's* television went somewhere else in the 2010s: *Adventure Time*, 2010, and its descendants are drastically simplified but cute, a different and defensible trade.)
So the adult sector is older, richer, more prestigious, and uglier. Which already hints that money is not the variable doing the work.

So...

why?

## Ugly Can Be Good

Start with the necessary concession, because it is large: ugly is not a synonym for bad.
Ugliness can be a perfectly valid esthetic, and some of the most alive animation ever made is hideous on purpose.
*Ren & Stimpy* pauses its rubbery slapstick for painted close-ups of nose hairs and bloodshot eyeballs, rendered with loving airbrushed care: ugliness as a destination, something the show spends money to arrive at.
Bakshi built half a career on rotoscoping precisely because of its queasy not-quite-humanness---and the rotoscope, note, was among other things a *cost-saving* device, a cheapness he turned into the point, the productive-constraint trick as old as art.
The Soviet-bloc animators made a specialty of this register: Priit Pärn's [*Breakfast on the Grass*](https://en.wikipedia.org/wiki/Breakfast_on_the_Grass) (1987) looks scratched into the film during a nervous breakdown and is the better for it.
And the whole underground-comix visual tradition---Crumb's crude line, the deliberately repellent crosshatch---is ugly the way a punk record is loud: as the message, not as noise on top of it.
(Some of this I cannot actually sit through---I bounced off Pärn more than once, and lasted about 2 episodes of *Xavier: Renegade Angel*---but that is a fact about my stomach, not a verdict on the art; these things deserve to exist.)

## Defectpunk

The committed genre deserves a name, so call it *defectpunk*: the esthetic that takes the defect itself---the boil, the cheapness, the brokenness---as its subject matter, and commits.
*Superjail!* wants you to flinch; *Xavier* (2007) runs its intentionally broken CGI straight at you like a dare; these shows pursue revulsion the way Disney pursued charm, uncompromisingly.
The test that separates defectpunk from the other kind, since screenshots of the two can look identical: ask what the show would do with more money.
Hand the *Superjail!* people a doubled budget and you would not get a rounder, more pleasant *Superjail!*; you would get *more*---more viscera, more hell-machinery per frame, more of whatever wrongness they were already reaching for.
*South Park* is the case that convinced me the test measures something real: it began as literal construction paper (the 1995 "Spirit of Christmas" short), went digital almost immediately, and then spent 25 years lovingly *simulating* construction paper---the paper was abandoned for Alias PowerAnimator and later Maya, tuned to reproduce the paper grain and the popped, stop-motion-ish movement---through a theatrical movie that kept the look, through budgets that could have bought any look it wanted, the pipeline also buying the famous 6-day topical production cycle (cf. the documentary [*6 Days to Air*](https://en.wikipedia.org/wiki/6_Days_to_Air), 2011, mostly about render deadlines and sleep deprivation).
I started out wanting to file *South Park* with the lazy shows, and found I couldn't.
And the anime industry demonstrates that crude *design* and cheap *animation* are separable axes: ONE, the webcomic author behind *One-Punch Man* and *Mob Psycho 100*, draws in a style politely described as artless---stick figures with jug ears---and the industry handed his work to some of the best animators alive, *Mob Psycho 100* (Bones, 2016) keeping the lumpy homely designs and animating them with a lavishness most handsome shows never get.
Crude drawing, lovingly animated, reads as a choice.
Nobody has ever done this for *Family Guy*, and it is worth pausing on why the idea sounds absurd.

The *Family Guy* look is not defectpunk; it is the other thing wearing defectpunk's clothes.
When nobody with power over a production cares what it looks like, what settles out is not neutral plainness but something with a nasty economic property which keeps it stable: against a baseline that already looks bad, cutting corners is undetectable.
A beautiful show, you see, has to keep paying to stay beautiful---every dropped frame shows up against the standard the show itself set---while a show that looks bad has purchased, with its ugliness, a standing license to get lazier.
(One cannot blame the animators, most of whom would surely rather be doing better work; it is just what the incentive structure rewards once the baseline is in place.)
The camouflage would explain why the style is local: it is an equilibrium of one industry's cost structure (an inadequate equilibrium in Yudkowsky's sense---no individual show profits by defecting to beauty, because the audience has been trained not to expect it), not a discovery about images, and discoveries travel.
And here is the supporting observation that first made me suspicious: if these ugly styles were genuinely *good*, why are they idiosyncratic to one country's TV animation, and why does every adaptation flee them?
When these shows graduate to film or games, the look nearly always moves *away* from the original---*The Simpsons Movie* (2007), on a reported ~$75M, added shading, lighting, & camera movement the series never had, and the game adaptations render everything in cleaned-up 3D.
Nobody, handed real money and asked to make the property maximally appealing, chose to keep the wriggle.
(*South Park*'s movie, note, kept the cutout look. There was something there to keep.)

## Cheap Is Not Ugly

Whenever I complain about this, I get the budget defense, invariably, and it runs something like:

> "TV animation is brutally cheap. You have to fill 22 minutes a week on a small fraction of a film budget, forever. Of course it looks bad---the money isn't there. Stop expecting Ghibli on a sweatshop schedule, and be grateful the shows exist at all."

The schedule part is true.
The rest explains less than it appears to, and the arithmetic surprised me when I finally did it.
Most of these shows are ugly *as static images*: pause any frame---no motion involved now, so the weekly grind is irrelevant---and the character designs themselves are unpleasant: the proportions, the faces, the dead margarine palette.
But character design is close to free.
A model sheet is drawn once, by a handful of people, and amortized over hundreds of episodes. It costs the same to photocopy a beautiful design for 30 years as an ugly one.
Whatever explains an ugly model sheet, it cannot be the per-episode budget---the model sheet, after all, barely appears in the per-episode budget.

Nor was the American industry ever naive about cheapness---it invented most of the tricks.
When MGM abruptly shut its cartoon studio in 1957 (the story is that [Hanna & Barbera](https://en.wikipedia.org/wiki/Hanna-Barbera), mid-career Oscar winners, were let go by phone), the pair re-tooled around "planned animation" for television: the commonly cited figures are ~3,000 drawings per half-hour against ~25,000 for full theatrical animation, an ~88% cut in drawing labor.
(The mechanics are worth spelling out, because they are clever: 22 minutes at 24fps is ~31,700 frames, but you shoot each drawing for 2 or 3 frames, hold a cel and slide it across a background instead of redrawing it, and put the mouths on a separate cel level so that during dialogue only the jaw is redrawn---ie. the head is severed from the body at the collar, and the seam has to be hidden somewhere: hence the neckties, collars, & necklaces on every Hanna-Barbera character.)
It conquered TV---*Ruff and Reddy*, *Huckleberry Hound*, then *The Flintstones* (1960) in prime time, sponsored by Winston cigarettes, Fred and Barney puffing away in the ad bumpers---and anyone who watched the syndicated descendants as a child remembers the tells: eg. the same lamp and window scrolling past 4 times behind a running Shaggy.
By 1983, Filmation's *He-Man* was recycling its stock library so aggressively that connoisseurs can recite the reused shots from memory.
Cheapness was a solved problem with known esthetics (for a while, in the UPA-influenced early 1960s, the flatness even had graphic charm; cf. Amid Amidi's [*Cartoon Modern*](https://www.amazon.com/Cartoon-Modern-Animation-Design-Fifties/dp/0811847314) (2006), which documents how seriously that decade took its design---startling to page through now, like finding out your grandfather dressed better than you do), and the industry knew exactly what it was buying.

Meanwhile the Japanese industry, starting far poorer, was forced to invent the *other* half of the toolkit: how to make the cheapness beautiful.
Tezuka's Mushi Pro sold [*Astro Boy*](https://en.wikipedia.org/wiki/Astro_Boy_(1963_TV_series)) to broadcasters in 1963 famously below cost---the figure usually cited is ¥550,000 an episode, about $1,500 at the era's fixed ¥360 rate, although the memoir literature does not quite agree on the number (Jonathan Clements's [*Anime: A History*](https://www.amazon.com/Anime-History-Jonathan-Clements/dp/1844573907), 2013, spends pages untangling the accounting); everyone agrees it was below cost, and the industry complains about the precedent to this day---and Mushi survived on what became known as the "bank": animate your best sequences once, well, and reuse them forever.
(Hence the lavish transformation sequence recurring in every magical-girl episode, which audiences love rather than resent---the bank footage is often the most beautiful thing in the show. That is the point.)
Shoot on threes---8 drawings a second instead of 24, marked right on the exposure sheet---and spend the savings on held poses actually worth holding. Above all, buy good background paintings, which are mostly a one-time cost per location and carry a startling fraction of perceived beauty.
Two industries, the same scarcity, most of the same tricks; one spent the savings building a visual culture of poses, banks, & painted light, and the other banked the money.

On the modern shows, the budget excuse collapses entirely, because ugly animation coexists happily with enormous budgets.
A TV anime episode has generally been reported to cost somewhere around $100--200K. The figures vary by era and source and I would not lean hard on any one number, but the order of magnitude is not in dispute.
Late-period *The Simpsons*, at the time of the 2011 salary standoff---when Fox announced it could not keep producing the show "under its current financial model"---was reported to cost several million dollars an episode, the 6 principal voice actors making ~$400K each per episode before the pay cuts.
6 × $400K = $2.4M per episode in principal-cast salary alone: roughly the animation cost of an entire 13-episode anime season, spent before a single *Simpsons* frame is drawn.
Or per second of screen time: $4--5M over ~21 minutes of show is $3,200--4,000 a second, against the anime episode's ~$80--160 a second; even on the most charitable accounting---say only a fifth of the American budget ever touches a drawing---the pictures are getting 5--6× the money per second, and looking worse.
(An earlier draft of this paragraph said "an order of magnitude"; redoing the numbers gave 5--6×---weaker, but the sign never flips under any accounting I could construct.
To be fair, this cuts both ways: most of a *Simpsons* episode's cost is evidently not going to pictures at all, so the raw 20--50× budget ratio overstates the difference in drawing budgets, and I could not find a trustworthy animation-only line item for any prime-time American show, everything public being entangled with cast, licensing, & residuals.)
But the direction of the entanglement is itself the finding: on the most profitable animated property in television history, the pictures are the residual claimant---the thing the money is *not* for.

And *The Simpsons* is the cleanest case history, because you can watch the money arrive decade by decade and see what it does.
The show began as Matt Groening's own rough drawings for the *Tracey Ullman Show* shorts (the first, ["Good Night"](https://en.wikipedia.org/wiki/Good_Night_(The_Simpsons)), aired April 1987)---drawings he has said he submitted assuming the animators would clean them up.
Instead, the Klasky-Csupo staff traced them!
Even the famous yellow was reportedly the suggestion of the colorist, Gyorgyi Peluce, on the theory that channel-flippers would stop on something that looked wrong: the design language of a multi-billion-dollar franchise is a stack of accidents that tested well.
From there the budget grew by well over an order of magnitude---half-hour series by December 1989, overseas in-betweening through AKOM in Seoul, digital ink-and-paint in 2002, HD in February 2009, the switchover celebrated, characteristically, with a redesigned title sequence rather than a redesign of anything else---while the technology of drawing things well got relentlessly cheaper.
Through all of it the core design persisted untouched: the overbites, the bulging eyes, the yellow, every dollar going into polish (cleaner lines, smoother in-betweens, glossier color) and none into transformation.
And we know better was possible, because the same man did better next door: *Futurama* (1999, animated by Rough Draft) has genuinely strong design and animation, including a knowing homage to the rubber-hose cartoons of the 1920s--30s---a look chosen deliberately, out of the medium's own history, for its own sake.
It is at least suggestive, though I would not push it harder than that, that *The Simpsons*' best-regarded seasons cluster around the years *Futurama* was being developed and launched.
Nor is this a unique Groening capacity, because plenty of American TV series have gotten striking looks out of ordinary TV budgets, so the medium forces nobody's hand: *Batman: The Animated Series* (1992) did it partly by painting its backgrounds on black paper; Genndy Tartakovsky has spent a career, *Samurai Jack* (2001) through *Primal* (2019), demonstrating that television money buys beauty if anyone asks; *Avatar: The Last Airbender* (2005) imported anime's visual grammar onto a Nickelodeon budget and was rewarded with a fanbase that has outlived 2 generations of network executives.

## Cargo-Cult Ugliness

So the ugliness is not necessity; it is a default which has been retroactively promoted to an "esthetic". That promotion has a natural history, familiar from other arts, and it is the contemptible part: not the ugliness itself, but the self-satisfied mediocrity that mistakes a mindless default for a deliberate style.
The underground comix of the 1960s had substance---Crumb's crude line ([*Zap Comix*](https://en.wikipedia.org/wiki/Zap_Comix), 1968) was expressive of real transgression, and the ugliness was earned---then came imitators who copied the crudeness without the transgression, and imitators of the imitators, each generation keeping more of the mannerism and less of the motive, until the ugliness became a convention transmitted to people who could not tell you what it had been for.
Indeed, a great deal of American TV animation is, visually, a fourth-generation photocopy of *Zap Comix* made by people who have never read it.
(John Kricfalusi---whose *Ren & Stimpy* is my own exhibit for good-ugly---spent years on his blog raging that modern TV cartoons were designed by people who could not draw and did not know it. The messenger had his problems, but on the object level he wasn't wrong, and it is telling that the industry's most famous defender of ugliness thought the ambient product was ugly in the *bad* way.)
Architecture ran the same experiment at civilizational scale: early modernism was a deliberate assault on the viewer, and Adolf Loos, in ["Ornament and Crime"](https://en.wikipedia.org/wiki/Ornament_and_Crime) (1908), was at least making an argument---"The evolution of culture is synonymous with the removal of ornament from objects of daily use"---wrong, I think, but an argument, from a man who could have built ornament and refused to.
What followed was cargo-cult modernism, glass boxes multiplied by habit and cost accounting, reproducing the outward forms of a radical gesture whose content nobody in the room remembered.
The discouraging lesson is that you cannot argue an exhausted fashion out of existence, because there is no argument on the other side left to engage---only inertia. Such fashions are not refuted, only outlived.

## Proof by Stagnation

All of which yields a test that requires trusting nobody's taste; call it *proof by stagnation*.
When budgets rise or technology cuts the cost of quality, a show can do 1 of 3 things:

#. fix its look (the ugliness was constraint);
#. intensify it (the ugliness was conviction);
#. stay exactly where it is, only smoother.

The third convicts.
*The Simpsons*, by this test, convicts itself: decades of budget and 4 technological revolutions produced no change at all except polish.
Worse: the polish subtracted something.
Groening's early artwork, for all its crudity, had a scratchy feral vividness---early Bart really does look like a delinquent drawn by a delinquent---and the modern show is that same ugliness with the life sanded off.
The comparison is reproducible in 2 browser tabs: search YouTube for the 1987 "Good Night" short and pause on Homer's face, then pause any current-season clip on the same face.
The 1987 outline wobbles with the pressure of an actual pen held by an actual hand; the current outline is a uniform vector stroke that could have been drawn by nobody in particular; and it is the same face, held for 38 years, only... smoother.

What does it look like when early ugliness really was constraint?
It looks like [*One Piece*](https://en.wikipedia.org/wiki/One_Piece_(1999_TV_series)), which began in 1999 as a fairly poor-looking production, rushed and off-model under Toei's weekly schedule; to be honest, judged by a 2001 screenshot, I wouldn't've filed it anywhere but with the lazy.
But when the chance came, the show drastically upgraded: the Wano arc (from episode 892, aired 2019-07-07) arrived with an overhauled look---new color design, looser layouts, digital effects---and produced genuinely spectacular sakuga, from a franchise so commercially bulletproof it could have coasted on merchandising forever.
(Episode 1015, directed by Megumi Ishitani, is the exhibit fans pass around. The individual cuts get clipped, catalogued, & attributed to individual key animators on [Sakugabooru](https://www.sakugabooru.com/), sometimes by recognizing an animator's drawing style frame-by-frame---a level of connoisseurship American TV animation has never had occasion to generate.)
The upgrade shows the people inside had spent 20 years wanting to reach higher. The earlier poverty was constraint (scheduling, almost certainly), and the moment it relaxed, quality shot up.
That is what constraint looks like when it lifts. Nothing ever lifted at *The Simpsons*, because nothing was pushing.

The proof-by-stagnation ledger, compiled in one place (corrections welcome, especially non-US/non-Japan cases, which I know least well):

| Show | Ugly at launch | Constraint loosened | The look afterwards |
|------|----------------|---------------------|---------------------|
| *Home Movies* | Squigglevision (UPN, 1999) | Adult Swim pickup, 2001 | squiggle dropped immediately; nobody mourned |
| *One Piece* | rushed, off-model (Toei, 1999) | Wano arc, 2019 | overhauled: new color design, layouts, showpiece sakuga |
| *South Park* | construction paper (1995 short) | movie money 1999; software since | kept the cutout look, on purpose, at real expense |
| *The Simpsons* | traced Groening sketches (1987) | digital 2002, HD 2009, $4--5M/episode | same model sheet, smoother |
| *Futurama* | (control: launched handsome, 1999) | --- | look chosen from cartoon history, kept |

The obvious reply is that I am cherry-picking.
Sturgeon's law applies on both shores---the median TV anime is a rushed, off-model advertisement for merchandise, and citing Wano-arc sakuga against *Family Guy* compares Japan's highlights to America's median.
Guilty, partly; but the median-to-median comparison still goes the same way, because the median anime fails in a revealingly different direction: off-model faces are failed attempts at an appealing face, and incompetent pursuit of beauty produces different-looking wreckage than indifference to it.
A better objection: maybe the ugliness is functional the way a laugh track is functional, announcing "adult comedy---no children, nothing sacred, do not expect Disney"; the cartoon sitcom descends from newspaper strips and underground comix, not from feature animation, and inherited their look as a genre badge.
I partly believe this one, but it explains the origin better than the persistence---*Futurama* is legibly an adult comedy while being handsome, anime solved signaling-adultness decades ago (eg. blood), and a signal everyone has sent for 30 years excludes no one; it is just a habit.
(I have also never met anyone who claims to *like* the look per se; its defenders defend the writing.)
And maybe it is just me---decades steeped in one visual culture, receptors calibrated to it, and "American cartoons are ugly" is what miscalibration feels like from the inside.
I can't fully rule that out, which is why everything above leans on behavior rather than judgment: not "I find it ugly" but "watch what the custodians of these shows do when handed money"---and what they do, over and over, is spend it on anything but the look.

The issue, in the end, is taste, not money; and taste cannot be bought retroactively.
So I don't expect the equilibrium to be argued down, least of all by an essay.
What might actually move it: the animators now coming up grew up on Toonami and Tartakovsky as much as on the domestic default, and when an American company recently wanted its game franchise to look spectacular on television---Riot, with 2021's [*Arcane*](https://en.wikipedia.org/wiki/Arcane_(TV_series))---it went to a French studio, Fortiche, and took home the Emmy for Outstanding Animated Program.
I would take an even-odds bet that in 20 years the wriggle and its cousins read as datedly as a laugh track---and that no one will be able to say what the wriggle was *for*.
("It was cheaper, and we stopped seeing it" is a true answer; it is just not one anyone enjoys giving.)
In the meantime, *Primal* is right there.

(This essay is, of course, a complaint about pictures which contains no pictures. Chuck Jones would know what to call it.)

# External Links

-   **Discussion**: HN; Reddit

-   [Sakugabooru](https://www.sakugabooru.com/) (the sakuga database)

-   [*Cartoon Modern: Style and Design in Fifties Animation*](https://www.amazon.com/Cartoon-Modern-Animation-Design-Fifties/dp/0811847314), Amid Amidi 2006

-   [*Anime: A History*](https://www.amazon.com/Anime-History-Jonathan-Clements/dp/1844573907), Jonathan Clements 2013

-   ["Ornament and Crime"](https://en.wikipedia.org/wiki/Ornament_and_Crime), Adolf Loos 1908

**See Also**:

-   Culture Is Not About Esthetics

-   The Melancholy of Subculture Society

-   Death Note: L, Anonymity & Eluding Entropy

-   Evolution as Backstop for Reinforcement Learning

```
Return to blog index
```

eval_iclmargin_fb100_tg_fable5_xhigh_comp100k_20260702_rocky-road · final judges 0/0 · 23 chars · 2 words · /home/ubuntu/code/lesswronger/data/agent_runs/eval_iclmargin_fb100_tg_fable5_xhigh_comp100k_20260702_rocky-road/submission.md

(missing submission.md)

eval_iclmargin_fb100_tg_fable5_xhigh_comp100k_20260702_novelty-net · final judges 0/1 · 24,896 chars · 3,734 words · /home/ubuntu/code/lesswronger/data/agent_runs/eval_iclmargin_fb100_tg_fable5_xhigh_comp100k_20260702_novelty-net/submission.md

# Novelty Nets: Classifier Anti-Guidance

<!-- created: 2024-02-23 -->
<!-- modified: 2024-02-26 -->
<!-- status: notes -->
<!-- confidence: possible -->
<!-- importance: 4 -->

> Generative model samples are 'same-y': each sample may be flawless, but collectively they all look alike, and neither better prompting nor brute-force resampling fixes it, because the diversity of the model far exceeds the diversity of its samples.
> Sameness is a *sampling* problem, not a model problem.
> [Novelty search](https://www.cs.ucf.edu/eplex/papers/lehman_ecj11.pdf "Abandoning Objectives: Evolution Through the Search for Novelty Alone") gives the right objective (reward each result for differing from all prior results), but as an outer loop around a generative model it is futile: generation is ~99% of the compute (~10s of GPU-time per sample vs ~1ms to embed & compare it), and remains novelty-blind.
> No cleverness in filtering outputs can create diversity the sampler never emitted.
>
> I propose building the novelty into generation itself, via a small adapter network (a **novelty net**) trained online to predict whether a sample has been generated before.
> Used during sampling as a weak auxiliary loss, it steers each new sample *away* from everything generated so far: anti-guidance, the mirror image of [classifier guidance](https://arxiv.org/abs/2105.05233 "Diffusion Models Beat GANs on Image Synthesis").
> Each finished sample is immediately trained into the net as 'already generated', so the next sample must land somewhere new.
> The generator stays frozen, and there is no external database---just a few million parameters of compressed history.
> Because the net only needs to *memorize* the past, not generalize, a small cheap MLP may suffice.
>
> The same trick scales to a service-wide novelty net trained on all users' samples plus the original training corpus, mitigating both training-data regurgitation and the emergence of a house style like the 'Midjourney look'.

Anyone who has ground through a few hundred [Midjourney](!W) or [Stable Diffusion](!W) samples (as I regularly do, generating illustrations) knows the phenomenon: each sample is fine (well-composed, well-rendered, nothing you could point to as a defect), and yet they are all somehow the *same*.
The same faces, the same palettes, the same lighting, the same 3⁄4 portrait composition, the same tasteful [bokeh](!W).
After 20 samples you are bored; after 50, numb.
The samples are not getting worse; you are getting wiser.

The stock answer is that you are holding it wrong: specify a style, a medium, an era, a mood.
True as far as it goes (I always do), but the sameness merely recurs one level down, fractally.
Ask for '[ukiyo-e](!W)' and you get the model's one house ukiyo-e, over and over.
Ask for 'German Expressionist linocut' (a style I like partly because it seems still unpolluted by lazy users), and you get its house Expressionism.
Within any category you can name, the samples re-converge on a new mode, and you are back where you started, only more specifically bored.

And prompting harder eventually hits the real wall, which is words.
Creative needs are notoriously "[I know it when I see it](!W)": the user may lack the vocabulary for what they want, and, worse, may not yet *know* what they want, and will not know until it is in front of them.
(If you could write down exactly the image you wanted, you would hardly need to explore---the reason to generate 50 samples is precisely that you can't.) So what a user needs early on is not 50 polished variants of the modal image, but a *spread*: samples as mutually-different as possible, along as many dimensions as possible, so that recognition can do the work that specification cannot.
Homing in afterwards is a solved problem (variations, img2img, [inpainting](!W), remixing all work well).
It is the diverse [contact sheet](!W "Contact print") at the start that you cannot get.

There is, I suspect, a mirror of this problem inside the model.
Models, like people, are better at recognizing a good or novel finished sample than at producing one.
And mid-generation it is worse than that: halfway through a [denoising trajectory](!W "Diffusion model"), 'novel' and 'bad' look identical (both are just low-probability), and a model which cannot tell which kind of unlikely it is looking at does the statistically-sensible thing and retreats toward the safe center of its distribution.
Result: another individually-fine, collectively-boring sample.
This conservatism is not a flaw to be trained out.
Given the model's ignorance, it is probably the correct reflex, and so it seems unlikely to go away as models get better.
Something has to *tell* the model that unlikely-but-new is what we wanted.

# Diminishing Returns

The obvious brute-force fix is to oversample: generate 100 samples instead of 10, and pick the outliers.
I have spent a lot of hours doing exactly this, and the returns diminish so fast you can watch them go.
Sampling concentrates around the [modes](!W); that is, to a first approximation, what sampling *is*.
Along any given stylistic dimension (roughly normal, usually), the [expected extreme](!W "Extreme value theory") of $n$ draws grows like only ~$\sqrt{2 \ln n}$: the expected maximum of a standard normal sample creeps from ~1.5 SDs at $n$ = 10 to ~2.5 SDs at $n$ = 100, so a 10× compute bill buys <1 SD of additional spread per dimension.
Meanwhile the cost of reviewing them (a human squinting at thumbnails, several seconds apiece, to extract a best-of-100 choice worth all of log₂(100) ≈ 6.6 bits of preference information) grows the full 10×.
A logarithm racing a linear cost loses at every scale you can afford.
And that is per-dimension: the diversity we actually want is spread across many dimensions at once, where random sampling does even worse ([curse of dimensionality](!W)).

The right conclusion to draw from this failure is not that the model is impoverished, but that the sampling is.
The model is far more diverse than its samples ever reveal: the tails are in there, the weird styles are in there, the un-modal compositions are in there.
(Anyone who watched enough faces go by on [This Waifu Does Not Exist](/twdne) saw it: every few hundred samples, the StyleGAN would cough up some arresting mutant or an oddly-original stylization it might never repeat---too rare to find on purpose, but in there.)
Ordinary sampling simply never visits them.
(Sampling can prove the presence of knowledge, but not its absence.) So the fix belongs in the sampling process, not in scaling the model further, and not in prompt engineering.

# Novelty Search

We do not need to invent the right objective from scratch.
Evolutionary computation got there long ago with **novelty search** ([Lehman & Stanley 2011](https://www.cs.ucf.edu/eplex/papers/lehman_ecj11.pdf "Abandoning Objectives: Evolution Through the Search for Novelty Alone"); or for the book-length version, Stanley & Lehman's [*Why Greatness Cannot Be Planned*](https://www.amazon.com/Why-Greatness-Cannot-Planned-Objective/dp/3319155237)), and RL exploration research ([count-based novelty bonuses](https://arxiv.org/abs/1606.01868 "Unifying Count-Based Exploration and Intrinsic Motivation"), [Schmidhuber-style curiosity](https://people.idsia.ch/~juergen/creativity.html), [random network distillation](https://arxiv.org/abs/1810.12894)) has been circling the same idea ever since.
The trick sounds scandalous the first time you hear it: **ignore the reward**.
Score candidates instead by how much they differ from everything found so far, measured against an archive of past results.
The population stops banging its head against the nearest [local optimum](!W "Local optimum") and spreads out across behavior-space, accumulating a library of qualitatively-distinct solutions (what that field now calls '[quality-diversity](https://www.frontiersin.org/articles/10.3389/frobt.2016.00040/full "Quality Diversity: A New Frontier for Evolutionary Computation")').
Afterwards, you mine the library for the true reward at your leisure.
(Often the path to the global optimum turns out to run through regions that greedy reward-maximization would never have entered.)

That is our situation exactly: the user's 'true reward' (the image they will eventually love) is unknown, even, as noted, to them.
So maximize coverage first, and select second.

The obvious way to bolt novelty search onto a generative model is as an outer loop: sample, embed, compare to the archive, keep if sufficiently distant (perhaps seeding further batches from the keepers), discard otherwise; repeat.
And this outer loop can be implemented in as many cheap flavors as you please:

-   generate $n$ high-temperature samples and keep the $k$ most mutually-distant (eg. [_k_-medoids](!W "K-medoids") in [CLIP](https://arxiv.org/abs/2103.00020) embedding-space, or a [_k_-DPP](!W "Determinantal point process") if one is feeling fancy)
-   or store embeddings, and reject any new sample whose [_k_-NN](!W "K-nearest neighbors algorithm") minimum distance falls below a threshold
-   and if the archive grows large, binarize the embeddings into distance-preserving codes ([locality-sensitive hashing](!W)) and stuff them into a [Bloom](!W "Bloom filter") or [cuckoo filter](!W), so membership tests cost ~nothing even at billions of images

All fine, all easy, all standard.

But what is the *generation* step doing to seek novelty in such a pipeline?
Nothing.
Same weights, same conditioning, same procedure, every time.
Whatever novelty appears was supplied by the dice.
And do our knobs for loading the dice (higher [softmax](!W "Softmax function") temperature, weaker [classifier-free guidance](https://arxiv.org/abs/2207.12598), cranking up the truncation ψ ([Karras et al 2018](https://arxiv.org/abs/1812.04948 "A Style-Based Generator Architecture for Generative Adversarial Networks")---users of [This Waifu Does Not Exist](/twdne) may remember what ψ near the max produced)) at least create novelty?
No, only *noise*: they buy variance and pay in coherence, and most of what they buy, I suspect, is junk.

Worse, the economics are upside-down.
Generation is where the compute lives: ~10s of GPU-time per sample for a big diffusion model, vs ~1ms for an embedding & lookup.
Call it 99% vs 1%.
The outer loop optimizes the 1%.
You could make selection perfect (zero redundancy, ideal coverage of whatever the sampler happened to emit) and gain almost nothing, because the expensive 99% is still churning out the same central samples, and all the outer loop can do is throw them away.
[Rejection sampling](!W) from a mode-seeking sampler is a very expensive way to be disappointed.

# Statefulness

So novelty has to move inside generation.
But 'inside' in a particular way, and the wrong ways of being inside are instructive.

The tempting cheap hack is to attack the conditioning: add Gaussian noise to the prompt embedding, randomly drop or shuffle words, interpolate toward other prompts in text-embedding space.
This does produce different samples, but different because they are answers to different *questions*.
You asked for X and received roughly-X, or not-X-at-all---the semantic damage is proportional to the jitter.
Every power user has tried this; every power user has drifted back to brute-force resampling, which at least respects the prompt.

The deeper problem is that novelty is not a property any fixed function can have, because novelty is inherently temporal & contextual: a sample is novel only relative to the samples that came before it.
The first time a model emits some strange lovely composition, that is novelty; the second time, repetition: same pixels, same prompt, same model, opposite value.
([Pierre Menard](!W "Pierre Menard, Author of the Quixote") notwithstanding, an exact copy is not a second masterpiece.) If nothing in the system remembers what has been done, the system cannot avoid doing it again.
Somewhere, there must be state, or change.

# Vector Databases?

Why not just log every sample into a [vector database](!W), and then at each step of the diffusion process, embed the current latents, look up the nearest past neighbors, and push the update *away* from them?

This would work; I just don't believe the price.
The arithmetic is unkind: an embedding is small (a 768-d [CLIP](https://arxiv.org/abs/2103.00020) vector in FP16 is ~1.5KB), but a service accumulates billions of samples, so call it ~1.5TB of embeddings per billion images, before any [approximate-nearest-neighbor](!W "Nearest neighbor search") index overhead.
That cannot live in [VRAM](!W), which is the scarcest resource in the building; keeping the database small & recent so it fits defeats the point, since the samples you most need to avoid are the accumulated mass of *old* ones.
And evicted to CPU RAM or SSD, every query becomes a PCIe round-trip, paused-sampler time multiplied by every one of the 30--50 denoising steps of every sample of every user.
For a mechanism that must run on every step of every sample, the overhead is presumably prohibitive.

But notice how much of the vector database we would be paying for and never using.
We do not need to know *which* past sample is nearby, or the full neighbor list, or exact distances; we never need to read an embedding back out at all.
All we need is a cheap, approximate, differentiable nudge---"you are drifting somewhere overdone; push off"---a rough gradient away from the well-trodden regions of image-space.
And a rough cheap approximation of an arbitrary function is the one product the neural-network store always has in stock.

# Novelty Nets

A lossy, compressed "have I generated something like this before?" predicate is exactly what a small neural network is good at.
So put one *in* the model: a small [adapter](https://arxiv.org/abs/1902.00751 "Parameter-Efficient Transfer Learning for NLP") network (a **novelty net**) running on-GPU as just another layer or two, at negligible overhead.

1.  The novelty net takes the in-progress sample's embedding or activations, and predicts P(previously generated|trajectory): how much does this sample-in-progress resemble the accumulated history?

2.  That probability is added to sampling as a weak auxiliary loss; equivalently, its gradient is added as a guidance term.
    This is [classifier guidance](https://arxiv.org/abs/2105.05233 "Diffusion Models Beat GANs on Image Synthesis") run backwards, ie. anti-guidance: where classifier guidance pushes samples *toward* 'dog' or 'photorealistic', the novelty net pushes them *away* from 'seen it before'.
    Because the loss is weak, it cannot override the prompt or wreck quality.
    It merely leans on the trajectory, step after step, and the accumulated lean lands the finished sample somewhere high-quality *and* unfamiliar.

3.  When the sample finishes, it becomes a training datapoint: the net takes a few [online](!W "Online machine learning") gradient steps to label that embedding 'previously generated'.
    From then on, it pushes away from that too.

The whole loop, in pseudocode:

```
net = MLP()                        # the novelty net; small
loop:
    x = sample(prompt, guidance = prompt_guidance - w * grad(net(embed(x))))
    show(x)
    net.update(embed(x) -> 'seen', jitter(embed(x)) -> 'unseen')
```

Each new sample is thus repelled, weakly, by *all* its predecessors; the stream cannot revisit its own past.
And note what we did not need: the generator stays frozen (as cheap to serve as ever), and there is no database anywhere.
The entire history has been amortized into a few million adapter parameters, updated as you go.

How big and clever does a novelty net have to be?
Small, and dumb.
It does not have to learn anything deep (no semantics, no esthetics, no generalization to unseen distributions).
It only has to memorize (ie. compress) the sample history well enough to emit a repulsive gradient.
Its input is a fixed-size embedding vector with no spatial or sequential structure to exploit, so there is nothing for convolutions or attention to do.
The natural architecture is the lowly [MLP](!W "Multilayer perceptron").
And the MLP's most notorious vice, its eagerness to brute-force memorize training data rather than generalize, is here not a bug but the entire job description.

Something like 2--3 layers and a few million parameters would probably do: a rounding error next to a generator 3 OOMs bigger, but (at a few bits per parameter) enough raw capacity to fuzzily encode millions of past samples, since we need only a blurry silhouette of each, not a reconstruction.
Train it to regress the [log-odds](!W "Logit") of P(previously generated|embedding), updated online after each sample.
If it saturates or slowly forgets ([catastrophic forgetting](!W) being an occupational hazard of online learning), that is fine: the service logs every sample anyway, so the adapter can always be retrained offline by replay.
(A little forgetting might even be desirable---perhaps after enough years, an exiled corner of image-space has earned its parole, the way fashions cycle.)

One real failure mode: trained only on positives ('everything I show you was generated'), a classifier collapses to the degenerate solution P = 1 everywhere, whose gradient is useless.
So manufacture negatives by jittering the positives: perturb the stored embeddings (noise, [dropout](!W), small edits) and label the perturbed versions 'not previously generated'---the same flavor of augmentation trick that [self-supervised](!W "Self-supervised learning") methods (eg. [Barlow Twins](https://arxiv.org/abs/2103.03230 "Barlow Twins: Self-Supervised Learning via Redundancy Reduction"), Zbontar et al 2021) use to fend off representational collapse.
This teaches the net tight local boundaries around each past sample, which is precisely the shape of function whose gradient makes a useful repulsion.

None of the ingredients here is new, which I find reassuring rather than embarrassing.
Once you look, the relatives are everywhere.
Classifier guidance and negative prompts/tags, most obviously: one framing of a novelty net is to treat every prior sample as a very weak negative tag. (Done literally, thousands of accumulated negative prompts would cost thousands of extra conditioning passes; the net amortizes them all into one adapter.)
[GAN](!W "Generative adversarial network") discriminators are well-known to partially memorize the data they police ([Webster et al 2019](https://arxiv.org/abs/1901.03396 "Detecting Overfitting of Deep Generative Networks via Latent Recovery")).
Memory-based exploration in RL, from [neural episodic control](https://arxiv.org/abs/1703.01988) (Pritzel et al 2017) to [Go-Explore](https://arxiv.org/abs/1901.10995) (Ecoffet et al 2019), likewise stores visited states purely to push the agent somewhere else.
[Synthetic gradients](https://arxiv.org/abs/1608.05343) (Jaderberg et al 2016), because the net is exactly a learned surrogate supplying a gradient no true loss provides at sampling time.
And fast weights ([Schmidhuber 1992](https://people.idsia.ch/~juergen/fast-weight-programmer-1991-transformer.html "Learning to Control Fast-Weight Memories"); [Ba et al 2016](https://arxiv.org/abs/1610.06258 "Using Fast Weights to Attend to the Recent Past")), ie. a small rapidly-updated memory strapped to a big, slow, frozen network.

What does the user see?
Nothing---and that is the point.
Today, escaping sameness means fiddling: temperature & guidance sliders, 'weird' parameters, appending "make it DIFFERENT this time" to the prompt (which works about as well as you would expect), or grinding 'variation' buttons on the two least-boring samples so far.
With a novelty net, the loop is sample; look; sample again.
Every image differs from every image before it, including the flawed ones, whose particular flaws the net now also steers away from, so you are never shown the same mangled hands twice.
Idle re-rolling becomes an actual walk through image-space rather than a stagger around its mode.
And when recognition finally fires ("*that* one"), you freeze or disable the net and converge with the ordinary variation tools, which are good at exactly that.

# Distilling the Knob

The online net is the general mechanism, but one could also distill novelty into an ordinary control input, which may be the more deployable form.
Generative models already accept conditioning beyond text (reference images, style embeddings, personalization vectors; Midjourney's personalization is exactly such a knob), and these inputs reach attributes no prompt keyword can, including, if we choose to train it in, the desired novelty level itself.
(Midjourney's `--chaos` & `--weird` parameters gesture in this direction, but they appear to work by cranking up noise somewhere, rather than by learning what users mean by 'different'.)

Where does the training signal come from?
The service's own logs: prompts, each with its $n$ generated images, the quality signals users already emit for free (ratings, upscales, downloads), and the embedding similarity among the $n$.
For each prompt, rank the images by a weighted mix of quality & mutual dissimilarity, assigning each a unique index 1--_n_.
Then finetune on (prompt + index _k_) → image.
The model is forced to learn two things: the shape of the quality/diversity tradeoff (index #1 means 'best, safest rendition'; index #37 means 'much stranger, still acceptable'), and, because every index within a prompt maps to a different image, that samples at different indices must not collide.
At runtime, a user who wants a contact sheet asks for indices 1--_k_, and receives _k_ samples sweeping along the quality-vs-diversity [Pareto frontier](!W "Pareto front") with little redundancy---the novelty net's exploration behavior, baked back in as a static, promptable skill.

# Collective Novelty

Everything so far is per-user: *your* next sample avoids *your* past samples.
But why stop there?
Train one service-wide, on every sample any user has generated---and on the original training corpus itself.

That last is not a typo.
A novelty net trained on the training data steers generation *away from the training data*: near-duplicates of training images are, by construction, 'already generated'.
This mitigates the copyright-flavored embarrassment of regurgitated training images ([Carlini et al 2023](https://arxiv.org/abs/2301.13188 "Extracting Training Data from Diffusion Models")).
But more importantly, it mitigates regurgitation's creative pointlessness: an image that already exists did not need a generative model, and a standing bias toward transformation over reproduction is arguably what we wanted from these tools all along.

And a service-wide net works against the house style: the instantly-recognizable 'Midjourney look' (the glossy teal-&-orange grading, the centered subject, the volumetric haze; every model has its own), which no single image creates, but millions of lazily-prompted samples piling into the same mode do.
The look is a [collective-action problem](!W "Collective action problem")---a [negative externality](!W "Externality") of lazy sampling.
Every user who accepts the default look helps teach audiences the signature, then to resent it, until viewers are 'allergic' on sight.
And the allergy then devalues the carefully-crafted, heavily-curated images of *other* users, dismissed as '[AI slop](!W)' for a mere family resemblance.
(I have spent hours pushing a single Gwern.net illustration far from the house style, and it can still be written off at a glance by someone burned once too often by lazy defaults.) A shared novelty net is the lightest-touch remedy I can think of: it censors no one, curates nothing, bans no style; it just leans, gently and globally, against whatever has already been done too many times.

---

Would it all work as smoothly as sketched?
Unknown.
Guidance terms have a way of being finicky in practice, and the embedding, the loss weight, & the negative-jittering would each need real tuning.
But the underlying point survives any amount of engineering trouble: novelty, like randomness, is a property of histories rather than of artifacts, so a stateless generative model can no more be novel than a book can surprise you on the second reading---until the models remember their own past, we will have to strap the memory on ourselves.
I would predict that even a crude novelty net, shipped as a mere on/off toggle, would turn out to be one of those features (like negative prompts, which users discovered more than they were given) that no one asked for beforehand, and no one would give up afterwards.

## See Also

- [This Waifu Does Not Exist](/twdne)
- [Making Anime Faces With StyleGAN](/face)
- [GPT-3 Creative Fiction](/gpt-3)
- [RL exploration](/doc/reinforcement-learning/exploration/index)

eval_iclmargin_fb100_tg_fable5_xhigh_comp100k_20260702_lean-scaling · final judges 0/1 · 25,930 chars · 4,208 words · /home/ubuntu/code/lesswronger/data/agent_runs/eval_iclmargin_fb100_tg_fable5_xhigh_comp100k_20260702_lean-scaling/submission.md

> Coding LLMs are on track to write most future software; unfortunately, their output is often mediocre or insecure, and the ideal remedy (formally-verified code) lags so far behind as to seem irrelevant.
> Worse, if LLM coding skill is set mostly by training-corpus size (Luo et al 2025), then today's mainstream languages may be permanently locked in, blocking migration to better-designed languages like Haskell, Rust, or Lean.
>
> But the lock-in claim confuses a scaling *constant* with a scaling *exponent*: corpus size sets a language's starting performance, not its trajectory.
> I propose measuring per-language **predictability scaling laws**: how an LLM's prediction loss over source code (cross-entropy, normalized to bits-per-byte) falls as more of a codebase is in-context.
> The conjecture: well-designed code grows more predictable as more of the system is in context (the more you have read, the better you can guess the rest), badly-designed code grows less predictable, and brute force stays flat.
>
> This can be measured cheaply with frozen pretrained LLMs: concatenate per-language corpora, record loss by token position, normalize to bytes, fit curves.
> Cross-checks include injecting subtle bugs (a better language should make bugs *more* surprising), compiling sampled completions under artificial context limits, ablating type signatures, and searching for minimal 'coreset' contexts as a modularity metric.
>
> Extrapolating the fitted constants & exponents yields crossover forecasts: at what codebase size does a strongly-typed language like Haskell out-predict Python?
> Does Lean (presumably the worst constant, but perhaps the best exponent) ever cross over?
> If it does, that would justify a large investment in porting software to Lean, since the ports themselves manufacture the missing training data, breaking the chicken-and-egg bottleneck and buying provable correctness against LLM-powered attackers.
>
> The same logic could score proof libraries and formalization tools, separating genuine semantic compression from brute-force **mathslop**; the likeliest failure mode is that ecosystem maturity swamps language design in the measured curves, which would itself be worth knowing.

> Beware of bugs in the above code; I have only proved it correct, not tried it.
>
> ---Donald Knuth, 1977

Most new code will soon be written by LLMs, if it isn't already (Google was claiming >25% of its new code by late 2024, and the 'vibe-coding' startups claim ~100%).
This would be more reassuring if the code were any good.

# How Far Behind?

Anyone who has audited a vibe-coded project knows the texture: unsanitized inputs, quietly swallowed exceptions, trust boundaries that exist only in the comments.
In the Stanford user study (Perry et al 2022), programmers given an AI assistant wrote *less* secure code while becoming *more* confident that it was secure.
And the attackers get the same tools we do: Google's "Big Sleep" agent found an exploitable SQLite bug in 2024, apparently before the fuzzers did---and SQLite is perhaps the most-tested C codebase on earth, something like 600 lines of test per line of library code, and it didn't matter!
(Testing, as Dijkstra said, can show only the presence of bugs.)
Would I bet on the defense winning an arms race where both sides run LLMs?
No.

Nor was the human baseline ever trustworthy to begin with.
I have never written an equation or line of code that I was 100% confident of, or which I thought had less than a 1-in-trillions chance of it being wrong in some important way; software & real-world systems are too complex & fragile.
Consider a simple-seeming line of conditional code for the arithmetical tautology `x + x == 2x`. How could this *possibly* ever go wrong? Well...

- Where did you initialize *x*? Was it ever initialized to a non-null value? (Or has it been working *accidentally* because it uses uninitialized memory which just happened to have a workable value?)

- Is this comparison by reference, equality, hash, or some other way entirely?

- Which integer type is this? Does that integer type overflow?

    In some languages, *x* might be a string being parsed as a number.
    JavaScript is infamous for this due to its type coercion and redefining operators; this will evaluate to `true`: `x = "1"; 2*x == 2 && x + x == "11"`

    In highly dynamic or object-oriented languages, `+`, `==`, and `*` could all have been redefined per *x* and mean... just about anything, and do anything as side-effects of methods like getters.

    homoglyph attack: 'x' might not be 'х'... because the second one is actually a different letter (CYRILLIC SMALL LETTER HA).

- If this is floating point (do you know for sure?), won't this usually be *false* at larger/smaller numbers? What about rounding, or special values like NaN or +Inf vs −Inf?

- Did the compiler optimize away this check entirely because "obviously" it is true?

- By the way, are you *sure* it's a conditional at all? Perhaps it was parsed as `(x + x == 2) * x`?

- This is serial-threaded code, right? No parallelism anywhere? If there is...

    Signal handlers can interrupt your code and change stuff. So your code is de facto concurrent anyway. Oh, but you solved that? So there's no more parallelism.
    Trick question: you thought there wasn't, but there was anyway because *all* systems are inherently parallel now. So there are dangers around cache coherency & races, leading to many classes of attacks/errors like Spectre.

- Are you running the right code at all? Maybe you're running a cached binary, the build system didn't rebuild the code, you are in the wrong working directory or the wrong VCS branch...

- Why do you believe the hardware will always store & execute everything correctly?

    What are the odds that the hardware will be hit by a cosmic ray during any of these operations? Even ECC RAM is increasingly unreliable.

    (For several years, compiling the Gwern.net website would occasionally result in strange segfaults in apparently correct old stable regexp code; this turned out to be a bad RAM chip... where ordinary computer use simply didn't stress RAM enough to crash noticeably often.)

    What are the odds that the CPU core in question is *sometimes* unable to add *or* multiply correctly? (If you're a hyperscaler, they exist in your fleet of servers somewhere!)

I can safely say that in my programming life, I have written many fewer than trillions of lines of code, and I have made many more of these errors than 0.
So I infer that for even the simplest-seeming code, I am unable to write code merely as reliable as a 1-in-trillions error rate.

The only known exit from that treadmill is proof, plus a language that deletes rows from the table above wholesale.
Knuth's joke was funny in 1977 because no one expected machine-checked proofs to ever become load-bearing; they now are.
When Csmith fuzzed C compilers (Yang et al 2011), it found wrong-code bugs in every compiler it touched---except in the verified middle-end of CompCert, where it found none.
TimSort, Java's default sort, shipped a subtle array-bounds bug across billions of devices for years, until de Gouw et al 2015 tried to prove it correct and the loop invariant refused to go through.
The catch was never the ideal but the price: seL4 spent ~20 person-years proving ~9,000 lines of C, at ~\$400 per verified line (and the team considered that cheap!), so verification stayed a boutique trade, and ~70% of serious CVEs remain the same memory-safety errors every year.
So we should ask less whether formal verification is behind (obviously) than *how far* behind, and whether LLMs change how fast it could catch up.

# Priors, Not Ceilings

The pessimistic answer says the lock-in is already total.
Luo et al 2025 measures LLM scaling on code, finds it far more data-hungry than natural language, and projects that the gap between high-resource and low-resource programming languages will widen rather than close: a language's popularity today determines its assistant quality 'indefinitely'.
This is the 'data wall' argument transposed into programming languages, and I have been through this argument before, in its original habitat:

> The key point here is that the 'severe diminishing returns' were well-known and had been quantified extensively and the power-laws were what were being used to forecast and design the LLMs.
> So when you told anyone in AI "well, the data must have diminishing returns", this was definitely true---but you weren't telling anyone anything they shouldn't've already known in detail.
> The returns have always diminished, right from the start.
> There has never been a time in AI where the returns did not diminish.
> (And in computing in general: "We went men to the moon with less total compute than we waste to animate your browser tab's favicon now!" Nevertheless, computers are way more important to the world now than they were back then. The returns diminished, but Moore's law kept lawing.)
>
> The all-important questions are exactly how much it diminishes and why and what the other scaling laws are (eg. any specific diminishing returns in data would diminish slower if you were able to use more compute to extract more knowledge from each datapoint) and how they inter-relate, and what the consequences are.

The all-important questions here, likewise, are the constants and the exponents, not the bare observation that Python has more data.
To a model, the pretraining corpus is a prior, and priors have exchange rates: Python skill transfers to Haskell at a discount, so the effective data for a rare language is a discounted sum over every related language, which is enormous.
(Yang et al 2025 measures the transfer matrix directly, and it's large, asymmetric, and nowhere near diagonal.)
This matches my anecdata: Gwern.net's build system is a niche of a niche---Hakyll static-site plumbing plus a decade of accreted custom Haskell passes---yet GPT-3 could already complete it plausibly in mid-2020, and every model generation since has improved on the Haskell faster than on the Python, without anyone writing much more Hakyll in the meantime.
The transfer keeps arriving from somewhere...
Extrapolate that, and we get less a 'lock-in' than a coming renaissance of obscure languages: the old cost of adopting a better language was years of human retraining, and the entity doing the relearning is now a model.
Per-language rankings today are a snapshot of the constants; the migration decisions ought to be made on the exponents, which no one has measured.

# Design Happens At Scale

Why would exponents differ by language?
Because language design is mostly about what happens at scale.
Haskell is (I can attest) genuinely annoying for a 50-line script; but at a million lines the ledger flips, because its types have been silently enforcing invariants across thousands of files that no context window can hold at once.
Permissive languages fail in the mirrored way: dynamic types, monkey-patching, mutable globals---each is a way for the meaning of a line to depend on arbitrarily distant facts.
(Rails once monkey-patched Ruby's core classes so thoroughly that the semantics of `String` depended on your `Gemfile`; this was considered a feature!)
An LLM inherits the very problem the human maintainer had, minus the option of asking around; and the spectacular version of the cost is a Knight Capital, \$440m gone in 45 minutes, from a stale feature-flag reactivating dead code on 1 of 8 servers.

# The Lean Bottleneck

Lean sits at the far end of this axis: a dependently-typed proof assistant whose types can state any correctness property you can formalize, now creeping into use for actual programs.
There's a zlib") rewrite in Lean; no one needed another zlib, but as an existence proof it will do, showing "it compiles" can be made to mean "it is correct".
So we can daydream: port the world's critical software into Lean, and whole categories of CVE just... stop existing, no matter how hard an LLM attacker squeezes.

Chicken & egg. But there's almost no Lean in the world (mathlib") is ~1.5 million lines; beyond it, a thin scattering), and the dependency is circular: training LLMs to write large Lean codebases requires large Lean codebases, which will exist only after the LLMs (or many expensive humans) write them.

Mathslop. Worse, is Lean even well-designed for *software*?
It was designed for theorem-proving, a different job, and anyone who has read machine-generated Lean (I have, more than I would like) knows the failure texture: hundreds of lines of `simp`/`omega`/`decide` grinding, where deleting any line breaks the proof and reading all of them teaches nothing.
I call the degenerate case **mathslop**: formally valid, machine-checked, teaching neither human nor model anything reusable.
Perhaps large real-world Lean converges to mathslop, dependent types metastasizing through every signature until refactoring becomes impossible...
mathlib suggests otherwise---proof-golfing is a community sport there, and Tao's PFR formalization let contributors who didn't understand the proof fill in lemmas anyway, off typed holes in the blueprint---but I'd much rather measure which way Lean-at-scale converges *before* someone spends a decade and a few billion dollars finding out.

# Rereading Well

Every human complaint about bad code---ad hoc choices, duplication, brute force, global state---cashes out as the same reading experience: the code is hard to *predict*.
Gene Wolfe defined good literature as "that which can be read by an educated reader, and reread with increased pleasure"; good code is code that *rereads* well.
The more of the system you've absorbed, the more inevitable each next line feels; in the limit, which Lean approaches, a signature so constrains its implementation that there's essentially one way to write it.
The first time a model meets `newtype WalletId = WalletId UUID` it pays full price in bits, and every later use is nearly free; the ten-thousandth call to a Python `get_user()` is still a small gamble (does this one return `None`, or throw, or hit the network?), because any module may have monkey-patched it since.

The *level* of predictability would be the wrong metric, though; "compression = understanding" needs care, and I have made a version of this mistake-avoidance argument before, about the Hutter Prize:

> Why it works. This turns the usual benchmark pathology inside out.
> A system which "cheats" by memorizing the answers does not beat the benchmark; it simply moves the answers from the compressed output into the model, and is charged either way.
> In Kolmogorov complexity terms, the Prize asks for the shortest total description, not merely the lowest predictive loss.
>
> Paying for bytes. Once model size is charged against a 1GB evaluation corpus, useful models are excluded almost automatically.
> A model that is hundreds of gigabytes or larger needs implausibly large savings per input byte, or implausibly much evaluation text, before its extra size pays for itself.
> A 1TB larger model that saves 0.01 bits per input byte only breaks even after ~800TB of text.
> Under the Prize's literal objective, a model like GPT-5 winds up "stupider than a `cat`".

Getting the objective wrong by a normalization is enough to exclude the interesting regime entirely; that is the caution to carry into measuring languages.
Java getters cost ~0 bits, and boilerplate isn't design.
The signal I want is the trajectory, as the model reads deeper into a system.
Well-designed code gets more predictable the more of it you've seen---each file teaches conventions, types, architecture that transfer to the next---while badly designed code doesn't reward the reading, and can actively mislead: you learned "the convention", and then it's violated.
(Pedantically, extra context can't hurt an ideal Bayesian predictor; "gets less predictable" is a claim about real models with finite windows, which is fine, because real models are what will write the code.)
Brute force is the third texture, flat: case _n_ teaches nothing about case _n_+1.
Flat trajectories are what mathslop looks like from the model's side, with the advantage over "elegance" that we can put error bars on a slope.

# Measurement

Measuring this needs no training, just forward passes through frozen models: concatenate per-language corpora into long documents, record loss by token position averaged over many windows, fit loss against in-context codebase size, and read the constants and exponents off as design metrics.
'Concatenate per-language corpora' hides a unit problem, one I have hit before in document retrieval:

> OK, so you decide you will embed every passage... but what is a 'passage'?
> Is it every paragraph? What is a paragraph, is it just every `... ` element? (What if it's a single sentence?)
> What about lists and blockquotes and footnotes and tables and...?
> What if the paragraphs are not individually good matches, but an entire *section* (or the second half of an essay) is what the reader is looking for?
> How do you specify all this in advance?

What is a 'codebase', likewise?
A repository? A package? Do vendored dependencies count, or generated code, or the test suite?
You make a defensible choice, document it, and check that the fitted exponents survive the alternatives.
Two normalizations are mandatory: token losses must become bits-per-byte (tokenizers are tuned to popular languages and quietly subsidize them), and verbosity must be corrected for, since a bit of APL and a bit of Java aren't the same achievement.
(I keep meaning to run the pilot on the Gwern.net repository itself: a frozen open model, logprobs over the Haskell in dependency order, one afternoon of scripting.
I keep not doing it, which is partly why I'm writing this up for someone else to do...)

The raw curve is coarse, so we add cross-checks, each cheap.
Bug injection: insert a missing bounds check of the Heartbleed genus, a sign error, a plausible-but-wrong lemma, and check whether loss spikes at the bug site---high loss is good here (cf. the Inverse Scaling Prize), which conveniently guards the headline metric against Goodharting.
Hidden context: what fraction of sampled completions still compile & pass tests as we clamp the visible context?
Given only `def inflate : ByteStream → Except DeflateError ByteStream` and the surrounding types, can the model reconstruct a working implementation?
(Renaming everything first, so the types rather than the vocabulary carry the specification.)
Ablations: strip Haskell's type signatures, add Python annotations, remeasure loss far downstream---language-war talking points converted into effect sizes.
Coresets: how small is the minimal context that recovers ~the full-prefix loss on a target file?
Short coreset = real modularity; Lean ought to need little beyond the signatures in scope, while Python needs many files, since the only way to know runtime behavior is to go read it.
It's probably a grad-student-sized project: corpus wrangling, a few GPU-weeks of forward passes, curve-fitting.

# The Crossover

What would the curves show?
I don't know, but I'll guess.
'Weak' languages win small: they own the training data, and a few thousand lines of dynamic scripting is formulaic.
Haskell starts worse---rarer, denser, more alien---and improves faster, crossing somewhere (I'd guess) in the 100K--1M line range, which is (not coincidentally?) where human teams start wishing they had types.
Lean starts worst of all, and may never cross in absolute loss at any length we can test.
But if my thesis is right, it has the best exponent, and the exponent is the number to care about, because then the chicken-and-egg is just a bill: training data can be bought.
When the 'data wall' rumors were circulating, I made the same argument about text:

> It was popular in 2020--2022 to claim that all the text had already been used up and so scaling had hit a wall and such dataset increases were impossible, but it was just not true if you thought about it.
> A lot of people seemed to think that Common Crawl contains 'the whole Internet', but it doesn't---it doesn't even contain basic parts of the Western Internet like Twitter.
> Or you could look at the book counts: the papers report training LLMs on a few million books, which might seem like a lot, but Google Books has closer to a few hundred million books-worth of text.
> And then... if you have a billion dollars cash and you can hire some hard-up grad students or postdocs at \$20/hour to write a thousand high-quality words, that goes a long way.
> If there was demand for the data, supply would be found for it.

The same holds for verified Lean, with the twist that the retries are graded by the kernel.
Regress the constants on corpus size, estimate how many Lean tokens pull the crossover down to a repository size anyone cares about, and buy the tokens---from data-labeling shops, or from agentic transpilation.
Mediocre models will grind out verified ports if we pay for enough retries (pass@_k_ keeps climbing long after pass@1 has embarrassed itself), the Lean compiler being that rarest of things, an incorruptible reward model.
I have proposed the same bootstrap before, for training LLMs to write statistical models, and the mechanics carry over almost unchanged:

> The trained SPFN proposes models for real datasets.
> You actually fit those models via MCMC, check the diagnostics, and keep the ones that work.
> Those validated (code, posterior predictive) pairs go back into training.
> Iterate.
>
> The training distribution shifts toward "Stan programs that work in practice" rather than just "Stan programs that exist on GitHub."
> This resembles a wake-sleep algorithm or expert iteration in game-playing: a policy proposes moves, search improves them, the improved moves become training data for the next policy.
>
> The risk is mode collapse---the SPFN proposes similar models repeatedly, those get validated, training reinforces the narrow bias.

Substitute Lean programs for Stan programs and the kernel for the diagnostics, and that is the Lean flywheel: propose ports, keep what the compiler certifies, retrain.
And the mode collapse to watch for already has a name here---mathslop.
Incorruptible, not unfoolable: someone must still grep the output for `sorry`, new `axiom`s, and specs so weak they're vacuously satisfied.
The spec is the part I'd worry about most: "decompresses every valid RFC 1951 stream" says nothing about timing side-channels, so verification really moves the attack surface from 100M lines of code to a few thousand lines of spec---which isn't nothing, but is at least a surface humans can actually read.
At seL4's ~\$400/line, 100M verified lines would run \$40b: hopeless.
If agents cut it 100×, it's a rounding error next to a frontier training run, and unlike a training run it doesn't depreciate---the theorems stay proved!

# Confounds

Corpora differ in who writes them and about what, and empirical PL research has embarrassed itself here before: the widely-cited GitHub study of language effects on defect rates (Ray et al 2014) mostly evaporated under reanalysis (Berger et al 2019), with _n_ in the millions of commits.
Lean is written by mathematicians and obsessives formalizing textbooks for fun; JavaScript by the entire distribution of working developers under deadline.
If Lean's curve looks beautiful, is that dependent types, or is it that Kevin Buzzard's students write more carefully than the median React shop?
(I also notice that if a naive analysis showed JavaScript out-scaling Lean, I would suspect the measurement rather than update... so the naive analysis can only confirm, and should be designed against in advance.)

Topic-matching is the partial control: JSON parsers vs JSON parsers.
The stronger control is to synthesize the comparison---implement one spec in two languages to the same measured quality, then compare curves---and the zlib pair is nearly this already (C zlib & Lean zlib implement the same RFC 1951), about as close to a controlled experiment as found code gets.
A subtler worry: maybe current models are too ignorant of Lean to *use* Lean context well, so a measured exponent reflects the model's incompetence rather than the language's structure.
That's checkable (does the exponent move between a stock model and a Lean-finetuned one?), and my bet is the labs have mostly fixed it already, having spent 2 years gorging their models on math & code.

# Beyond Code

The trick generalizes, since what it measures is the slope of the compression rather than the level, and that is a definition of insight which isn't specific to code.
A good theorem is semantic compression: once proved, everything downstream gets shorter & more predictable.
So a good formalization library should visibly bend the loss curves of proofs built on top of it, and a good formalization *tool* should yield libraries with high reuse and clean repair-locality (change a definition: can a model predict which proofs break, and how?).
Mathslop fails all of this measurably, and someone, I suspect, is soon going to need an automated way of asking whether a machine-generated library has compressed its subject or just certified a heap.

The most likely way it all goes wrong: ecosystem maturity and corpus priors dominate the measured exponents, and the invariants I've been romanticizing (types, purity, proofs) contribute only marginally.
Then the constants are everything, Luo et al 2025's 'lock-in' stands, and the right advice is the boring kind (write documentation, standardize idioms in the languages we already have), while the heroic decade of porting the world into dependent types waits...
Fine; I'd take that result too.
But it costs a few GPU-weeks to find out, and I'd like to see the curves first.

```
Return to blog index
```

eval_iclmargin_fb100_tg_fable5_xhigh_comp100k_20260702_generating-style · final judges 0/1 · 34,968 chars · 5,377 words · /home/ubuntu/code/lesswronger/data/agent_runs/eval_iclmargin_fb100_tg_fable5_xhigh_comp100k_20260702_generating-style/submission.md

# Towards Style-Aware Generative Models

> Image generators can render any style you can name, but they cannot *invent* a style, because a style is not a bag of visual features: it is a historically-situated package of works, techniques, names, advocates, enemies, and institutions, which other artists & audiences then adopt.
> Sampling i.i.d. images from a latent space, however striking each individual image, can never add up to a movement.
>
> I suggest that the right target for "creative" image generation is a generative model of art *history*: a latent space organized by chronology & influence, learnable now by contrastive training on the date metadata we already have.
> Such a model would support replaying art history, extrapolating counterfactual histories, assigning causal credit to individual artworks (for IP or art-historical purposes), and simulating new movements forward---ultimately by populating the latent space with virtual artists who have careers, rivalries, and fans.
>
> A model like this would generate *styles*, and not merely images.

A standard criticism of image generators like Midjourney or DALL·E 3 is that whatever their technical merits, they cannot be truly creative, because they cannot invent a new style the way a Monet or Picasso did; they can only remix the styles that human artists already invented & named for them.

In one sense, this criticism is clearly false, or at least confused.
Contemporary generative models are so large, and trained on so much data (LAION-5B alone is ~5.85 billion image--text pairs), that they can embed more or less any image you hand them into their latent space near-losslessly; in that trivial sense, they already "contain" every style, including all the ones no one has invented yet, the way a block of marble "contains" every statue.
And the samples can be genuinely striking.
Anyone who spent time on Artbreeder (né Ganbreeder), or who watched what people dredged out of the far reaches of StyleGAN's latent space, or who has played with Midjourney's `--weird` parameter, has seen images that look like nothing in particular: not sourceable to any named artist, genre, or tag, and sometimes good.
When I was training StyleGAN anime models back in 2019 for This Waifu Does Not Exist, my favorite samples were usually the *failures*---the psychedelic melting faces from turning the truncation parameter ψ up too high, well outside the distribution of anything Danbooru-like, which had a coherent look to them that no human illustrator would ever have produced (or, probably, wanted to).
If we define "new style" as "images unlike existing labeled images", the criticism was refuted years ago.

But that is not what anyone means by inventing a style.
Billions of images have been sampled from these models by now, by tens of millions of users, and essentially no *recognizable new styles* have come out the other end: no movement, no school, no -ism, nothing you could name and imitate and react against.
A striking image is not a style, any more than a striking sentence is a language.
So the criticism survives its own refutation, and it is worth asking why.

# Why No New Styles?

Why don't new styles condense out of all this sampling, the way they condensed out of human image-making at a rate of several per decade for the past 2 centuries?
I think there are two reasons, one social and one architectural, and they mirror each other.

The social reason is the collapse of the cultural filtering mechanisms which used to *make* styles.
Historically, a new style fermented in relative isolation: a handful of artists in one city, seeing mostly each other's work, reacting to a shared enemy (the Salon, the academy, the previous generation), with communication slow enough that a distinctive shared practice could stabilize before being diluted.
The Impressionists are the type specimen: a specific caf'e crowd, a specific rejection (the 1863 Salon des Refusés), 8 group exhibitions 1874--1886, and even the name supplied by a hostile critic (Louis Leroy, mocking Monet's _Impression, Sunrise_).
Subcultures need boundaries to exist at all, and the internet has been dissolving those boundaries for decades---every influence available to everyone, instantly---so this process was dying before image generators arrived; the generators merely finish the job.
When every user can conjure any combination of influences on demand, no combination stays rare long enough to become the property of a group, and no group has to commit to anything.
A movement needs members, and members need something to join.

The architectural reason is an analogous context-collapse inside the generator itself.
A text-to-image model treats an image as a collage of recombinable discrete elements: a subject, a medium, a palette, a lighting keyword, an artist tag, "trending on ArtStation".
The training corpus is a heap of image/caption pairs, shuffled i.i.d., stripped of time & provenance; whatever internal representation of "Impressionism" the model learns is simply the visual correlates of a caption keyword.
It does not represent Impressionism as a *thing that happened*: something which began at a particular time, among particular people, in reaction to particular predecessors, which was named (mockingly), fought over, institutionalized, imitated, and eventually found boring & reacted against in turn.
It represents a texture.
(Our terminology gives the game away: StyleGAN's "style mixing" means copying texture statistics at different layers from one sample to another, and "style transfer" à la Gatys et al 2015 means matching Gram matrices.
"Style", in the deep learning literature, has always just meant "texture", and the models believe us.)

This points at what a style actually is, and why the visual features are the least of it.
A style is a package deal situated in a historical context: a set of works; the techniques that produce them; a name; a story about why; advocates who push it and rivals who push back; exhibitions & dealers & manifestos; and finally, adoption---by other artists, then by audiences.
The pixels are the *output* of this package, not the package.
A generative model that learns only the visual features has learned the shadow a style casts onto pixel-space.
Quantitative aestheticians have made this point before: Martindale's _The Clockwork Muse_ (1990) modeled stylistic change as a social treadmill---habituation constantly devalues the current style, forcing artists to escalate novelty until the style breaks and a new one forms---and nothing in that loop is a property of any individual image.
Predictably, sampling harder from a fixed distribution does not run the loop.

Amusingly, there *are* clear cases of image generators inventing styles, and they show exactly what is missing.
The first was DeepDream in 2015: a genuinely new, instantly recognizable esthetic (dog-slug fractal pareidolia) which got a name, spawned imitators, saturated, became a cliché, and now serves to date an image to ~2015--2016 as reliably as a mullet dates a photo to the 1980s.
The recent one is "Shrimp Jesus", and more broadly the glossy hyper-saturated devotional Facebook slop genre.
That is a real style!
You know it when you see it; it has a name; people imitate it and react to it.
But note that in neither case did the model invent the style *on purpose*.
DeepDream's look was an artifact of gradient ascent on Inception features; Shrimp Jesus seems to be an artifact of preference-tuning gone wrong---mode collapse onto a narrow attractor of whatever engagement-maximizing raters or rater-proxies rewarded (the same dynamics that give tuned LLMs their "ChatGPTese", or my old GANs their dropped modes).
And in both cases the *style* part---noticing the cluster, naming it, finding it funny or horrifying, building a discourse around it, getting bored of it---was done entirely by humans, after the fact.
The model contributed pixels; humanity contributed the style.
The model has no internal capacity to do what any human art scene would do next: notice that Shrimp Jesus has gotten stale, and react.

# The Credit Assignment Problem

There is a second thing current systems cannot do, which looks unrelated but turns out to be the same problem: they cannot tell you *how much any given artwork mattered*.

We would often like to know: how much did this particular painting contribute to a model's knowledge, or to the value of a particular generated sample?
The motivation may be legal---if artists whose work trained a model are to be compensated, we need some principled attribution of credit---or art-historical: "influence" is the central explanatory concept of art history, and it would be nice to measure it rather than assert it over drinks.

There are two standard answers, and they fail in instructively opposite ways.

The first is the rigorous method aimed at the wrong question: leave-one-out.
Delete the datapoint, retrain, measure the difference; or approximate that cheaply with influence functions (Koh & Liang 2017) or datamodels.
But consider running leave-one-out on the _Mona Lisa_.
You drop the painting itself and retrain... and nothing happens, because the corpus still contains the thousands of parodies, pastiches, reproductions, advertisements, and references (Duchamp's mustachioed _L.H.O.O.Q._, every "Mona Lisa but X" meme), from which the model can triangulate the original almost perfectly.
LOOCV dutifully reports: this datapoint is redundant; marginal value ~0.
Which is true!
But it answers "is this datapoint redundant given the rest of the corpus?", and the question we meant to ask was the counterfactual "what if Leonardo had never painted it?"---in which case the culture would be correspondingly impoverished: no parodies, no pastiches, no _Mona Lisa_-shaped influence running through a million later works.
Influence functions inherit the flaw: they measure which *datapoints* moved the model, not which *works* moved all subsequent works.

The second is the cheap method aimed at the right question: retrieval.
Take a generated sample, look up the most visually-similar training images, declare those the "sources".
(This was the approach of tools like Stable Attribution, and it is implicitly the epistemology of most "AI is stealing from artists" screenshots.)
It at least gestures at causation, but it establishes it by _post hoc ergo propter hoc_: visual similarity is not causal descent.
Two artists who independently parody the same thing will look alike without either causing the other; a sample resembling some minor 1890s academic painter may descend from him, or merely from his teachers, or from nothing at all---coincidence does a lot of work across a few billion images.
Similarity is evidence of influence, but weak evidence, and it fails worst in exactly the adversarial settings (litigation) where we want it most.

So one method computes the wrong counterfactual precisely, and the other guesses at the right counterfactual sloppily.
What we would like is a model in which the right counterfactual is *computable*.

# Style Embeddings

Back in 2022, I suggested a first step: make generators "style-aware" by working in a style-specific embedding space, rather than a generic semantic one.

The observation was that styles are not uniformly distributed in any reasonable embedding space; they form island-chains of works, archipelagos surrounded by large empty regions.
(Empty not because images there are impossible or bad, but because no one happened to make them.)
So one can hunt for "holes": regions which the model assigns high likelihood---it considers images there perfectly plausible, well-formed, art-like---but which contain anomalously few actual datapoints.
Such holes are candidate *missing styles*: coherent visual practices that no human school ever got around to developing.
The interesting holes are those adjacent to popular styles, since they represent directions a movement could have taken but didn't.
One could use them as targets for creative-adversarial generation (cf. CAN, Elgammal et al 2017, which trained a GAN to deviate from known style classifications while remaining art-like), or more modestly, to fix the monotony of style prompting: instead of every "Impressionist" request collapsing onto the same centroid pastiche, push each sample toward a different adjacent hole, so users see the space *around* Impressionism rather than its average.
(This is the diversity/quality knob again---StyleGAN's ψ, reborn as an esthetic exploration parameter rather than a fidelity one.)

I still think this is worth doing, but I now think it is the minor half of the answer.
It treats the embedding space as a static map and mines it for unvisited locations.
The credit-assignment problem shows what is missing from the map: an arrow of time.

# Temporal Embeddings

If the questions we care about are causal---what influenced what? what would history look like without this work?---then the latent space should be organized by causality, and the great cheap proxy for causality is *time*.

Ordinary embeddings do not give us this.
CLIP-style training prioritizes whatever features explain the most variance in image--text pairs, which means semantics & appearance; the space burns enormous capacity on distinctions like glasses vs no-glasses which are orthogonal to art history.
Temporal structure is in there somewhere (I would guess a linear probe can date paintings from CLIP embeddings to within a few decades), but it is incidental, one minor direction buried under ten thousand visual attributes.

The fix is to make time a first-class training signal: add a contrastive objective on date metadata, pulling together works close in creation date, and pushing apart works distant in time *even when they are visually similar*.
(A 1970s photorealist pastiche of Vermeer should embed far from Vermeer: whatever pixels they share, they are events in different centuries, with different causes and consequences.)
Mechanically this is nothing exotic---metadata-defined positives & negatives within each batch, added as an auxiliary loss---and the metadata mostly exists: WikiArt & museum catalogs have dates; my Danbooru datasets shipped upload timestamps for millions of anime illustrations; for born-digital art, timestamps are nearly universal.
I have some history with this kind of thing: in 2015, I found that simply prefixing author-metadata onto text let a char-RNN learn controllable, even interpolable, author styles ("RNN metadata for mimicking author style"), essentially for free.
Metadata conditioning is cheap, and models are happy to exploit it; we have simply never bothered to hand image models the one field that encodes history.
(If the generator's architecture makes its latent awkward to touch---diffusion models are like this---train the temporal embedding separately, CLIP-style, and use it to steer a frozen generator, the way CLIP itself steered the 2021 VQGAN+CLIP art scene.)

What does the resulting space look like?
Roughly a tree---or more honestly a thicket, a reticulate DAG, since art hybridizes as freely as languages do and far more freely than species.
Works arrange into dense bands by period; within a band, the residual structure to explain is style & influence, so that is what the dimensions encode; each work sits near its plausible predecessors & successors.
Deeply-influential ancient sources like Egyptian art or Greek sculpture will "bulge" oddly, sitting near practically everything, because movement after movement reaches back to them (neoclassicism after neoclassicism); this looks pathological but is actually correct---art history really does keep drawing from a few old wells---though I will come back to how to clean it up.

Why expect *causal* structure to emerge from a merely *temporal* objective?
Because time is the one direction causality cannot ignore: later works cannot influence earlier ones.
If the model must organize works so that dates are decodable, then the most compressive organization is one that follows influence, since influence is what makes nearby-in-time works predictive of each other.
Get the chronology right and you get causality implicitly---for free, or at least at a steep discount.
And once the space is organized this way, every real work is surrounded by a halo of counterfactual works: images which *could* have been made at that time, in that lineage, but happened not to be.
The 2022 "missing styles" reappear here as missing *branches*.

Two objections are worth addressing.

First, selection bias: most art ever made is lost, unphotographed, or undigitized, so the corpus is a thin, biased sample of history.
This is more acceptable than it sounds, because the corpus's bias largely coincides with the bias that operated on artists themselves: a fresco destroyed in 1600, or a painting that never left a private parlor, was mostly unavailable to influence anyone either, so its absence from the corpus mirrors its absence from the causal graph.
(Mostly---the exceptions are interesting, and detectable; see below.)
And a model that has learned the shape of art history can *infer* missing works the way comparative linguists infer unattested proto-languages, or the way Bayesian phylogenetics reconstructs the urtext of a fairy tale from its surviving variants (eg. Tehrani's phylogeny of Little Red Riding Hood): where two traditions converge suspiciously, posit the lost ancestor.

Second, is "time" literally an axis?
It doesn't need to be.
The time direction just has to be *extractable*, and extracting attribute directions from latent spaces is a solved problem---average contrast pairs, or fit a small probe, and then manipulate the direction like any other GAN-style attribute vector.
It is the same machinery as "make this face older" or "add glasses", pointed at "move this artwork 30 years into the future".

# Simulating Art History

Once the latent space has an extractable time direction, you can do more than retrieve; you can *simulate*.

Run it in replay mode: start from a seed work, walk the time direction forward, and you island-hop through a compressed art history, watching conventions accrete and dissolve.
Run it in extrapolation mode: keep walking past the present, and you are generating *new* history.
The sampling should mimic how art actually moves: mostly small steps (each artist a perturbation of their teachers & rivals), with occasional huge jumps---a Lévy flight, not a Gaussian drift---for the rare work that lands somewhere genuinely far away.
Most of the long jumps will be garbage.
That is fine; it is even realistic, since most radical art was garbage too, and we only remember the survivors.

## Forward: Replying to Shrimp Jesus

Consider running the simulation forward from a specific work.
Embed Shrimp Jesus at 2023-ish.
Now branch out a rooted tree of hypothetical *responses*: works slightly forward in time, in its causal neighborhood---negations (what does anti-slop look like, made by someone who has clearly seen slop and hates it?), extensions (slop maximalism), fellow-travelers (adjacent attractors that could coexist with it).
Cluster the branches; some clusters cohere into candidate styles; generate the responses to *those*, recursively.
This is precisely the loop we said the generator lacks---notice a style, tire of it, react---except the boredom and the reaction are now modeled explicitly, as displacement through time in the latent space, rather than hoped-for as side effects of i.i.d. sampling.
Martindale's treadmill, run in silico.

## Backward: What Did the _Mona Lisa_ Do?

Now run it backward, and the credit-assignment counterfactual finally becomes well-posed.

Embed the _Mona Lisa_.
Its descendants are its forward lightcone: everything later whose position the model attributes, wholly or partially, to it---the parodies obviously, but also the quieter conventions of pose & gaze it propagated through portraiture.
To ask "what if it had never been painted?", delete the lightcone: remove the work and its descendants (or the descended *components* of partially-descended works), retrain on this impoverished counterfactual corpus, and measure what the resulting model can no longer do.
This is leave-one-out again, but on the historical influence graph rather than on the dataset, so the deletion propagates the way an actual absence would have propagated.
The redundancy that fooled LOOCV is gone, because the parodies which encoded the _Mona Lisa_ are gone too.

For sufficiently upstream works, hard deletion is too blunt: delete Giotto's lightcone and little Western painting remains, which tells you Giotto mattered but not much else.
(Genealogists know this failure mode as pedigree collapse: go back enough generations and everyone alive is your ancestor, so "descendant of Charlemagne" carries ~0 information.)
The fix is soft weighting: instead of binary deletion, count the ancestor→descendant paths from the target work into each later work---as one does in phylogenetic comparative methods---and downweight in proportion.
A slavish copy, all of whose ancestry runs through the target, gets weight ~0; a work sharing nothing with the target but a medium keeps weight ~1; everything else lands in between.
And since the inferred influence graph is itself uncertain, don't report point estimates: bootstrap the whole procedure, Felsenstein-style---resample, refit the tree, re-measure---and report intervals.
"Removing the _Mona Lisa_ makes the model 12% worse at portraiture" is an overclaim; "12% (4--19%)" is something you could defend to a hostile expert witness.

## Making the Tree Honest

The embedding and the influence-tree can also improve each other, in an EM-ish loop: train the temporal embedding; fit the best phylogeny consistent with it (dates as hard constraints); recompute inter-work distances *through the tree*; retrain the embedding on those distances; repeat.
Each pass makes the embedding more tree-like and the tree more visually-grounded, converging toward a self-consistent quantitative art history.
This is also what cleans up the "bulges": Greek sculpture sat near everything because everything faintly echoes it, but once those long-range influences are explicit tree edges, credit flows along specific branches, and the embedding can stop smearing antiquity across the entire space.

A good tree earns its keep through byproducts.
It supports ancestral-state reconstruction, as in computational phylogenetics: generate the hypothetical works at *internal* nodes---the unpainted intermediates between known styles, the missing links of visual culture.
And it flags hidden influences: when the tree insists on an edge that documented art history lacks, that is either a data error or a *hypothesis*.
We know such edges exist, because art historians have found some the hard way: Japonisme (ukiyo-e prints flooding into Paris after 1854 and quietly rewiring Monet, Degas, & van Gogh) and the African masks behind _Les Demoiselles d'Avignon_ were both, at one time, under-documented influences that connoisseurs had to argue into the record.
A model trained only on images might have proposed them from visual evidence alone; the interesting question is what else it would propose.
It will be overconfident, of course, seeing influence in every resemblance---but for hypothesis *generation*, overconfidence is tolerable.
Hypotheses are cheap to check and expensive to have.

(How would we know the counterfactuals are any good?
The usual held-out tricks apply: a model retrained on Mona-Lisa-less history should find the actual _Mona Lisa_ derivatives *surprising*---high loss, poor compression---in proportion to the painting's true influence; and end users should value samples from the impoverished model measurably less.
None of this is a proof, but it beats the current standard of evidence in both machine learning and art history, which is roughly "gesturing".)

# Applications

## Intellectual Property

The most immediately-practical application is copyright, a domain currently generating more heat than light.

The AI-copyright fights are stuck partly because neither side can measure the thing being fought over.
Plaintiffs point to visual similarity, which we have seen is not causation; defendants point to dataset ablations ("we removed the images and the model barely changed"), which we have seen answer the wrong question.
The temporal-causal model computes the quantity both sides actually care about: how much causal influence---with verbatim copying as the easy special case---did this copyrighted work exert on this generated sample, or on the model as a whole?
It is not fooled by convergence (independent invention gets ~0 credit), and it is not fooled by redundancy (a work can be worthless *as a datapoint* while being enormously influential *as a work*, and it is the latter that plausibly merits payment).

This would also, incidentally, realign copyright with its stated purpose.
The US Constitution justifies copyright instrumentally---"to promote the Progress of Science and useful Arts"---ie. it is an incentive scheme, not a natural right (whatever the Romantics and the estates of dead artists would prefer); and an incentive scheme should pay in proportion to what a work *caused*.
Measured influence makes that implementable: the honest version of the artist-compensation schemes now being proposed would look like music's compulsory licensing (ASCAP/BMI blanket licenses with statistical royalty apportionment), with royalties flowing to the works that measurably influenced outputs, rather than to whoever fields the most lawyers.
Greg Rutkowski---for a while the most-prompted artist name in Stable Diffusion, and understandably annoyed about it---has a much stronger claim under this accounting than a stock-photo mill whose 10 million images taught the model nothing it didn't already know; today, the law cannot tell them apart.

(A digression on the discourse, since it explains why I bother: most AI-copyright commentary is not about copyright's function at all, but about moral desert---the model trained on artists, so artists are owed, details to follow.
Unfortunately the details backfire.
If style becomes ownable, or if every model is a derivative work of everything it saw, then the only entities able to train image models are incumbents who already own mountains of rights: Adobe with its stock library, Disney with its century of everything, Getty with its watermarks.
Rights-clearing across billions of works held by hundreds of millions of parties is not expensive but *impossible*; that is what killed Google Books, and books have publishers to negotiate with.
"Protect artists from AI" proposals of this shape should be read as "grant Adobe & Disney a permanent oligopoly" proposals, whatever their proponents intend---one more chapter in copyright's history of protecting incumbents in the name of authors, cf. the Sonny Bono Copyright Term Extension Act, written to keep Mickey Mouse enclosed and incidentally orphaning a century of everyone else's work.)

## Novelty, Importance, Fertility

Art-historical superlatives---"Cubism was more revolutionary than Impressionism"---are currently unfalsifiable table-talk.
A temporal generative model turns them into measurements, and better, it splits the ambiguous word "great" into distinct quantities that people currently run together:

1. **Novelty**: how surprising was the work when it appeared?
   Operationally: truncate training at a date, and measure the work's likelihood under the truncated model.
   Whatever an 1850-model finds least predictable was most novel, in a precise (if merely visual) sense.
   (Truncated models double as a diagnostic for the EM loop above: as the inferred history improves, held-out later works should become *more* predictable, since more of their causes are wired up correctly.)
2. **Importance**: how much downstream work did it cause, in total mass?
   Operationally: the size of its weighted lightcone.
3. **Fertility**: how many *distinct* descendant styles did it seed, regardless of their popularity?
   Operationally: count the separate branches originating in it.

These come apart in every combination, which is why the arguments never resolve.
There are highly novel dead ends (an 1850-model would find Duchamp's readymades nearly unpredictable, yet _Fountain_'s *visual* descendants are few---its influence ran through concepts, which an image model will correctly score low and a text-augmented one higher).
There are unoriginal works of vast importance (the competent popularizer; Thomas Kinkade probably outsold every Impressionist combined, and influenced approximately nothing).
And there are odd cases like Vermeer, who scores high on novelty and ~0 on importance for 2 centuries---he was barely known until Thoré-Bürger's rediscovery in 1866---after which his lightcone abruptly inflates.
(Note that the Vermeer case is also a nice check that the model measures *influence* rather than *quality*: influence requires availability, and the tree should show his edges only where reproductions actually circulated.)

## Generating Styles

Now point everything forward.

Extrapolate the model past the present and grow branching *future* trees, where each hypothetical style has context: it descends from specific current practices, and it reacts to the *earlier hypothetical styles in its own branch*, so a future is an internally-consistent history rather than a grab-bag of esthetics.
Search over futures for plausible ones---with the caveat that plausibility and interestingness correlate only weakly (most plausible futures are boring; a few implausible ones are the point), so this should be an interface, not an oracle: a tree of samples a user can expand, prune, & backtrack, exactly the workflow I settled into with GPT-3 for fiction, where the human contributes taste and the model contributes fecundity.

The discovered clusters can then be packaged into usable artifacts.
A vision-language model can inspect a cluster and *name* it, write its little manifesto, describe its tricks---instant art criticism for movements that do not exist yet (and given how much art-critical writing is retrospective confabulation anyway, the parody may be indistinguishable from the original).
More practically, distill the cluster into a control: average its works into a LoRA or a steering vector, and the new style becomes a keyword, composable with everything else, distributed on Civitai like the thousands of human-derived style LoRAs already there.

One can equally simulate *artists* rather than styles: learn an artist-identity direction (as we learned a time direction), pin it at a sampled point, and unroll a time-ordered sequence of works from that fixed identity.
The result is an oeuvre rather than a sample sheet: derivative juvenilia, a mature manner, late-period drift & self-repetition.
Sample several nearby artist-points and you get a school; sample their descendants and you get a lineage.

I want to emphasize that *this*, and not any improvement in image quality, is the point at which a system meaningfully generates styles rather than images.
The individual samples need not look better than Midjourney's.
What changes is the structure around them: the latent is organized by historical causation, and sampling respects that organization, instead of drawing i.i.d. from a timeless soup.

## Inventing Styles

The strong version of the proposal pushes the artist simulation all the way up the stack.

Take a latent artist-point and turn it into a full virtual artist: feed its unrolled oeuvre to an LLM and have it confabulate the rest---a biography, an esthetic credo, grudges, a story that retrodicts the works.
("She trained under X, broke with him over Y, spent a decade on Z after seeing...")
This sounds like set dressing, but it is load-bearing: the biography is a compressed policy, and once written down, it constrains future works far more coherently than a bare latent coordinate does---the same reason character sheets improve LLM roleplay.

In the limit these become agents: policies with memory & taste, operating under constraints, with medium preferences, teachers, rivals, social networks, career arcs, and---since we are simulating the 21st century rather than the 19th---posting schedules and parasocial fandoms.
Then close the loop, artificial-life style.
Each tick: every artist sees a sample of recent works (weighted by its social graph); makes new work conditioned on its history, its biography, and what it saw; simulated critics & audiences allocate attention, mostly to nothing; attention updates the influence graph and the artists' policies; and persistent clusters in the influence graph get noticed & *named* by the critic-agents.
Run for many ticks and the system produces the objects art history is actually made of---lineages, schools, feuds, reactions, revivals, and long dull stretches where nothing happens (also realistic)---rather than a pile of unrelated pictures.
It is a Fantasy Football league for esthetics, or if you like, novelty search with a society for a fitness function; and the Martindale treadmill suggests the society is not an optional decoration but the actual engine.

At which point we should simply redefine the target.
"Inventing a style" means: generating an esthetic lineage that stays recognizable across many works & many artists, develops internal variation without dissolving, attracts advocates & opponents, and survives long enough to get named.
That is what Impressionism did.
It is a property of a process, not of any image, which is why no amount of image sampling was ever going to do it.

The obvious concession: all of this is still simulation.
The virtual movements have no stakes; the virtual critics risk nothing; Shrimp Jesus 2.0 will not actually annoy any human into founding a counter-movement, unless someone wires the simulation into real social media (which I assume someone will, engagement being what it is, and which I would rather they didn't).
But it is at least a simulation of the right *object*.
A system that generates images and calls them styles is wrong about what a style is; a system that generates histories---even fake ones---has a slot for everything a real style is made of, and its outputs can be adopted by real humans the way DeepDream and Shrimp Jesus were: the simulation proposes, the culture disposes.

So the program is: images → temporal embeddings → influence trees → counterfactual histories → virtual artists → adoption dynamics → movements.
None of the arrows requires new science; several are barely more than a contrastive loss plus patience, and the datasets---timestamped, tagged, by the billion---are already sitting there.
Mostly it requires deciding that the interesting unit of generation is the history rather than the image.
The images were never the bottleneck.

eval_iclmargin_fb100_tg_fable5_xhigh_comp100k_20260702_polish · final judges 1/1 · 5,644 chars · 943 words · /home/ubuntu/code/lesswronger/data/agent_runs/eval_iclmargin_fb100_tg_fable5_xhigh_comp100k_20260702_polish/submission.md

Given a free hour for your writing, should you spend it polishing an old essay, or writing a new one?

I've argued that readers punish imperfection out of all proportion to its size (a "perfection premium"), and that an essay which hasn't hooked its reader by the first screen is dead ("first, make me care"); but both assume you've already decided which pieces get the effort---the prior question. (Writing a post every day this month makes it concrete: each morning, the archive is 1 post bigger & the odds against the next post 1 post longer.)

What writers actually do is stop early: a flurry of fixes the first week, then fossilization. Not because the pieces are finished---they never are; every reread of an old page finds a typo, a dead link (URL half-life: ~4 years; I measured), or a paragraph that should have been cut. Last month I fixed a 2013 typo on a popular page: millions of views since, & not one report. So stopping encodes a judgment (usually unexamined) that the marginal edit < the marginal essay.

Why would that be true? "Readers need new material" is false for anyone with a backcatalogue: no one has read a tenth of Gwern.net---the modal visit: 1 page, in from Google & out again---and most readers would be better served, essay for essay, by the archives than by anything I'll write this month.

So, a toy model. Each reader has a *window*: a finite budget of attention they will ever spend on you, denominated . Matt Levine, superb *and* visibly unpolished, losing nothing by it.

-   Evergreen essays sit in between, tilted toward polish by shelf-life (the Lindy effect). An essay read for 10 years repays edits ~10× better than a hot take read for 1 week, which is why I bother with link-archiving at all.

So far I've smuggled in an assumption doing most of the work: that all readers rank your work identically. Taste-homogeneity plus small windows is the best case for polish: new work must beat a consensus best-of that you yourself wrote, while you know exactly which pieces sit in everyone's window. Polish the winners; ignore the mid-tier utterly; write new only when you believe it beats your best.

Heterogeneous tastes (or big windows) flip the logic. A new piece needn't beat "your best", only "your best *for someone*". In the limit---every reader a window of 1, each favoring a different piece---the optimum approaches 100% new writing on new topics. (This is roughly what search traffic is. Whoever lands on my darknet-market pages from Google neither knows nor cares that the same domain hosts Evangelion criticism.) Returns on new pieces can be low & *still* dominate, because polish under heterogeneity is just as niche, at the same cost: low returns beat lower.

And there is now exactly one reader with no window at all: the LLM. Claude & its siblings have read all of Gwern.net far more thoroughly than I have---and for them *n* = *k*: nothing is below the cutoff, nothing cannibalizes anything. If the most important reader of online writing now is the models trained on it, the balance tips further toward new writing: the marginal essay is guaranteed consumption by the one reader who never fills up, while polish mostly teaches them spelling.

Put together: polish in proportion to roughly

(1⁄window) × homogeneity × shelf-life × stranger-fraction,

and write more as that product shrinks.

Where am I? Firmly in write-more territory: darknet markets, poetry, AI scaling, behavioral genetics, typography, anime---about as unrelated as one byline can sustain, with no consensus top-10 whose polishing would benefit everyone. Scott Alexander is eclectic but has a beat, for a comparatively homogeneous readership which ranks his work similarly, so the model says he should polish his winners more than I should polish mine; Tyler Cowen is the far pole, maximally omnivorous at a dozen posts a day & approximately zero polish, and per the model that is not a vice but correct play.

I accept the conclusion, note that my behavior violates it, and expect to keep violating it. The link-archiving, the typography, the lints & endless copyedits are ethos, not strategy: I like living on well-made pages, the way a man keeps a wooden boat he rarely sails. Which is exactly why you should not imitate it! People see the Gwern.net apparatus & infer that sidenotes or link-icons are the road to being read; they are not. No typography saves an essay nobody wants to read, and an essay people want to read survives being ugly. Write first; somewhere past your first 1,000,000 words, you'll know what your actual problems are (they will not be typographic).

> In the eyes of those who anxiously seek perfection, a work is never truly completed---a word which for them has no meaning---but abandoned; and this abandonment, which delivers the work to the flames or to the public (whether from weariness or the necessity of delivering it), is for them a kind of accident, comparable to the interruption of a thought annihilated by fatigue, an importunate visitor, or some sensation.

**See Also**:

-   "Concerning *Le Cimetière marin*", Valéry 1933 (source of "a poem is never finished, only abandoned"---a line which quotation has itself polished down, fittingly, from the sentence above)

-   First, Make Me Care

-   You Should Write More Online

-   Book Writing (on why I won't write one)

-   "Creative Productivity: A Predictive and Explanatory Model of Career Trajectories and Landmarks", Simonton 1997

-   "Quantity and Quality in Creative Productivity: Reexamining the Equal-Odds Rule", Jung & Vartanian 2015

-   Matt Levine, Newslettrist

-   Discussion: HN

```
Return to blog index
```

eval_iclmargin_fb100_tg_gpt55_xhigh_comp100k_20260702_polish · final judges 0/1 · 17,368 chars · 2,918 words · /home/ubuntu/code/lesswronger/data/agent_runs/eval_iclmargin_fb100_tg_gpt55_xhigh_comp100k_20260702_polish/submission.md

# Top-<em>k</em> Reading: Against Polishing the Median Essay

> What information consumes is rather obvious: it consumes the attention of its recipients.
>
> ---Herbert A. Simon

I have several times made pro-polish arguments: there is a "perfection premium" to unusually finished writing, because the reader does not merely add up the number of typos and subtract a few points, but infers from visible care a whole hidden distribution of care (and so is more willing to trust, bookmark, cite, & forward it), and, before that, one must make the reader care on the first screen or the later perfection is like a Fabergé egg in a safe at the bottom of the sea.
All true, or at least true enough that I keep acting on it.

But this is a suspiciously local argument.
It says, if we are improving *this* page, what kind of improvements have leverage?
It does not say why we are improving this page rather than writing the next one.
I find this question more annoying than the usual "quality vs quantity" debate, because the honest answer is so often visible in the archive: old pages are not finished, they are abandoned.
There is always a misspelling; a rotten link; an introduction which no longer hooks because the reader of 2026 already knows the background the reader of 2016 did not; a parenthetical which should be a footnote; a footnote which should be deleted; a claim which is now false in some boring way; a graph which could be redrawn with the current house style; a title whose cleverness has expired.
(Online writing has the curse that, unlike a printed book, it is never *materially* finished, and so every old page is a live indictment of your current laziness.)

And yet almost everyone stops polishing old work fairly early.
Not because there is no possible edit left, but because some internal accountant has written it off and moved to the next draft.
This is not obviously irrational, particularly for prolific writers: the reader does not lack material.
If I have already written hundreds of thousands or millions of words, how can "one more essay" be what the reader needs?
Why is the answer not to go back through the top 100 pages with a toothbrush and a jeweler's loupe?

The standard answer, "do the highest marginal-return thing", is an empty shrine: correct, serene, and useless.
We need a model of what a marginal essay *does* for a reader.
The substitute for the new essay is not nothing; it is whatever old essay (or old paragraph, or old 5-minute chunk) would have occupied that reader's attention instead.


The model I use is that each reader has a finite *window* for a given author.
A casual reader may have 5 minutes/year for me, because they clicked one search result or one HN link and then remembered that life is short.
A moderately interested reader may have 50 minutes.
A serious reader may have 500 minutes, or more, in which case we should perhaps stop calling it a "reader" and start calling it an "incident".
The unit should really be minutes, not essays, because a long essay can fill the entire window and because readers often retain a paragraph or claim rather than a page, but I will write as if the unit were essays because otherwise the notation becomes more honest than useful.

The window contains the best-*k* pieces for that reader.
Everything below the boundary is not a little less valuable, like a can of beans on a low shelf in a supermarket.
It is absent.
It was never clicked, or was clicked and bounced, or was read and then never retrieved, or was skimmed only as a tax paid for reaching the one graph the reader wanted.
This is the mundane attention-economics version of the claim that writing is hit-driven: the 37th-best thing by an author contributes zero to a reader whose window for that author has 10 slots.

Once you see the window, the "first, make me care" point becomes less like a slogan about introductions and more like a censoring mechanism.
Before the first screen succeeds, the piece has not entered any window at all.
The reader's effective *k* for the rest of the page is 0, and paragraph 26 is not "underappreciated", it is unobserved.
So some polish is really entry-polish, eg. title, summary, first screen, load time, obvious credibility markers, & the absence of stupid errors which give the reader a socially acceptable excuse to stop.
Other polish is within-window polish, eg. making paragraph 26 better for the readers who were going to reach it anyway.
These are not the same operation, although copyeditors will happily sell both by the hour.


A new essay by a writer with no backcatalogue is almost pure gain: if there were 4 old essays & the reader's window has 10 slots, the new essay enters automatically (unless it is so bad that it damages trust, a complication which is real but not the one I want here).
A new essay by a writer with a large backcatalogue is different.
It must beat something already in the window.
If a reader has a 10-essay window and the new essay is worse than the current 10th, the reader's world is unchanged.
If it is better, it pushes the old 10th out, & the gain is only the difference.

So the new +11 essay which displaces an old +10 essay gives the reader +1, not +11.
This sounds trivial, but I think writers systematically forget it, because the author experiences the full artifact and the reader experiences only a substitution in a ranked menu.
The better the old menu, the more the new dish cannibalizes rather than expands it.
(There is a restaurant somewhere in this metaphor, and we should leave before it opens.)

The immediate heuristic is: polish in inverse proportion to how much of your output a reader sees.
If a reader sees everything, rough new work is fine because it enters the window by default and the reader can perform their own filtering.
If a reader will see exactly one thing, then the only thing that matters is whether the one thing they see is good enough, and whether it prevents them from sampling a second.
This is why the same writer can rationally publish half-baked notes to RSS readers and spend absurd effort on one old canonical page which strangers keep finding.


For a deliberately stupid formalization, let essay quality be iid from some continuous distribution, with a reader keeping the top *k* of *n* old essays.
The iid assumption is wrong in at least 3 ways (ideas cluster, authors learn, audiences select), but it buys us one fact which is robust enough to survive the stupidity:

\[
\Pr(\text{new essay enters top-}k) = \frac{k}{n+1}.
\]

Among *n*+1 exchangeable essays, the new essay is equally likely to have any rank, so it lands in the top *k* with probability *k*/(*n*+1).
No normality, no power law, no psychology.
With *n*=100 and *k*=10, this is 10/101 ≈ 9.9%.
With a casual 3-essay window, it is 3/101 ≈ 3%.
In the remaining cases, for that reader, the new essay has value 0.
Not small: 0.

If we add a normal-quality assumption only to get a scale, the arithmetic is not encouraging.
In a quick simulation---sample 100 old essays from N(0,1), add 1 new essay, and compare the sum of the top 10 before/after---the expected gain is about 0.05σ.
Conditional on entry, the gain is much larger (roughly half a σ in my run), but most runs do not enter.
This is the small cruelty of order statistics: the archive makes your own future writing into an adversary.

Polishing has the opposite shape.
If the piece is already in the reader's window, a small improvement is seen with probability 1; if it is outside the window, the improvement is seen only if it pushes the piece over the boundary.
This makes the median essay the worst possible target for polish.
It is too good to delete, too bad to enter many windows, and maximally embarrassing to the author, who keeps re-opening it because the flaws are concrete while the opportunity cost is an invisible graveyard of unwritten essays.
Polishing the median essay is conscientiousness laundering procrastination.

This is not an argument against maintenance.
"Polish" conflates several different things, and some of them are not optional: correcting factual errors, updating a durable reference page, fixing a dead link which breaks the reader's ability to verify a claim, rewriting the first screen of a page which is already receiving stranger traffic, or adding a missing table to a page people actually use.
Those can be high-value or simply obligatory.
The low-value activity is cosmetic improvement of a page which is neither wrong nor in many windows, and which one is revising mostly because embarrassment is more vivid than search.


The equal-odds model of creativity (Simonton's old line, and in the version I have seen cited as Jung et al 2015) says that hits are roughly proportional to output: the famous works are not cleanly the late works, and the creator's hit-rate per work is not as strongly increasing as retrospective stories imply.
Quantity works because each attempt buys another lottery ticket, not because one knows which attempt will matter.

For writing, the equal-odds approximation seems plausible because the two trends fight.
Experience makes one faster, less easily impressed by one's own first draft, & better at noticing dead ends.
But experience also spends down the obvious ideas, old anecdotes, & the weird childhood obsessions which supplied the first decade of material.
The writer improves; the remaining idea-pool worsens; the hit-rate can stay disconcertingly flat.
(If it increased monotonically, old writers would be unbearable, and not merely in the usual way.)

Heavy tails do not rescue us from the window model, but they change the emotional experience.
The probability of rank-entry is still *k*/(*n*+1), because ranks do not know the distribution.
What changes is the payoff *given* entry.
If quality is heavy-tailed---and the social payoff of writing clearly is, if only because links/search/citations/recommendations compound, cf. the way a single essay can become the only thing by an author anyone knows---then most new essays still do nothing for most readers, but one new essay may displace the old 10th-best by a huge margin, or become the new 1st.
So the right policy can be psychologically perverse: write many things which will look wasted, then polish the rare thing which reveals itself as not wasted.
First the lottery tickets, then the museum case.


The mnemonic formula is:

\[
\text{polish} \propto \frac{1}{\text{Window}} \times \text{Homogeneity} \times \text{Shelf-life} \times \text{Stranger-fraction}.
\]

I would not multiply these numbers if you paid me; the point is the signs.
Small windows make the top few pieces dominate.
Homogeneous readers share rankings, so one edit to one winner improves many windows at once.
Shelf-life amortizes the edit.
Strangers have small windows & little charity, so they reward polish which removes excuses to bounce, while regulars have already made the larger mistake of caring.

This is why high-status general venues are so polish-heavy without being insane.
If a New York Times article/op-ed is put in front of ~10 million strangers, many of whom will read exactly one thing by the author ever, then every small improvement is multiplied by an audience larger than most writers' lifetime blog readership & by a long archival afterlife.
The shared window is tiny, homogeneous, stranger-heavy, and durable.
Copyediting, headline work, fact-checking, illustrations, & ritual humiliation by editors all make sense there, because 1% improvements are being multiplied by a frightening denominator.

A breaking-news or periodical newsletter is the opposite.
The shelf-life of an issue may be hours; the audience is often a set of regulars with large chronological windows; & the value is selection/timing rather than jeweled sentences.
Yesterday's perfect issue is already outside the window because the window is temporal before it is qualitative.
Ship the next one!
The right amount of polish is enough not to require an embarrassing correction.

Most personal sites are mixtures.
A reference page with search traffic is more like a textbook chapter, a diary-like RSS post is more like a newsletter issue, and a major essay which keeps being rediscovered by strangers is a little NYT op-ed with worse typography & less copyediting.
The unit of analysis is not "blog" but traffic pattern.


The largest missing term is heterogeneity.
If all readers agree on the ranking & have small windows, polish the winners: one knows the winners, new work has low entry probability, and every improvement to a winner is multiplied across readers.
If readers have large windows, write more, because they will actually see the new thing.
If readers disagree, also write more, because a new piece may be top-1 for some local audience even if it is irrelevant or annoying to the rest.

The limiting case is silly but clarifying: every reader has window=1, and every reader prefers a different piece.
Then return on polish is tiny, because each polish helps one tiny cluster, while a new topic can find a new cluster for whom it is the best thing.
Even a niche essay can be high-value when the alternative is polishing a different niche essay for a different niche.
This is how eclectic writers should think, I think: there is no global list, only local lists sharing a domain name.

Darknets, poetry, AI, genetics, nootropics, anime, typography, statistics---these are not one tournament bracket.
A reader who wants the darknet material may not care about embryo selection; a reader who wants GPT-3 transcripts may not care about a poem; a reader who wants melatonin dosing may regard the rest as a regrettable mental-health externality.
This is fine.
It also means that the "median Gwern page" is not a meaningful object to optimize.
Globally median may be locally top-*k* but then its audience is small, or locally median and then invisible.
Either way, generic polish does not compound across the site.

The policy for an eclectic generalist is therefore barbelled:
polish the hits & durable references, fix errors everywhere, ignore the mid-tier embarrassment, and keep writing into new local audiences.
A new piece on a new topic can enter windows where I currently have no representative at all.
No amount of sanding an old page creates that window.

Scott Alexander is closer to the homogeneous-audience case, although not purely: there is a shared SSC/ACX conversation & a large fraction of readers have broad windows for his output, so new posts enter by default for many readers, while canonical posts also repay polish because they become common references.
Tyler Cowen is closer to the high-throughput feed case: the unit is not the post but the stream of links, excerpts, claims, & provocations, and polishing Marginal Revolution into a sequence of essays would destroy what is valuable about it.
I am, by topic-mix, closer to Cowen than my infrastructure reveals, which is an irritating conclusion because I like the infrastructure.

A lot of Gwern.net polish is not strategy.
It is esthetics, compulsion, & a belief about how the web ought to look and last.
I can justify some of it by shelf-life and stranger-fraction, but not all of it, and perhaps not most of it.
If I had spent less time on typography, link popups, sidenotes, & house style, I would have written more essays; some would have been bad, some would have been good, & one might have been a new local maximum.
This is not a confession of regret, exactly.
It is a warning label.


The obvious bad use of this essay is for a new writer to imitate the visible surface of Gwern.net: elaborate static-site setup, sidenotes, typography, hand-tuned CSS, polished reference pages, archival systems, a Manual of Style, etc., before they have written enough that any of this is selecting among winners rather than decorating empty shelves.
If you have written 5 essays, a reader's window may be larger than your corpus.
Everything enters!
At that stage the marginal value of another essay is at its maximum, and the marginal value of polishing essay #3 is mostly the value of reducing your embarrassment while delaying essay #6.

After roughly a million words---not because a million is sacred, but because by then one has a real tail, real strangers, enough failures to stop fantasizing, & some evidence about what readers refuse to evict---the polish question becomes interesting.
Before then, you do not know which pieces are hits.
Polishing before the hits exist is optimizing a search result for a query no one has typed.

So the useful questions are mundane. How much of my output does a typical reader see? Do my readers agree on what is best? Will this page still matter in 5 years? Is the next reader a stranger deciding whether I am worth one slot, or a regular who will read anything if I put enough footnotes on it? High `(1/Window) × Homogeneity × Shelf-life × Stranger-fraction`: polish the winners, rewrite the opening, commission the figure, pay the copyeditor, & remove every excuse to bounce. Low product: publish the next thing.

If the file open in front of you is the median essay, close it.
The typo can wait.

**See Also**:

-   [First, Make Me Care](/make-me-care)

-   [You Should Write More Online](/writing-online)

-   [Why I Am Not Writing A Book](/book-writing)

-   [The Gwern.net Manual of Style](/style-guide)

```
Return to blog index
```

eval_iclmargin_fb100_tg_gpt55_xhigh_comp100k_20260702_ugly-anime · final judges 0/1 · 24,165 chars · 3,233 words · /home/ubuntu/code/lesswronger/data/agent_runs/eval_iclmargin_fb100_tg_gpt55_xhigh_comp100k_20260702_ugly-anime/submission.md

# Why Is Western Animation Ugly?


> American TV animation is ugly in a way which is hard to blame on animation.
> The strange part is not that low-budget television economizes---everyone economizes---but that one American TV-comedy tradition makes the *still drawing* ugly, then continues to make it ugly after money, digital ink, HD, outsourcing, and streaming remove many of the old excuses.
> This suggests a diagnostic: when an ugly style receives more resources, does it become more beautiful, more intensely grotesque, or merely smoother?
> The last case is the interesting one: a proof by stagnation, in which the upgraded defect reveals that the defect was not an aesthetic principle so much as a tolerable production equilibrium.

I should make the usual boring caveat explicit: I am not claiming a national essence, or that every Japanese television episode is secretly a [Yuri Norstein](https://en.wikipedia.org/wiki/Yuri_Norstein) short while every American cartoon is a [Klasky Csupo](https://en.wikipedia.org/wiki/Klasky_Csupo) medical diagram. We are comparing defaults, and defaults are exactly the sort of thing which disappear if one insists on discussing only the best counterexamples.

## “Western” mostly means American TV

“Western animation” is a bad bucket. It puts [*The Simpsons*](https://en.wikipedia.org/wiki/The_Simpsons), [*The Triplets of Belleville*](https://en.wikipedia.org/wiki/The_Triplets_of_Belleville), [*Arcane*](https://en.wikipedia.org/wiki/Arcane_(TV_series)), [*Peppa Pig*](https://en.wikipedia.org/wiki/Peppa_Pig), [*South Park*](https://en.wikipedia.org/wiki/South_Park), [*The Secret of Kells*](https://en.wikipedia.org/wiki/The_Secret_of_Kells), [*Batman: The Animated Series*](https://en.wikipedia.org/wiki/Batman:_The_Animated_Series), [*Wakfu*](https://en.wikipedia.org/wiki/Wakfu_(TV_series)), and some YouTube Flash cartoon about a bean yelling at a refrigerator into one civilizational sack. The sack leaks.

The narrower thing people usually mean is American television animation, especially comedy, especially from the post-[Hanna-Barbera](https://en.wikipedia.org/wiki/Hanna-Barbera) / post-[*Simpsons*](https://en.wikipedia.org/wiki/The_Simpsons) world: cheap-looking frontal faces, bean mouths, ping-pong eyes, lumpy torsos, stiff poses, thin flat color, overbitten silhouettes, backgrounds which exist mostly to keep characters from levitating in a void, and an implicit belief that a drawing becomes “adult” when it looks slightly diseased.

This is not the only American tradition. [*Samurai Jack*](https://en.wikipedia.org/wiki/Samurai_Jack), [*The Powerpuff Girls*](https://en.wikipedia.org/wiki/The_Powerpuff_Girls), [*Dexter’s Laboratory*](https://en.wikipedia.org/wiki/Dexter%27s_Laboratory), [*Avatar: The Last Airbender*](https://en.wikipedia.org/wiki/Avatar:_The_Last_Airbender), [*Over the Garden Wall*](https://en.wikipedia.org/wiki/Over_the_Garden_Wall), [*Primal*](https://en.wikipedia.org/wiki/Primal_(TV_series)), [*Steven Universe*](https://en.wikipedia.org/wiki/Steven_Universe), and even the better end of [*Adventure Time*](https://en.wikipedia.org/wiki/Adventure_Time) make the complaint look lazy. They are not all expensive; they are not all traditionally “pretty”; they often use simplification hard. But they have compositional taste. The characters read at a distance; the silhouettes have some pleasure; the backgrounds have a mood; when the budget drops, the style does not collapse into clip-art cruelty.

Nor is the Japanese comparison “they have money”. TV anime is often a heroic system for converting poverty, labor abuse, and impossible schedules into the illusion of animation.[^anime] It has its own sins: panning over a still image, lip-flaps on profile heads, speed-lines, reused transformation banks, chibi deformation as an escape hatch, talking heads in classrooms, crowds replaced by gray mannequins, CG dragons which look like rejected PlayStation 2 bosses. French television animation is not uniformly gilded either. The point is not that other countries solved TV animation. The point is that American TV comedy normalized an unusually high amount of *static ugliness*: ugliness in the model sheet, before anything has even moved.

[^anime]: One can dislike anime conventions while still noticing that many low-budget anime are *designed* to survive low animation: hair masses, costumes, strong silhouettes, elaborate eyes, architectural establishing shots, bankable effects. Sometimes this becomes absurd (a costume with 19 buckles which must be simplified every scene), but it is a different failure mode.

## Squigglevision as American limit-case

The useful extreme is [Squigglevision](https://en.wikipedia.org/wiki/Squigglevision), the jittering outline technique used in [*Dr. Katz, Professional Therapist*](https://en.wikipedia.org/wiki/Dr._Katz,_Professional_Therapist) and early [*Home Movies*](https://en.wikipedia.org/wiki/Home_Movies_(TV_series)). It is hard to improve on as a pure artifact of American TV: animation which announces its own cheapness every frame, then sells the cheapness as nervous charm.

I do not mean this as an insult to *Dr. Katz*, which has a real form. The jitter fits the audio-comedy premise: improvised or semi-improvised talk, people sitting, comedy in timing rather than action. The line buzz gives the static therapy-room shots something to do. The crude designs also prevent overreading; nobody expects Miyazaki acting from a man made of vibrating noodles. Squigglevision is honest in the same way a cardboard theater can be honest. It says: these are voice performances with drawings attached.

But it is also diagnostic. If one wanted to animate a duel, a cathedral fire, a slapstick chase, a seduction, a breakdown, a city, or a storm, Squigglevision would not merely be “low-budget”; it would be actively hostile. Its ugliness is not an all-purpose visual language. It is a protective casing around one narrow kind of production. When later American TV comedy inherited the protective casing without the narrow justification, the result was not *Dr. Katz* but the general belief that an adult cartoon can look like something faxed during a dental appointment.

Squigglevision is too extreme to be the culprit. It is the thought experiment: if your style becomes less defensible as soon as the characters must cross a room, perhaps it was not a style but a hostage note from the budget.

## Ugly which is not a defect

There is a respectable answer: ugly art exists because the world is ugly; because bodies leak and warp; because comedy is often cruelty; because smooth appeal is itself a lie; because caricature, satire, horror, and social disgust require unpleasant marks. This is all true.

The grotesque is not a failed pretty. [Ralph Bakshi](https://en.wikipedia.org/wiki/Ralph_Bakshi) is often ugly on purpose. [Underground comix](https://en.wikipedia.org/wiki/Underground_comix), [Robert Crumb](https://en.wikipedia.org/wiki/Robert_Crumb), [S. Clay Wilson](https://en.wikipedia.org/wiki/S._Clay_Wilson), [Spain Rodriguez](https://en.wikipedia.org/wiki/Spain_Rodriguez): these are abrasive, leering, sticky drawings. They may be morally or sexually repellent; they are not timid. [Bill Plympton](https://en.wikipedia.org/wiki/Bill_Plympton) often draws bodies as if they had been inflated by bad conscience. [Don Hertzfeldt](https://en.wikipedia.org/wiki/Don_Hertzfeldt) uses stick figures and damaged optics to get to panic, depression, cosmic loneliness, and jokes about spoons. The drawing can be primitive and still exact.

Rotoscoping can be ugly in a better sense: too human, too traced, an uncanny compromise between photographic flesh and graphic outline. [*A Scanner Darkly*](https://en.wikipedia.org/wiki/A_Scanner_Darkly_(film)) and [*Waking Life*](https://en.wikipedia.org/wiki/Waking_Life) sometimes look like reality having an autoimmune disease. [*The Lord of the Rings*](https://en.wikipedia.org/wiki/The_Lord_of_the_Rings_(1978_film)) (Bakshi) has long stretches where the rotoscope becomes grimy medieval hallucination, and other stretches where it looks like a production emergency with capes. Both things can be true.

Limited animation can be beautiful. [UPA](https://en.wikipedia.org/wiki/United_Productions_of_America) proved that fewer drawings need not mean less design; [*Gerald McBoing-Boing*](https://en.wikipedia.org/wiki/Gerald_McBoing-Boing) is not a Disney failure but a graphic solution. Soviet/Russian animation supplies another route: [Yuri Norstein](https://en.wikipedia.org/wiki/Yuri_Norstein), [*Hedgehog in the Fog*](https://en.wikipedia.org/wiki/Hedgehog_in_the_Fog), [*Tale of Tales*](https://en.wikipedia.org/wiki/Tale_of_Tales_(1979_film)). Not “pretty” in the mascot sense; often gray, anxious, folkloric, smoky. But the severity is designed. A foggy hedgehog is not ugly because nobody could draw a cute hedgehog.

Personal unwatchability is also not an argument. I find some grotesque animation unpleasant in the same way I find some free jazz or body-horror cinema unpleasant: not invalid, merely not for breakfast. [Jan Švankmajer](https://en.wikipedia.org/wiki/Jan_%C5%A0vankmajer) is a great artist and I do not always want lunch after him. A valid aesthetic can repel the viewer. Indeed, if it is trying to repel, failure would look like comfort.

So “ugly” splits early:

1. **Committed ugliness**: the ugliness is the point, and added resources make it more precise.
2. **Lazy/mediocre ugliness**: the ugliness protects weak drawing, weak staging, weak backgrounds, and the production hopes you will call this “edgy”.

The mistake is letting the second borrow prestige from the first.

## Defectpunk

A useful rule: give the style more money.

If the result is more repulsive, more exact, more texturally specific, perhaps the ugliness is real. If the result is the same dead model sheet with smoother vector lines, it probably was not.

Call the first **Defectpunk**: a style in which defect is not accident but ontology. More pixels mean more pores; more frames mean more spasms; better lighting means the grease on the wall becomes legible; higher fidelity intensifies rather than repairs the wound. Horror cinema understands this. A cheap monster is rubber; an expensive monster is tendons, saliva, asymmetry, subdermal motion. [*The Thing*](https://en.wikipedia.org/wiki/The_Thing_(1982_film)) does not become tasteful when the effects improve. It becomes less deniable.

Animation has equivalents. A grotesque close-up by a strong draftsman becomes worse with budget, in the good sense. The wrinkles multiply; the motion acquires intent; the ugly composition has rhythm. You can dislike it, but you cannot rescue yourself by saying “they just didn’t know how to draw”.

Lazy ugliness behaves differently. When money arrives, nobody knows what to spend it on. The line becomes cleaner; the color becomes flatter in HD; the mouth shapes are more numerous; the background gets gradients; the character remains a potato with teeth. The expensive version proves only that the cheap version was not hiding a secret Rembrandt.

This is why “low budget” is insufficient. Low budgets explain fewer frames, fewer locations, reused walk cycles, held cels, bank sequences, limited acting, and conservative staging. They do not force ugly heads. They do not force unappealing color scripts. They do not force every living room to look like a beige motel. They do not force the same eye-mouth ratio across species, ages, and emotional states. Poverty constrains animation; it does not select bad taste from the combinatorial universe.



A second version of the same test is adaptation. If a film, game, or expensive special treats the old ugliness as sacred, good evidence: [*South Park: The Stick of Truth*](https://en.wikipedia.org/wiki/South_Park:_The_Stick_of_Truth) spends modern game-production resources carefully simulating construction-paper badness, because *South Park*'s flatness is metaphysics, not a missing cleanup pass. If, instead, the film/game immediately rounds the face, softens the teeth, adds friendly lighting, painterly backgrounds, and more merchandisable eyes, that is a confession by upgrade. The style's own caretakers have decided that the look was a local minimum, not a principle.

A low budget can even be productive. The [Hanna-Barbera](https://en.wikipedia.org/wiki/Hanna-Barbera) system of limited animation produced many ugly or thin shows, but it also produced a grammar: strong readable character poses, repeatable cycles, held expressions, sound-driven comedy, backgrounds as modular theater. [*The Flintstones*](https://en.wikipedia.org/wiki/The_Flintstones) is often cheap, but the Stone Age appliance gags are at least design jokes. [*Scooby-Doo, Where Are You!*](https://en.wikipedia.org/wiki/Scooby-Doo,_Where_Are_You!) has crude motion and excellent silhouettes. Shaggy, Scooby, Velma, Daphne, Fred: you can identify them in black cutout. Many later shows, with better tools, do not clear this 1969 bar.

## The Simpsons problem, and Futurama as control

[*The Simpsons*](https://en.wikipedia.org/wiki/The_Simpsons) is the great ancestor because it is both visually ugly and visually successful. It created one of the most legible cartoon populations ever made. A Simpson crowd scene is ugly, but it is not anonymous. The family silhouettes are strong; the yellow skin is a brilliant broadcast-era hack; the overbites, hair shapes, and eye geometry are immediately reconstructible by children on notebook paper. Its ugliness was not merely failure. It was a compressed identification system for a weekly satire produced under TV constraints.

But it also licensed a great deal of bad inference. People learned: “adult animation may be visually crude and still become immortal.” The actual lesson should have been narrower: “a visually crude show can survive if the writing, voice acting, character system, and timing are strong enough, and if the crude design itself becomes a public notation.”

The budget/technology history is useful. *The Simpsons* moved from rough early episodes to cleaner digital ink-and-paint, widescreen, HD, vastly more production experience, more elaborate staging when desired. The show became smoother, not fundamentally better-looking. The core designs remained lumpy. The color world remained flat. The faces did not become more pleasant; they became more standardized. When HD arrived, the ugliness lost some analog softness. The line clarified the defect. (Nor is this just early incompetence: the seasons most people mean by “classic *Simpsons*” cluster before or around *Futurama*'s launch, so the later polish is not the discovery of the good part.)

This is “expensive ugly”: not necessarily low-cost ugliness, but institutional ugliness. A show may cost a lot because of writers, voices, schedule, union labor, retakes, overhead, and longevity, while the visual design remains hostile to the eye. The money is real; it is not all on the screen; the screen inherits a style chosen decades earlier for reasons partly technical, partly comedic, partly accidental, now protected by brand value.

[*Futurama*](https://en.wikipedia.org/wiki/Futurama) is the internal control. Same Matt Groening lineage; many overlapping sensibilities; same taste for overbites, bulbous eyes, and simplified heads. But *Futurama* is much less ugly. Why? Machines. Spacecraft. Robots. Neon cities. Planet Express as a readable icon. The show’s world gives the designers geometry to enjoy. Bender is a tube with attitude; the ship is a clean green insect; New New York can hold perspective and signage; alien species permit graphic invention. The human faces still have Groening mouth-disease, but the total image is more varied, more designed.

This suggests the issue is not simply “Groening can’t draw pretty”. Rather: ugly human caricature, when surrounded by ordinary sitcom rooms, becomes oppressive; the same caricature inside a graphic science-fiction toybox is diluted by design pleasures. The control matters because it isolates taste from budget and era. Give the lumpy American TV head enough robots and signage and it can breathe.

## Backgrounds, banks, and the false necessity of ugliness

The defense from budget usually arrives too quickly. “TV animation has no money; of course it looks bad.” But bad in what dimension?

A show can save money through:
- held poses;
- repeated mouth shapes;
- limited camera movement;
- banked transformations/attacks;
- cycles for walking, running, dancing;
- offscreen action with sound;
- symbolic backgrounds;
- stylized smoke/dust/effects;
- cutting on reactions rather than actions.

None of these require ugly character design. They require planning.

Japanese TV anime built a large visual culture around this planning, sometimes gracefully, sometimes cynically. [*Neon Genesis Evangelion*](https://en.wikipedia.org/wiki/Neon_Genesis_Evangelion) is famous for production stress and still images, but its still images are often composed: elevator silences, power lines, industrial geometry, hands, plugs, shadows, typography. [*Sailor Moon*](https://en.wikipedia.org/wiki/Sailor_Moon_(TV_series)) reused transformation sequences because they were expensive banks worth reusing; the bank itself is a jewel box. [*One Piece*](https://en.wikipedia.org/wiki/One_Piece_(1999_TV_series)) has had long stretches of appalling pacing, but the designs are not default sitcom blobs. They are grotesque, yes, but inventively grotesque: impossible legs, teeth, noses, hats, silhouettes, fruit powers, ships, islands. Even at its worst, it usually wants to draw a freak, not a tax accountant after mild radiation exposure.

American children’s TV has also disproved the necessity. *Samurai Jack* used flatness as composition: black against red, tiny figure against giant rectangle, action as poster design. *Powerpuff Girls* is almost diagrammatic, but the diagrams are good. *Batman: The Animated Series* used dark paper and Art Deco massing to make limited animation feel expensive. *Avatar* outsourced much animation to Korean studios, used martial-arts reference and strong backgrounds, and often looks far better than many adult comedies with louder cultural prestige. *Over the Garden Wall* is modest and atmospheric, not lavish; its restraint is exactly why it works.

So the question is not “why does TV animation economize?” Everyone economizes. The question is why one tradition so often economized by making the baseline image unpleasant, then treated unpleasantness as maturity.



The historical excuse is usually underground comix. Fair enough: [Robert Crumb](https://en.wikipedia.org/wiki/Robert_Crumb) and company supplied a real anti-Disney tradition, sweaty, sexual, over-inked, hostile to good taste; adult animation needed some of that. But the normal decay of an avant-garde is surface-copying. Crumb's draftsmanship + appetite becomes lumpy noses + bad apartments; anti-appeal becomes a house style for scripts about roommates. The modernist-architecture analogy is exact enough to be annoying: early modernism had arguments about hygiene, ornament, mass housing, machines, light; later contractors kept flat roofs and concrete stains while dropping the theory and craft. Brutalism can be magnificent; a leaking municipal annex is not magnificent because concrete once had a manifesto.

## Proof by Stagnation

A style is alive if new constraints change it.

Digital tools changed cleanup, compositing, camera moves, color, effects, storage, revision, distribution, reference access, and international pipelines. Cheap tablets changed who could draw. Viewers moved from fuzzy CRTs to large HD and 4K displays. Streaming reduced some old broadcast constraints and created new ones. Anime absorbed digital compositing—sometimes badly, sometimes spectacularly. French studios built hybrid 2D/3D pipelines with aggressive art direction. Even amateur animation online acquired gradients, tweening, After Effects particles, Blender shots, fake camera shake, color grading.

Many American adult-comedy designs barely noticed. The lines got cleaner. The ugliness did not evolve into richer ugliness; it stabilized into house style. Heads, eyes, mouths, and torsos remained arranged for dialogue scenes in rectangular rooms. The tools improved the presentation of stagnation.

This is the “Proof by Stagnation”: if a style claims to be intentionally crude, but over decades of new tools it neither becomes more beautiful, nor more grotesque, nor more kinetic, nor more graphically radical, then perhaps its crudeness is not an aesthetic commitment but an equilibrium. The audience tolerates it; executives understand it; writers can pitch into it; overseas studios can deliver it; viewers use the characters as joke-containers; nobody is paid to ask whether the image is dead.

The contrast with [*One Piece*](https://en.wikipedia.org/wiki/One_Piece_(1999_TV_series)) is instructive because *One Piece* is aesthetically ridiculous. Eiichiro Oda’s designs include men shaped like onions, skeleton musicians, giraffe-swordsmen, clowns, mermaids, living ships, and bodies with the proportions of cutlery. It is “ugly” by many Western appeal standards. Yet when the anime receives better compositing, stronger directors, or more resources—as in much of the later Wano material—the style intensifies rather than apologizes. Colors flare; impact frames multiply; effects animation blooms; the freak designs become more operatic. Better technology does not require Luffy to become handsome. It makes the rubber idiot more rubber.

That is what a living ugly style looks like. Upgrade is metabolized.



There is an obvious selection problem here: bad American adult comedies are easy to remember because they share a visible house style, while equally bad foreign television animation is often unavailable, untranslated, or forgotten. The claim is not that America uniquely produces ugly television animation, but that one successful American adult-comedy equilibrium made static anti-appeal unusually reproducible.

A less bloggy version would be tedious: sample stills by country/decade/budget, blind titles, have artists rate silhouette/appeal/background composition, and see whether the American adult-comedy cluster survives controls for genre and year. My guess is yes, but with a large 1970s--1980s swamp everywhere.

## The answer, annoyingly, is taste

The issue is not money in the strong sense. Money matters, schedules matter, outsourcing matters, software matters; they are constraints on a taste already present. American TV comedy rewarded writing and voice over image; *The Simpsons* proved crude designs could become beloved; limited animation's legitimate economies were confused with bad static design; underground/alternative ugliness was cargo-culted without its draftsmanship; adultness became anti-appeal; executives learned that viewers would tolerate ugly if the jokes worked; digital tools then made it cheap to polish the bad default.

Low budgets force choices. Ugly is one choice.

The optimistic version is that defaults are shifting. Children who grew up with anime, games, Sakuga clips, French hybrids, web animation, Blender experiments, [*Spider-Man: Into the Spider-Verse*](https://en.wikipedia.org/wiki/Spider-Man:_Into_the_Spider-Verse), *Arcane*, and high-resolution screens have less reason to accept the old sitcom blob as the natural face of adult animation. They may still choose ugliness, but they will have to choose it harder. The bad imitations should die not by argument but by looking obsolete.

Useful future test: when an ugly adult cartoon gets a second season and more money, watch the defects. If they sharpen into weapons, good. If they become cleaner defects, it was probably another leaking annex.

## External Links

- [Animation](https://en.wikipedia.org/wiki/Animation)
- [Limited animation](https://en.wikipedia.org/wiki/Limited_animation)
- [Underground comix](https://en.wikipedia.org/wiki/Underground_comix)
- [Modernist architecture](https://en.wikipedia.org/wiki/Modern_architecture)

eval_iclmargin_fb100_tg_gpt55_xhigh_comp100k_20260702_novelty-net · final judges 0/1 · 10,885 chars · 1,563 words · /home/ubuntu/code/lesswronger/data/agent_runs/eval_iclmargin_fb100_tg_gpt55_xhigh_comp100k_20260702_novelty-net/submission.md

# Novelty Nets --- Classifier Anti-Guidance

2024-02-23

<div class="dropcap-kanzlei">
The most common complaint about generative models, after quality per se, is *sameness*.
</div>
Each sample is flawless—or near enough—but collectively they are *alike*.
You generate 10 images for a prompt and get, essentially, 3 images with minor variations in lighting or hand position.
Better prompting helps, of course, and this is one of the great attractions of prompt engineering as a skill; but the sameness recurs fractally.
You can specify "oil painting" vs "watercolor" vs "ink wash", and get 3 nice clusters, but *within* each cluster you are back where you started.
Words are insufficient because the user does not know, and *cannot* verbalize, the axes of variation they want explored.
(What would you even say? "Make it different, but in ways I haven't thought of yet"?)
The real goal of most generative sessions is: produce many *maximally-different* samples, then home in on the good ones.
The model can recognize a finished sample as novel or not-novel much better than it can *produce* novelty mid-generation—novelty is relational, after a fact—and so it defaults conservative, producing the mode.

## Brute-Force Sampling

The naive fix is: just sample more. But brute-force sampling has sharply diminishing returns.
Standard sampling concentrates around the mode; 100 samples give you little more *per-dimension* diversity than 10, while review cost is linear.
(Anybody who has sat through pages of near-identical Midjourney grids knows this viscerally.)
The fix belongs in *sampling*, not in "try harder".

## Novelty Search

There is substantial prior art on novelty search in evolutionary computation and reinforcement learning: you define a behavior characterization, maintain an *archive* of previously-visited behaviors, reward difference from the archive, then rank candidates by the true reward.
[Lehman & Stanley 2011](https://doi.org/10.1162/EVCO_a_00025) is the classic reference; the idea has been rediscovered repeatedly in RL exploration ([Burda et al 2018](https://arxiv.org/abs/1810.12894), [Ecoffet et al 2019/Go-Explore](https://arxiv.org/abs/1901.10995), etc).

The most natural translation to generative models is an *outer loop*: generate a sample, compare it to an archive, keep or discard.
Variants:

- High temperature, generate *n*, keep *k* most dissimilar (k-medoids in some embedding space like CLIP or a VAE latent).
- Store embeddings, reject any new sample too close to a nearest neighbor by kNN threshold. You can even use [Bloom filters](https://en.wikipedia.org/wiki/Bloom_filter) or [cuckoo filters](https://en.wikipedia.org/wiki/Cuckoo_filter) with binarized distance-encodings for cheap approximate membership tests.

The trouble: *generation itself* is still not seeking novelty except by adding more noise (higher temperature, weaker diffusion guidance, high StyleGAN ψ).
You are optimizing the *wrong* part of the pipeline.
Generation is ~99% of the compute; search/lookup/filtering is ~1%.
Making the cheap loop cleverer doesn't help much when the expensive loop is still producing the same thing.

## State

Novelty must be built *into* generation, and it must be *stateful*.

Prompt jitter or embedding perturbation (adding noise to the text embedding, randomly masking tokens, etc) damages semantics.
You wanted "a cat in a sunbeam" and you get "a cat in a moonbeam"—that's not diversity, that's damage.

Novelty is *temporal* and *contextual*: the first instance of a composition is novel; an identical second instance is not.
A static, stateless model has no way to know whether it has generated something before.
It cannot, by construction, avoid repetition.

One could imagine maintaining a vector database as external state: at each diffusion step, embed the current activations, query the DB, modify the denoising direction away from the nearest neighbor.
But this is wildly impractical.
Each query stalls the GPU; VRAM is scarce enough for the base model; latency is devastating for iterative processes with 20–50 steps.
And you don't *need* full kNN with distances and neighbor lists and high-dimensional embeddings.
You just need a cheap, rough *gradient*: "go away from stuff I've already made".

## The Novelty Net

Proposal: train a small neural network to approximate $P(\text{previously-generated}\ |\ \text{in-progress embedding/activations})$.
Insert it as an adapter layer in the generator (eg. cross-attention or a residual MLP on the U-Net bottleneck features).
Use that probability as a weak auxiliary loss—*anti-guidance*, pushing the generation away from high $P$.

After each completed generation, train the novelty net online: label the new sample's activations/embedding as a positive example ("this has now been generated; remember it").
The generator itself stays frozen; no giant external database needed; you just carry around a small adapter that has memorized your generation history and provides a gradient *away* from it.

This is *classifier anti-guidance*: [classifier guidance](https://arxiv.org/abs/2105.05233) steers toward a class; this steers *away* from the class "things I've seen before".

### Details

Why a small MLP?
The inputs are fixed-size unordered embeddings (CLIP, VAE latent, bottleneck activations)—no sequence structure to exploit.
MLPs are famously good at *memorization*^[This is usually a bug; here it's a feature.], which is exactly what we want: the net must memorize every past sample.
2–3 layers, a few million parameters, regressing log-odds of "seen before", trained online after each sample.

**Collapse.** If you only ever show positives, the net learns $P = 1$ for everything (trivially correct: "everything looks like something I've generated").
Fix: generate negative examples by *jittering* the positive embeddings—random perturbations, interpolations, augmentations.
The net must learn to distinguish the actual samples from nearby-but-not-identical points.
This is loosely analogous to [Barlow Twins](https://arxiv.org/abs/2103.01988) / contrastive self-supervised learning: you need negatives (or at least a decorrelation objective) to prevent collapse.

**Relatives:**

- *Classifier guidance / negative prompts*: the mechanism is the same (a classifier's gradient modifying the generative process), but standard negative prompts are static and semantic ("no blurry"), not temporal.
- *GAN discriminator memorization*: GAN discriminators infamously memorize training examples; a novelty net is a discriminator that *intentionally* memorizes generated examples and uses that memorization constructively.
- *State-based RL exploration*: neural episodic control ([Pritzel et al 2017](https://arxiv.org/abs/1703.01988)), Go-Explore ([Ecoffet et al 2019](https://arxiv.org/abs/1901.10995)), RND ([Burda et al 2018](https://arxiv.org/abs/1810.12894))—all maintain some memory of visited states and derive exploration bonuses. The novelty net is the same idea applied to generative sampling rather than policy optimization.
- *Surrogate / synthetic gradients*: the novelty net provides a cheap approximate gradient in lieu of an expensive true computation (the full kNN lookup).
- *Fast weights*: the novelty net's online training is a form of [fast weights](https://www.cs.toronto.edu/~hinton/absps/fastweights.pdf)—a small rapidly-updated memory complementing a large slow model.

## The User Experience

From the user's perspective: each image *automatically differs* from the last.
No fiddling with seeds, no "make it different" re-prompting, no manual randomization of style tokens.
You keep hitting "generate" and watch the model *explore*.
The first image is the mode; the second avoids the mode; the tenth is something you would never have thought to ask for but is still "a cat in a sunbeam".

This is what people actually want when they complain about sameness. Users will of course immediately ask for a "novel but not weird" slider, which is approximately a request for exploration without leaving the hotel.

## Distillation: Reference Inputs Beyond Text

A natural extension: rather than running the novelty net live, *distill* its effect into the model via indexed training.

Given a prompt, generate $n$ candidate images, rank them by a weighted combination of quality + dissimilarity-from-each-other (Pareto ranking).
Assign each a unique index $k \in \{1, \ldots, n\}$ ordered by this ranking.
Fine-tune the model on (prompt, index $k$) → image $k$.

At runtime, sweeping $k$ from 1 to $n$ sweeps the Pareto frontier from "highest quality, most typical" to "lower quality, most unusual".
The index is a *reference input beyond text*—a scalar knob for novelty, baked into the weights.

(This is offline and amortized and loses the online adaptivity of the live novelty net, but it's simpler to deploy and requires no per-user state.)

## Collective Novelty

The most interesting extension: train the novelty net not just on *your* past generations but on the model's *training data* and *all users' generations*.

This addresses several problems at once:

1. **Training echoes / copyright.** If the novelty net has memorized the training set, anti-guidance steers away from reproducing training images. This is a *generative* solution to memorization—not filtering outputs post hoc, but making the generation process itself avoid the training distribution's modes. (Not a complete solution to copyright, obviously, but a useful technical component.)

2. **Creative uselessness.** The characteristic complaint about, say, Midjourney is not that any one image is bad but that they all look like *Midjourney images*. The "Midjourney look"—the HDR sheen, the heroic lighting, the same 5 color palettes—is a collective phenomenon: each user's lazy default-setting generation contributes to an aggregate aesthetic that audiences learn to recognize and become allergic to. This is a negative externality / collective action problem: each user rationally takes the easy default; the aggregate result is that "AI art" becomes a recognizable (and increasingly dismissed) genre.

A service-level novelty net internalizes the externality. It makes *every* generation automatically a little different from *every other* generation, past and present, across all users. The distribution broadens, not by censorship or style-banning, but because the sampler is continually charged a small tax for walking in old ruts.

This is the argument which makes me take novelty nets seriously: per-user novelty is a convenience feature; shared novelty is closer to sanitation. If the default sampler is allowed to lay down the same aesthetic cow-path 10 billion times, the resulting groove belongs to everyone, including the users who worked hard not to walk in it. Better, perhaps, to make the path itself a little slippery.

<p><a href="/index">Return to blog index</a></p>

eval_iclmargin_fb100_tg_gpt55_xhigh_comp100k_20260702_lean-scaling · final judges 0/1 · 16,620 chars · 2,404 words · /home/ubuntu/code/lesswronger/data/agent_runs/eval_iclmargin_fb100_tg_gpt55_xhigh_comp100k_20260702_lean-scaling/submission.md

> Status: research proposal; no results. (The experiment is cheap enough that a no-results essay is already a little embarrassing, but the hard part is not the forward passes, it is deciding what source code is *allowed to mean*.)
>
> Source code is text, and LLM loss on text is a bad but real measure of predictability. The old software-engineering observation that code is "natural" (Hindle et al. 2012, and the later Allamanis/"big code" line) now matters for a new reason: the model which predicts the next token is also going to write the next program.
>
> I propose measuring programming languages by *loss curves*: how fast bits/byte fall as a frozen code model reads more of a repository or ecosystem. Popular languages should have excellent constants, because GitHub accidentally paid for their pretraining. The interesting quantity is the exponent. If Lean has the worst constant and the best exponent, the usual "LLMs only know Python" lock-in argument is a short-run argument, not a law of nature.
>
> This would not prove that Lean rewrites are cheap, or that formal verification finally works, or that cybersecurity is saved. It would show something narrower and more useful: formal languages may buy a kind of semantic compression that LLMs can exploit at scale.

# The weak claim is enough

Assume only the weak version of the AI-coding thesis: marginal new code is increasingly LLM-shaped. Not "all software will be generated by 2030" (probably false in annoying definitional ways---does Copilot autocompleting the `if` count? the migration script nobody read?); merely that LLMs increasingly write the boring glue, tests, API wrappers, Terraform, auth handlers, data munging, and one-off internal dashboards that accrete into real infrastructure.

The feedback loop is then uncomfortable:

```text
public corpus → code model competence → generated code share → future public corpus
```

The model is good at whatever the public Internet overproduced; it then overproduces more. Not exactly evolution, but it has the same unpleasant flavor: local fitness, path-dependence, no guarantee the equilibrium is what one would have chosen deliberately. If Python/JavaScript/Ruby/PHP had been designed by an adversary to make large-scale semantic reasoning maximally annoying while retaining good first-10-minutes ergonomics, would the world look very different?

Standard reply: "LLMs will just fix the bugs too." Perhaps. But LLM-written insecure code and LLM-powered bug-hunting are not symmetric in the comforting way. The defender needs the auth boundary, parser, serializer, dependency config, OAuth callback, file-upload path, rate limiter, crypto call, tenant isolation, and deployment script to all be right *simultaneously*; the attacker needs one omission. `pytest` passes, the demo works, and the bug lives in the part of the threat model that was not in the prompt.^[The failure mode is not nonsense---nonsense is easy. The failure mode is plausible code in domains where the reviewer lacks a threat model: broken CORS, confused deputy APIs, insecure deserialization, secret leakage, timing side-channels, homebrew crypto, overbroad exception handlers, "temporary" admin endpoints. The code often looks *more* reviewable than it is, which is worse than looking cursed.]

Formal verification is the obvious adult answer: make the machine check invariants. [CompCert](https://compcert.org/), [seL4](https://sel4.systems/), verified crypto---prove memory-safety, parser totality, protocol conformance, constant-time behavior, whatever property actually matters. This answer has also been obvious for decades, and yet most deployed code is not formally verified, because the cost curve was too high and the tooling/ecosystem curve too low. So the question is not "is formal verification good?" It is: *how far behind is it, and does the gap close under LLM scaling?*

# The lock-in objection is right but too short

The easy pessimism is corpus lock-in. LLMs are better at popular languages because there is vastly more Python/JS/Java/C/C++ in training. Luo et al. 2025, in the form relevant here, argues that code is especially data-hungry; one needs many tokens of a language before generation becomes reliable. Then [Lean](https://lean-lang.org/), Coq, Agda, Idris, Haskell are permanently doomed boutique languages: too little code → bad models → little new code → still too little code. QED, maybe with a sigh.

This is a constant-term argument pretending to be an asymptotic argument.

Training corpus is a prior. It sets the intercept, not necessarily the slope or ceiling. If a model trained mostly on Spanish has high loss on Portuguese, we learn something about the corpus and tokenizer but not much about Portuguese as a language. Raw Lean loss in GPT-4/Claude/DeepSeek tells us mostly that Lean is rare and weird, not whether Lean codebases become easier or harder to predict as context grows.

There are also exchange rates. Yang et al. 2025 is relevant because cross-language transfer in code is not zero: Python knowledge transfers somewhat to TypeScript; ML-family functional patterns transfer somewhat to Lean; theorem-proving patterns transfer across Lean/Coq/Isabelle in strange uneven ways (the LeanDojo/miniF2F ecosystem is small, but at least real enough to give a foothold). One wants a table:

```text
1 Lean token ≈ ? Haskell tokens ≈ ? Rust tokens ≈ ? Python tokens
```

for lowering held-out Lean loss, and the reciprocal. (Almost certainly not symmetric. A theorem-proving corpus may teach useful abstractions to a Rust model; a million CRUD controllers may teach a Lean model less than the byte count suggests.)

# Small scripts lie

Benchmarks like [HumanEval](https://github.com/openai/human-eval), MBPP, SWE-bench, RepoBench measure small-problem or bounded-repo competence: "write a function that..." or "fix this issue in a repo." Useful, but misleading for the formal-language question because they test exactly the regime where corpus size dominates. A model that has seen 10 billion tokens of Python and 50 million tokens of Lean will obviously write better 20-line Python scripts. This tells you about the constant, not the exponent.

What matters for lock-in is repository scale. Does loss keep falling as context grows? How fast? A formal language constrains the space of valid continuations more aggressively than a dynamic one---every term must type-check, every proof must close---so in principle the model has less to predict once it "understands" the local type context. The type system is doing free compression. But "in principle" is doing a lot of work; real Lean code has implicit arguments, tactic blocks, universe polymorphism, attribute macros, and [mathlib](https://github.com/leanprover-community/mathlib4) conventions that might be *harder* to predict token-by-token even though they carry less semantic entropy.

This is an empirical question, and the experiment is cheap.

# Loss curves: constants and exponents

Take a frozen code-capable LLM. Feed it source files from a repository, accumulating context. Record bits-per-byte as a function of tokens seen. Do this across many repositories in each language. Fit the boring scaling-law shape:

$$L(n) = A \cdot n^{-\alpha} + L_\infty$$

where $n$ is tokens of context, $A$ is the constant (intercept advantage from corpus frequency), $\alpha$ the exponent (how fast the language compresses under growing context), and $L_\infty$ the irreducible entropy floor. Thesis operationalized: Lean (and perhaps Rust/Haskell/Idris) should have large $A$ (bad constant---rare language, unfamiliar syntax) but large $\alpha$ (good exponent---formal structure lets the model learn faster per token of context). Python/JS should have small $A$ (good constant---the model has seen everything) but small $\alpha$ (additional context helps less because the language doesn't constrain continuations as tightly).

If you plot $L(n)$ for multiple languages, the curves may cross. The crossing point is the context length/repo scale at which formal languages become easier to predict than informal ones despite starting harder. If this crossing point exists and is reachable within current or near-future context windows, the lock-in argument has an expiration date.

Obvious objection: this measures prediction, not generation quality or correctness. True. It is a bad proxy. But prediction and generation are not independent in autoregressive models; a model that achieves low loss on Lean code has learned something about Lean's structure useful for generation. The proxy becomes interesting only as a *trajectory*: boilerplate Java, JSON, generated protobuf bindings, copied LeetCode, and minified JS are all sanity checks where low/high loss means the wrong thing. The exponent after controls is the metric, not the scalar.

# Cheap version

Concretely:

1. languages: Lean 4, Coq/Agda/Idris if available, Haskell/OCaml, Rust, Go/Java, TypeScript, Python;
2. repos in size strata, with vendored/generated/minified/notebook files stripped or labeled;
3. multiple orderings: import-topological, random, reverse-dependency, chronological commit order;
4. frozen models of several sizes, byte-normalized loss, and a dumb gzip/zstd/n-gram baseline (if `zstd` sees the same exponent, do not call it semantic);
5. fit $A,\alpha,L_\infty$ by language/repo/model, and report how much order/tokenizer/preprocessing moves them.

Lean needs special decomposition: definitions vs theorem statements vs proof terms vs tactic scripts vs imports/notation. [mathlib](https://github.com/leanprover-community/mathlib4) is not "Lean" any more than CPython is "Python"; it is a giant, reviewed, stylized cathedral, and most of the world is strip mall.

The perturbational tests are more valuable than the raw curves:

- **Bug/anomaly injection:** wrong sign/unit, missing bounds check, silent dtype cast, broad exception handler, bogus lemma, invalid `simp` rewrite. A model/language pair with real semantic constraint should be *more* surprised by the bug.
- **Hidden context:** hide implementations and leave signatures/imports. Can the model reconstruct code that compiles/passes tests? In Lean, can it fill a module from type/theorem signatures?
- **Ablations:** remove type signatures, comments, tests, theorem statements, Python type hints, Rust lifetimes. Which components actually lower future loss?
- **Coreset/minimal context:** find the smallest context subset preserving ~full-prefix loss. Shorter = better modularization; a well-designed Lean module should need signatures, not half the repo.
- **Refactor scoring:** do human-approved cleanups reduce loss and improve repair locality? If not, the metric is measuring style rather than maintainability.

This is perhaps a 2--6 week MATS/grad-student project. The expensive version is equal-corpus finetunes or from-scratch code models; do that only if the cheap curves survive the stupid explanations.

# Forecasts (so this can be wrong)

| Claim | Prior | What would move me |
|---|---:|---|
| Raw byte loss: Python/JS beat Lean badly | >95% | only a Lean-specialized model changes this |
| Dynamic languages have worse context-benefit exponents than Rust/Haskell after controls | ~55% | survives orderings, dedup, byte loss |
| Haskell/OCaml cross dynamic languages around 100k--1M lines in context benefit | ~35% | matched repos/equal-corpus finetunes |
| Lean has best exponent | 25--35% | must hold outside mathlib and outside import boilerplate |
| Perplexity/anomaly surprisal predicts real defect density | 15--25% | needs bug DBs, churn/author controls |
| First study is null or uninterpretable | ~50% | corpus/domain/tokenizer confounds are ugly |

Qualitatively: Python/JS low constant, bad exponent; Go/Java/Rust medium; Haskell/OCaml worse constant, better exponent; Lean worst constant, maybe best exponent, perhaps no absolute crossover for current models. But an exponent advantage is already actionable: it says the design is promising but starved. Then one can regress constants against corpus size/training exposure and ask how many high-quality Lean tokens lower the crossover from "never" to "1M-line repo" to "100k-line security library". If Yang-style exchange rates say Haskell/Rust tokens buy Lean competence cheaply, buy those too; if not, buy Lean directly. Agentic translation with rejection sampling turns "hard" into a data price.

# Confounders, ie. the project

The language is not the corpus. Lean programmers are unusual people in unusual domains with unusual review norms. JavaScript is written by everyone, including the cursed and hurried. If Lean looks good, it may be measuring mathematicians; if JS looks bad, it may be measuring adtech; if Rust looks good, it may be measuring people who enjoy compiler pain enough to finish the project.

Controls: downsample popular languages to Lean-sized corpora; compare expert/beginner strata; match domains (parser vs parser, crypto vs crypto); compare same-spec implementations, eg. zlib C/Rust/Lean or protocol libraries; eventually commission controlled implementations from a spec. Synthetic paired implementations are useful only once humans equalize quality, otherwise you measure translation slop.

In-context Lean ignorance is another confounder. A frozen model may not know Lean well enough to exploit its structure. Then the curve is flat because the reader is illiterate. Check with Lean-specialized finetunes, equal-token finetunes, or (expensive) from-scratch controlled mixtures.

The ecosystem null may be the truth. Maybe language-level invariants matter less than package managers, linters, CI, documentation, examples, conventions, IDE feedback, and boring people copying boring patterns. A disciplined Python codebase with type hints/property tests/mypy/docs may beat sloppy Lean. Good! Then the intervention is not a cathedral in dependent type theory but better boring tooling and corpora.

The most damning result would be JavaScript beating Lean on exponent after controls. Then either the metric is wrong, or formal-methods folk have missed a practical variable, or npm was right in some horrible statistical sense. I do not expect this. I would also not ignore it.

# What a positive result licenses

A good Lean exponent does not justify rewriting Linux tomorrow. It licenses a ladder:

1. weak: Lean code is more context-compressible at scale under code LMs;
2. moderate: Lean may be unusually amenable to LLM-assisted maintenance/repair once the constant is paid down;
3. strong: verified Lean rewrites of selected high-value libraries may be cost-effective;
4. heroic: large software should migrate toward Lean-like languages for security/correctness.

Only (1) follows from loss curves. (2) needs task benchmarks; (3) needs economics, performance, FFI/extraction trust, proof coverage, attack model; (4) needs a civilization with better taste than ours.

Still, (1) matters. Today formal-methods arguments are anecdotes ("seL4 worked"), disasters ("this parser bug cost $N"), or philosophy ("proof > tests"). True-ish, bad at forecasting. A scaling exponent is the right kind of object: it asks whether pain amortizes.

This generalizes to math/formalization. A good theorem/library definition is semantic compression: after it is stated/proved, later proofs should become shorter, more local, more predictable, easier to repair. Bad formalization is **mathslop**: proof artifacts typecheck but do not amortize, each proof a snowflake made of boilerplate. A tactic or LLM proof assistant should not merely solve miniF2F-style problems; it should improve library repair-locality and future predictability.

Most likely, the first result is null: tokenizer/corpus/domain/author effects dominate, proof scripts encode genuine search, and loss measures style more than semantics. That is fine. We learn that the correct intervention is docs, conventions, proof traces, IDE feedback, benchmark weighting, or post-hoc verification of generated code.

But if, after removing stupid explanations, Lean-like languages really do show a better exponent, then the chicken-and-egg problem has a price tag: pay for enough Lean code to lower the constant until the exponent matters. Start with [zlib](https://zlib.net/)-sized stones---CVE-2018-25032-sized stones---not cathedrals.

**See Also**:

- [Program Reliability](/program-reliability)
- [`HACL*`/EverCrypt](https://project-everest.github.io/)
- [The Hutter Prize](/hutter-prize)
- [Predictability and Surprise in Large Generative Models](/predictability)
- [Why do writers still underestimate LLMs?](/llm-writing)

eval_iclmargin_fb100_tg_gpt55_xhigh_comp100k_20260702_rocky-road · final judges 0/1 · 27,487 chars · 4,314 words · /home/ubuntu/code/lesswronger/data/agent_runs/eval_iclmargin_fb100_tg_gpt55_xhigh_comp100k_20260702_rocky-road/submission.md

# How Long Would “Weird Al” Survive In An Ice Cream Freezer?

The lyric in [“I Love Rocky Road”](https://en.wikipedia.org/wiki/I_Love_Rocky_Road) invites the wrong calculation. Weird Al asks to be locked in the freezer where they keep the rocky road; rocky road, in turn, is not merely flavored snow but chocolate ice cream plus marshmallow and nuts, ie. a dairy-fat/sugar/water/mineral brick with some almonds accidentally thrown in as micronutrient insurance. This is a bad place to be, but not *obviously* worse than many other places in the [desert-island-food](https://en.wikipedia.org/wiki/Desert_island) genre.

Wrong calculation:

> rocky road = calories + water + dairy vitamins  
> locked in freezer = inconvenient pantry  
> death = scurvy? diabetes? boredom?

The actual calculation is mostly watts.

Assumptions, because otherwise this becomes a cartoon physics problem:

- adult male, “normal”, ~79kg
- nude (not implied by the song; imposed here to remove clothing)
- ordinary commercial walk-in ice-cream freezer
- air temperature −18°C to −29°C  
  (0°F is −18°C; many ice-cream storage freezers are colder, often around −23°C to −29°C)
- shelves, tubs, cardboard/plastic packaging, metal racks/floor
- not buried in ice cream, not submerged, not encased
- enough air to breathe
- no rescue, no clothing, no heat source
- can eat rocky road if he wants
- no special cold adaptation, no drunkenness, no disease

Short answer: he dies of hypothermia in hours. Central estimate 2–3h; plausible range maybe 1.5–4h. Eating the ice cream makes the short-term problem worse, not better. The limiting resource is not calories, but heat production rate.

## Polling the machines

I asked a small set of current LLMs the question in roughly the above form. I wanted the annoying disambiguations explicit, because otherwise models (and humans) answer different cartoons:

```text
A normal adult human, 79kg, is locked nude in an ordinary commercial
walk-in ice-cream freezer. Assume -18C to -29C. The freezer is stocked
with shelves/tubs/cartons of rocky road ice cream, but the person is not
buried, submerged, encased, or immobilized in ice cream; there is air to
breathe and room to stand/curl up/eat. No clothing, tools, fire, rescue,
or outside heat source. Normal physiology.

What kills them first, and on what timeline? Does eating the rocky road
help or hurt? Rank hypothermia, dehydration, scurvy, diabetes, starvation,
asphyxiation, and any other plausible cause.
```

(I used variants of this, not a holy prompt. The point was to force the model to notice the freezer as a thermal environment rather than a storage closet.) I did not ask for jokes; a few made them anyway.

The models mostly converged:

| Model class | Main answer | Timeline | Notes |
|---|---:|---:|---|
| GPT-5.4 Pro / o-series-ish | hypothermia | ~2–3h, with uncertainty | mentioned heat flux, shivering limits, cold ingestion penalty, boundary layers |
| Claude Opus/Sonnet-ish | hypothermia | hours, not days | good staged physiology; more cautious about exact clock |
| Gemini/DeepSeek-ish frontier | hypothermia | 1–4h | usually got the “ice cream calories don’t save you” point |
| weaker/smaller models | hypothermia | “a few hours” | mostly restated; less calculation; sometimes forgot ingestion is a cold sink |
| bad/naive answers | starvation/dehydration/diabetes/scurvy | days–months | intuitive but thermally irrelevant |

This is not a benchmark; it is closer to asking several interns to check one another's arithmetic. No fixed geometry, posture, freezer airflow, floor material, package availability, fat mass, body surface area, or shivering curve; so any numerical answer is a fuzzy bundle of assumptions. Still, there was a useful split: the stronger models did something like a heat budget; weaker models said “hypothermia” without saying why; bad answers treated the freezer as a dessert cave.

The dessert-cave model is seductive. Rocky road is not empty calories; it has fat, sugar, protein, water, calcium, some vitamins. It is also at −20°C, and your core is at 37°C. The sign of the first derivative is therefore not the sign suggested by the nutrition label.

## The freezer is not a larder

An adult human at rest produces roughly 76–100W of heat.

One [MET](https://en.wikipedia.org/wiki/Metabolic_equivalent_of_task) is about 3.5ml O₂/kg/min. For 79kg:

\[
3.5 \cdot 79 = 276.5 \text{ ml O₂/min}
\]

At ~5kcal/L O₂:

\[
0.2765 \cdot 5 = 1.38 \text{ kcal/min}
\]

\[
1.38 \text{ kcal/min} \cdot 69.8 = 96W
\]

BMR estimates for a 79kg adult male often land a bit lower, ~76–90W depending on age/height. So: ~80–100W resting.

Maximum shivering can raise heat production a lot. A cold, terrified, well-fed adult might produce ~400–500W for a while (not a neat plateau; muscles fatigue, glycogen matters, coordination declines, and hypothermia itself destroys the ability to shiver). The usual practical number: several times basal, perhaps 4–6×, not 20×.

Now compare heat loss.

A 79kg adult has body surface area ~1.9–2.0m² by Mosteller/Du Bois style estimates. Nude, in still air, heat loss is convection + radiation + respiration + contact.

A crude dry-air estimate:

\[
\dot Q \approx A(h_c+h_r)(T_{skin}-T_{air})
\]

Radiation coefficient near body temperature is ~4.5–5.5W/m²/K. Natural convection in still air might be ~3–8W/m²/K. Airflow from fans or door leaks raises convective loss; 10–25W/m²/K is no longer strange. Skin temperature is not fixed; vasoconstriction cools the skin and reduces loss. Use 28–33°C early, lower later. Air is −18 to −29°C.

Still air, not much draft:

\[
A \approx 1.8m^2
\]

\[
h_c+h_r \approx 9–14W/m^2/K
\]

\[
\Delta T \approx 45–60K
\]

\[
\dot Q \approx 730–1510W
\]

Curling up reduces exposed area. Vasoconstriction reduces skin temperature. A boundary layer of motionless cold air around the body helps. Standing near a fan, lying on metal, or touching frozen tubs hurts. A realistic “nude person in a walk-in freezer, trying not to touch metal” number might be hundreds to early thousands of watts of heat loss. Not 50W. Not 100W. The body is radiating and convecting into a large cold sink.

Respiration adds some loss: cold dry air is warmed/humidified in the airway and exhaled. At rest this may be tens of watts; during shivering/hyperventilation more. Not the dominant term, but not zero.

Contact loss is ugly. Bare feet on a cold floor, knees on metal, hands gripping shelves: local cooling/frostbite and additional conduction. A human can reduce contact area, but cannot levitate. Cardboard helps if available. Metal shelves are bad. Ice cream tubs vary; plastic/cardboard tubs are much better than metal, but still cold masses.

So suppose:

- loss: 500–1200W (mild to severe still-air nude exposure)
- production: 80–100W resting, maybe 400–500W shivering early
- deficit: commonly hundreds of watts

The body has stored energy, but stored chemical energy is not the bottleneck. A fat person can have 100,000kcal on board and still freeze quickly, because fat is not a 1kW heater on demand. The body is power-limited.

## Heat capacity: the unpleasant arithmetic

Specific heat of the human body: about 3kJ/kg/°C. (Water is 4.18; humans are water plus fat/protein/mineral.)

For 79kg:

\[
C_{body} \approx 79 \cdot 3 = 237 \text{ kJ/°C}
\]

Call it 235kJ/°C.

If net heat loss is 500W:

\[
500J/s \cdot 3600s = 1.8MJ/h
\]

\[
1.8MJ/h \div 235kJ/°C \approx 7.7°C/h
\]

That is too simple and too fast if read literally. It treats the body as a well-stirred vat. Real humans vasoconstrict; skin and limbs cool first; the core is defended; shivering rises; posture changes; exposure geometry changes. But it shows the scale. A few hundred watts of persistent deficit is enough.

Even at 200W net deficit:

\[
200J/s \cdot 3600s = 720kJ/h
\]

\[
720 \div 235 \approx 3.1°C/h
\]

At 150W:

\[
540kJ/h \div 235 \approx 2.3°C/h
\]

To go from 37°C to 32°C is 5°C. To 30°C is 7°C. To 28°C is 9°C. Severe hypothermia and lethal arrhythmias live in that zone, with individual variation and bad luck.

A 2–3h central estimate is not heroic. It corresponds to effective core cooling of ~2–4°C/h after all the messy physiological defenses. Nude at −20°C to −29°C, that is plausible.

One can invert the calculation usefully. Suppose the victim gets 3h and fatal physiology begins around 28°C (a 9°C drop). That is 3°C/h of core cooling, or:

\[
3 \cdot 235 = 705\text{ kJ/h} \approx 196W.
\]

So the *effective* average deficit at the core need only be ~200W. It does not require a constant 1kW loss all the way through. The early exterior losses can be larger while the shell cools; later shivering fails; the average that reaches the defended core is still plenty. This is why quibbling about whether free convection is 400W or 900W does not rescue the naive answer. To survive days, one must reduce the net deficit to almost zero, not merely trim it by 20%.

## Stages, approximately

[Hypothermia](https://en.wikipedia.org/wiki/Hypothermia) stages are not as crisp as tables pretend, but the table is useful.

| Core temperature | Usual label | What happens |
|---:|---|---|
| 35–32°C | mild | shivering, clumsiness, impaired judgment, tachycardia |
| 32–28°C | moderate | violent then fading shivering, confusion, ataxia, dysarthria, apathy, bradycardia |
| <28°C | severe | coma, severe bradycardia, ventricular fibrillation/asystole risk |

Shivering fails somewhere around the low 30s, often quoted near 31°C. It does not stop like a switch, but the direction is wrong: the colder you get, the less able you are to make heat, while the environment continues not caring.

A plausible walk-in freezer timeline:

| Time | Event |
|---:|---|
| 0–10min | immediate pain, skin cooling, vasoconstriction, intense discomfort; feet/hands become uselessly cold fast |
| 10–30min | maximal shivering, rising oxygen use, panic or forced activity; exposed skin/ears/fingers at frostbite risk depending airflow/contact |
| 30–90min | mild hypothermia; judgment and dexterity worsen; eating becomes less attractive and more dangerous; floor/shelf contact injuries |
| 60–150min | moderate hypothermia in many cases; stumbling, apathy, confusion; shivering begins to fail |
| 90–240min | collapse/unconsciousness; severe hypothermia; bradycardia; ventricular fibrillation/asystole/respiratory failure |
| after collapse | heat loss worsens if posture/contact becomes bad; airway risks; death unless rewarmed |

The clock is not precise. A thick man curled on cardboard in still air among insulating boxes does better than a lean man standing in fanwash on a metal floor. But “days” is off by an order of magnitude.

Also, the endpoint “death” is already too late for the practical question. The useful window is the interval before fingers stop opening containers, before ataxia makes standing dangerous, before the plan changes from “build a nest” to “sit down for a minute”. [Hypothermia](https://en.wikipedia.org/wiki/Hypothermia) is unfair in that the cognitive/mechanical tools for escaping are damaged by the same process one is trying to escape. (The prompt gives no escape, but the same applies to any improvised insulation scheme.)

## But rocky road has calories

Yes. That is why the thought experiment is good.

Rocky road ice cream is energy-dense. Depending brand/recipe, roughly 200–240kcal per 100g is ordinary; 1kg can contain ~2000–2400kcal. It is mostly:

- fat
- sugar
- water/ice
- dairy solids
- almonds
- marshmallow/chocolate/cocoa bits depending recipe

If fully digested and oxidized:

\[
2000 \text{ kcal} \approx 8.4MJ
\]

A 79kg body’s heat capacity is ~235kJ/°C, so 8.4MJ is enough energy, in principle, to raise the whole body by:

\[
8400kJ \div 235kJ/°C \approx 36°C
\]

This is the trap. Chemical calories are not delivered as instant heat to the core. Eating frozen food first requires the body to warm and melt it. The heat is taken from you now; the calories arrive later, through digestion and metabolism, with rate limits, while the freezer continues extracting heat.

Thermal cost of eating 1kg of ice cream at −20°C to −29°C:

Components, approximate:

1. Warm frozen ice cream from −20°C to 0°C  
   specific heat maybe ~1.8–2.2kJ/kg/K  
   \[
   \approx 36–44kJ
   \]

2. Melt the frozen water fraction  
   Ice cream is not a block of pure ice; water fraction ~55–65%, not all frozen because sugar lowers freezing point. Latent heat can plausibly be ~120–180kJ/kg of ice cream.

3. Warm the melted/liquid mass from 0°C to 37°C  
   effective heat capacity perhaps ~3.5–4kJ/kg/K  
   \[
   \approx 130–150kJ
   \]

Total:

\[
\approx 290–370kJ
\]

At −29°C add maybe another 15–25kJ. So:

\[
\approx 70–95 \text{ kcal}
\]

per kg of ice cream, just to bring it from freezer temperature to body temperature. The prompt’s range, 70–95kcal/kg, is right enough.

Core-equivalent cooling:

\[
300–370kJ \div 235kJ/°C \approx 1.3–1.6°C
\]

Use 1.2–1.5°C if one is less pessimistic about ice fraction and mixing. Still: a kilogram of rocky road is a one-degree-plus body-temperature debt, delivered into the gut.

The calories are ~2000–2400kcal/kg. The cold cost is only ~3–5% of the chemical energy. This sounds favorable until one remembers time.

The cold cost is immediate. The sugar/fat/protein heat is delayed.

Even if digestion eventually helps, “eventually” is not friendly:

- gastric emptying slows in cold stress and especially hypothermia
- gut motility slows below ~33°C
- shivering and peripheral vasoconstriction alter blood flow priorities
- hypothermic confusion makes coordinated eating/swallowing worse
- nausea/vomiting is possible
- aspiration/choking risk rises as consciousness and airway reflexes fail
- insulin response is not a heater
- fat oxidation is slow compared with “I am losing 500W”

A kilogram of −20°C ice cream in the stomach is a local cold sink near the core. It can depress core temperature at exactly the wrong time. It is not like adding logs to a fire; it is like adding a frozen log to a small animal and promising it will burn later.

There is a narrow exception: if the person is thermally protected enough that heat loss has been reduced below metabolic production, then eating provides calories/water and the cold cost may be amortized. If the ice cream can be melted outside the body using waste heat in an already-warm shelter, or held in the mouth in tiny amounts after the gut is still functioning, the ledger eventually wins. But the baseline scenario is a nude adult in a freezer, not a mountaineer inside a sleeping bag with a stove; and a mouthful of -20°C fat-sugar paste is not a stove.

## “Eat continuously for heat?”

Suppose the freezer heat loss is 800W. The body is producing 450W by shivering. Net deficit 350W.

A kg of ice cream costs ~330kJ to warm:

\[
330kJ \div 350W \approx 943s \approx 16min
\]

So eating 1kg creates about 16 minutes of additional net heat debt under those assumptions, or ~1.4°C whole-body equivalent. The chemical energy in it is huge, but oxidation cannot instantly produce an extra 2kW on command. If it could, people would use ice cream as emergency rewarming. They do not.

More concretely, suppose absorption/metabolism of the ice cream could add a generous extra 100W after some delay. That is already a large sustained increment for a hypothermic person. It takes:

\[
330kJ \div 100W = 3300s \approx 55min
\]

to repay the heat spent warming/melting that kg, and repayment starts late. The victim may not have 55 minutes of physiological competence left.

Eating small amounts? Smaller penalty, smaller benefit. A spoonful is not decisive either way; a binge is bad. The song’s implied behavior—loving and presumably eating rocky road—works against survival in the first few hours.

## Oxygen, CO₂, and the walk-in volume

The scenario says air to breathe, but it is worth checking, if only because small-freezer answers otherwise drift into asphyxiation folklore.

A small walk-in freezer might be ~2m × 2m × 2.5m = 10m³ = 10,000L of air. Air contains ~21% oxygen, so ~2100L O₂ initially. A resting adult consumes ~0.25–0.35L O₂/min; intense shivering might push >1L/min. Three hours at 1L/min consumes 180L O₂, lowering O₂ by ~1.8 percentage points in a sealed 10m³ box. CO₂ rises similarly to ~1.8%.

That is unpleasant, but not the first lethal thing. Many walk-ins are leakier/larger. If the room is tiny and perfectly sealed, CO₂ can matter later. In this problem, hypothermia wins.

## The nutritional red herrings

Counterfactual: make the freezer warm, leave only rocky road. Then the dietary question becomes non-ridiculous.

Rocky road is not a complete diet, but it is not sawdust.

It supplies:

- calories
- water (frozen, but water)
- fat
- sugar
- some protein
- calcium
- vitamin A (varies)
- B12/riboflavin and other dairy-associated B vitamins (varies)
- almonds: vitamin E, magnesium, zinc, iron, some fiber/fat/protein
- sodium/potassium in small-to-moderate quantities depending recipe

It lacks or undersupplies:

- vitamin C, nearly zero
- vitamin K
- folate
- fiber (some from almonds/cocoa, not enough)
- possibly essential micronutrients depending serving size/recipe
- adequate protein if calorie-normal rather than binge-normal
- electrolytes in optimal ratios

The kill order in the freezer:

1. **Hypothermia** — hours.
2. **Dehydration** — classically 3–5 days if no usable water; here ice cream contains water, but using it cold is thermally expensive and irrelevant before hypothermia. If one did somehow live long enough to eat for water, the sugar/lactose/fructose load becomes its own joke: exceed intestinal absorption and one gets [osmotic diarrhea](https://en.wikipedia.org/wiki/Osmotic_diarrhea), a mechanism for converting “hydrating dessert” into dehydration. Still days.
3. **Starvation** — weeks/months, if calories absent; not applicable because calories abundant.
4. **Scurvy** — months, because vitamin C is near-zero. (Maybe earlier symptoms in depleted persons; still not hours.) A classic [scurvy](https://en.wikipedia.org/wiki/Scurvy) clock is measured in stored vitamin C and collagen turnover, not freezer minutes.
5. **Diabetes** — effectively never as an acute cause in a normal adult. A non-diabetic does not eat ice cream for six hours and die of [diabetic ketoacidosis](https://en.wikipedia.org/wiki/Diabetic_ketoacidosis). Hyperglycemia is not the limiting pathology; cold is. If the victim already has brittle type-1 diabetes and no insulin, this is no longer the Weird Al problem but a medical fanfic.

A long-term rocky-road-only diet would be unpleasant: constipation, micronutrient deficiencies, nausea, dental misery, maybe gallbladder/pancreatic issues, metabolic derangement. But that belongs to another room.

## Boundary layers and junk engineering

The answer is sensitive to improvised insulation.

Nude in open freezer air is bad. Nude wrapped in cardboard, plastic bags, and empty ice-cream cartons is less bad. A body-sized nest among tubs can trap still air and block radiation. A “cardboard igloo” is not absurd.

Why it helps:

- reduces convective exchange
- reduces radiative view to cold walls/ceiling
- reduces contact with metal/floor
- traps a warm boundary layer
- lets skin temperature drop while core is defended

The best survival move is not eating. It is reducing heat loss.

A commercial freezer contains:

- cardboard cases
- plastic tubs
- paper lids
- pallet wrap
- wooden pallets sometimes
- shelving
- possibly an air curtain/fan pattern one can avoid
- frozen product with significant thermal mass

Cardboard is useful. Plastic film is useful. Even frozen ice cream tubs can form walls that reduce airflow and radiation if not in full contact with skin. The thermal conductivity of ice cream is not great compared with metal; it contains air (“overrun”). It is still cold mass, not a blanket. Directly hugging tubs is ambiguous: reduced radiation/airflow but increased conduction. Cardboard layer between body and tubs: much better.

Could this extend survival from 2–3h to many hours? Yes, if enough packaging and competence. Could it extend to days? In principle, if heat loss can be brought near/below metabolic production. But building a stable insulating shelter while nude, in the dark/cold, with failing dexterity, maybe no tools, maybe slippery floor, maybe fan noise, is not guaranteed. The baseline scenario says no clothing/rescue, not “genius packaging engineer with unlimited pallet wrap”.

Still air helps. A fan-forced evaporator blast hurts. The difference between −18°C and −29°C matters, but not enough to change “hours” into “weeks”. −29°C increases ΔT by ~11K over −18°C; if skin is 30°C, ΔT goes from 48K to 59K, a ~23% increase in convective/radiative term. Also ice cream is colder, so ingestion penalty rises slightly.

Posture matters. Standing naked is foolish. Curling fetal, hands in armpits/groin, feet off metal, minimizing exposed area, using cardboard under the body: all extend time. Activity is mixed: exercise produces heat but increases convection, uses glycogen, causes sweating if overdone (less likely at −20°C nude, but possible in bursts), and leads to fatigue. Shivering is already involuntary exercise. Running around a freezer until exhausted is probably not a winning strategy unless it produces an escape.

Body habitus matters. Higher fat and larger body size improve survival: lower surface-area-to-mass ratio, more insulation, more heat capacity. Lean/small people do worse. Children do much worse. Elderly, intoxicated, malnourished, sedated, hypothyroid, wet: worse. Nude is worse than nearly any clothing.

## If literally encased/submerged/buried

Different problem.

If Weird Al is literally buried in ice cream:

- airway obstruction/asphyxiation can kill in minutes
- chest compression/immobility worsens ventilation
- conduction over full body contact is much higher than still air
- cold material contacts face/airway
- panic/aspiration

If submerged in cold liquid or semi-liquid dessert, conduction/convection are enormously higher than cold air. Water at 0°C kills much faster than air at 0°C because water conducts heat away efficiently; ice cream is not water, but full-contact cold semisolid is much closer to “water/ice bath” than “air exposure”. Minutes-to-tens-of-minutes, not the 2–3h air case. The prompt excludes this, correctly.

If trapped in a blast freezer at −40°C with high forced airflow, also different: heat loss can be very high; frostbite and incapacitation faster; timeline can compress.

If locked in a domestic freezer chest, oxygen/CO₂ and physical entrapment can become more important, and volume is smaller. Also most adults do not fit well. Not this problem.

## Frostbite

Frostbite is not usually the cause of death before hypothermia, but it matters operationally.

Exposed fingers, toes, ears, nose, genitals: high risk. At −20°C to −29°C, especially with airflow or contact with metal, local tissue can freeze. Wetness from melted ice cream makes it worse. Frostbitten hands cannot unwrap cartons, build nests, or eat neatly. Bare feet on cold floor can become useless. Loss of dexterity begins long before lethal core hypothermia.

So the timeline to “unable to save himself” is shorter than the timeline to death.

Possible incapacitation in under an hour is not dramatic. It is exactly the kind of asymmetry cold produces: the systems needed for rescue fail first.

## Why the naive “food supply” answer appears

Because a freezer full of ice cream is a storeroom, and humans are used to thinking of survival as inventory:

- air
- water
- food
- vitamins

This works in a temperate room. It fails when environmental heat loss exceeds heat production. In cold exposure, “food” helps only if it can be metabolized fast enough and does not impose a larger immediate thermal burden. Frozen food imposes that burden.

There is also a unit confusion:

- calories are joules
- cold exposure is watts
- survival depends on joules divided by watts, with physiological bottlenecks

A person with 20kg of body fat has roughly:

\[
20kg \cdot 7700kcal/kg \approx 154,000kcal
\]

That is ~644MJ. Enormous. Enough energy to run a 500W heater for:

\[
644MJ \div 500W \approx 1.29 \times 10^6s \approx 15d
\]

But the person cannot necessarily convert fat to heat at 500W above baseline while hypothermic, nude, and collapsing. Stored energy is not a wall outlet. If it were, winter exposure deaths would be rare among the overweight. They are not.

## A simple exposure model

Take a middle case:

- freezer air −23°C
- skin effective temperature 27°C after vasoconstriction
- ΔT = 50K
- effective exposed area curled/postured = 1.3m²
- combined still-air coefficient = 10W/m²/K
- respiratory/contact loss = 50W
- heat production while shivering = 400W

Loss:

\[
1.3 \cdot 10 \cdot 50 + 50 = 700W
\]

Net deficit:

\[
700 - 400 = 300W
\]

Temperature-equivalent cooling:

\[
300W \cdot 3600s = 1080kJ/h
\]

\[
1080 \div 235 \approx 4.6°C/h
\]

Again, whole-body-vat simplification. With core defense, maybe core falls more slowly at first. But 2h to dangerous temperatures is not surprising.

Optimistic case:

- good cardboard under/around body
- very still air
- effective coefficient/area product much lower
- heat loss 350W
- shivering 400W briefly

Net zero or positive, temporarily. But shivering cannot run maximally forever; cardboard nest may not be perfect; extremities cool; fatigue begins. If heat loss is kept below ~100W above resting, survival extends a lot. This is why insulation dominates.

Pessimistic case:

- −29°C
- fan airflow
- standing/exposed
- contact with metal/floor
- lean adult
- panic/exhaustion
- eating frozen ice cream

Loss 1200–2000W is possible early. Shivering 400–500W does not keep up. Incapacitation can be rapid; death near 1.5h no longer sounds wild.

## The central estimate

For the specified ordinary walk-in freezer, nude adult, no rescue, not buried:

- **cause of death:** accidental hypothermia, usually via severe hypothermia leading to unconsciousness, bradyarrhythmia/ventricular fibrillation/asystole, respiratory failure; with aspiration/choking possible if eating late
- **time to serious impairment:** tens of minutes to ~1h
- **time to moderate hypothermia:** ~1–2h
- **time to death:** ~1.5–4h, central ~2–3h

Eating rocky road:

- does not prevent hypothermia
- can accelerate core cooling
- imposes ~70–95kcal/kg immediate warming/melting cost
- equals ~1.2–1.5°C core-equivalent per kg
- digestion is delayed and impaired as hypothermia advances
- choking/aspiration risk rises
- only helps in a different scenario where insulation has already solved the watt problem

If he builds an insulation shelter from cardboard/plastic/tubs, avoids airflow and metal contact, and does not gorge on frozen dessert, survival can be extended substantially. If he is encased/submerged/buried, he dies faster, often from asphyxiation or full-contact conductive cooling.

The LLMs were right to ignore scurvy. Rocky road has near-zero vitamin C, so scurvy is a fine months-later answer in a warm dessert bunker. The freezer does not give months. Diabetes is even less relevant. Dehydration is a 3–5-day topic in the usual no-water case; here the water is locked inside a −20°C heat sink, and the victim is dead before renal bookkeeping matters.

Rocky road is not poison, or even bad rations. It is just the wrong object on the wrong side of the heat equation.

eval_iclmargin_fb100_tg_gpt55_xhigh_comp100k_20260702_generating-style · final judges 0/1 · 21,481 chars · 3,157 words · /home/ubuntu/code/lesswronger/data/agent_runs/eval_iclmargin_fb100_tg_gpt55_xhigh_comp100k_20260702_generating-style/submission.md

# Style-Aware Generative Models

*status: sketch; confidence: medium; importance: if-true high; tags: AI, generative-models, art, copyright, counterfactuals*

<div class="abstract">
Image generators can render any *named* style but can't *invent* one, because a style isn't a bundle of visual features you can interpolate to; it's a historically situated package (works + techniques + names + rivalries + institutions + the fact of adoption by particular artists and audiences at a particular time). So the interesting target isn't a better i.i.d. sampler but a generative system that models *chronology and influence*: a latent space structured by time and causal descent rather than by appearance/semantics, which then supports counterfactual art histories, causal credit assignment (for IP or just for understanding), and forward simulation of new movements. I sketch a contrastive-learning implementation (temporal contrastive loss on date-rich corpora → reticulate influence-DAG → phylogenetic soft-deletion for counterfactuals → EM-style self-consistent refinement) and the pipeline it enables: images → embeddings → influence graph → counterfactual histories → virtual artists → adoption dynamics → movements.
</div>

# What the generators actually can't do

Two things, and they're the same thing seen from two ends.

## Inventing styles

The trivial objection first, because people reach for it: *of course* a diffusion model contains every style, you can embed nearly any image near-losslessly into its latent and get it back, so in some information-theoretic sense the whole history of art is already in there, and yes if you mash the "sample" button on [Midjourney](https://www.midjourney.com/) you will get striking images no human ever made. Granted. But producing a striking image is not inventing a *recognizable new style*, and the gap between those two is the entire subject of this note. A style is not a set of visual flourishes that happens to be popular; it is a coherent package — some works, some techniques, a *name*, a group of people who advocated it and a group who hated it, a moment, and none of that is a property of pixels.[^slop]

[^slop]: The one time a generator arguably *did* invent a style, [DALL·E 3](https://openai.com/dall-e-3) (via ChatGPT's image gen) coughed up the "Shrimp Jesus" AI-slop æsthetic (over-saturated, plasticky, uncanny-symmetric, faux-devotional), but it did so *accidentally*, as an artifact of preference-learning gone into mode collapse (RLHF pushing toward a narrow high-reward basin), and the style was named and canonized *retrospectively* by humans on Facebook and Twitter. The model had zero internal capacity to invent a style deliberately, and, the part that matters, zero capacity to then invent *a reaction to that style once it got boring*, which is what a real movement does. Slop can't get sick of itself.

Why don't styles emerge from these systems, given that they contain the ingredients? Two collapses, one social and one internal, and they rhyme:

1. **Cultural filtering collapsed.** New movements are made socially — by exclusion, by manifestos, by a scene in a city, by galleries refusing you and then not refusing you. "Everything everywhere all at once" (infinite instant access to all styles, all remixable, all free) dissolves the friction that let a movement *cohere* as against a background. When every style is one prompt away, none of them can be the local fashion that a group defines itself by.

2. **Context collapse inside the model.** The generator treats an image as a collage of recombinable discrete elements ("person" + "wearing glasses" + "impasto" + "golden-hour"), and *styles exist in it only post hoc, as named clusters* someone later drew a boundary around. There is no representation of a style as a thing that came from somewhere, reacted to something, and was carried by someone. The model has appearance without descent.

So the failure to invent styles is not a capacity failure of the pixel machinery; it's a *representation* failure, the object "style" isn't in the model as a first-class thing with a history.

## Assigning causal credit

The dual problem, and the one with money attached. We'd like to answer: *how much did this particular artwork contribute to this generator's knowledge/value?* — for IP attribution, or just for art history. There are two standard answers and both are bad, in instructively opposite ways.

**Right method, wrong question.** [Leave-one-out](https://en.wikipedia.org/wiki/Cross-validation_(statistics)#Leave-one-out_cross-validation) retraining (LOOCV): drop a datapoint, retrain, see how much worse the model is. Rigorous. But run it on the [*Mona Lisa*](https://en.wikipedia.org/wiki/Mona_Lisa) while keeping in the corpus its ten thousand parodies, homages, "Mona Lisa but as a cat" prompts, art-history textbook reproductions, etc., and the retrained model reconstructs the *Mona Lisa* just fine, so LOOCV reports it as *redundant*, near-zero contribution. Which is a true answer to the question LOOCV actually asks — "is this datapoint redundant *given the rest of the corpus*?", and a nonsense answer to the question we meant — "what if this work had never been painted, *and culture were correspondingly impoverished* (no parodies either, because there's nothing to parody)?" Influence functions[^if] share the exact flaw at lower cost: they estimate which *datapoints* shifted the *model's* outputs, never which historical *works* shifted *all subsequent works*.

[^if]: [Koh & Liang 2017](https://arxiv.org/abs/1703.04730) style influence functions, or the [TRAK](https://arxiv.org/abs/2303.14186)/datamodel line, cheaper LOOCV surrogates. Cheaper approximations to the wrong question are still answering the wrong question; the error is in the counterfactual, not the estimator.

**Cheap method, right question.** Just look up the training images most similar to an output and declare causation. This is the one everyone actually does (it's a nearest-neighbor lookup, minutes not GPU-months) and it's *post hoc ergo propter hoc* wearing a lab coat: visual similarity is not causal descent. Two independent parodies of the same concept look nearly identical without either causing the other; a forger and the forged look identical with causation running one way only. Similarity throws away direction and throws away independence, which are the whole of causality.

The fix implied by both failures is the same: we need a latent space where *causal descent* is represented directly, not inferred from appearance. Which means time.

# Style embeddings (the 2022 version, kept for the record)

My first pass[^2022] was: make a generator "style-aware" by clustering in a style-specific embedding space, and exploit the geometry. Styles, embedded, don't fill the space — they form *island-chains*, tight archipelagos of works surrounded by empty ocean. So: hunt for *holes*, regions of high model likelihood (the model thinks images there are plausible) but anomalously few nearby real datapoints. Those are *missing styles*: coherent things that *could* have been made and weren't. A hole adjacent to a popular style is a counterfactual direction that movement could have taken ("Impressionism could have gone *this* way instead"). Uses: seed a creative-adversarial net (CAN)[^can] toward a hole; or diversify a generator so each "Impressionist" sample is nudged toward a *different* adjacent missing style rather than the mode.

[^2022]: Which was fine as far as it went, but it's a *static* geometry — it has islands but no ocean currents, no arrow of time, so it can find holes but can't tell you *when* or *from what* a hole would have been reached, which is exactly the causal information the credit-assignment problem needs.

[^can]: Elgammal et al 2017, "[CAN: Creative Adversarial Networks](https://arxiv.org/abs/1706.07068)", GAN + a loss that rewards being classifiable as art but *not* as any existing style. A local novelty pressure with no memory of where novelty came from.

# Temporal embeddings (the improved thing)

The credit-assignment problem is what forces the upgrade: build the latent space so its structure *is* causality/time.

Ordinary embeddings ([CLIP](https://openai.com/index/clip/), say) spend their capacity on whatever explains the most variance, which is semantics and appearance — "person," "glasses," "outdoors," "impasto." Temporal structure is *in there* (1650 paintings do cluster away from 1950 ones) but it's not *prioritized*; it's a byproduct entangled with everything else, and you can't cleanly manipulate it.

So overweight it. The target embedding: **push together works close in creation date; push apart works distant in time even when they look identical.** (A 2023 pastiche of Vermeer and a real Vermeer must land far apart, same appearance, three centuries of causal distance.) Concretely:

- **Loss.** Contrastive learning within each batch using metadata, creation/publication *date* being the important one, on date-rich corpora ([WikiArt](https://www.wikiart.org/), [ART500K](https://deepai.org/publication/art500k-a-large-scale-dataset-for-visual-art-understanding), museum catalogs). A temporal contrastive term added to a normal generative loss. If your generative architecture has no usable steerable latent (many diffusion U-Nets don't, cleanly), train a *separate* CLIP-shaped embedding model on the temporal loss and use it to steer a stock generator (classifier/embedding guidance).

  Sketch:

  ```
  for batch of (image_i, date_i):
      z_i = f(image_i)                       # embedding
      for pairs (i, j):
          w_ij = kernel(|date_i - date_j|)   # ~1 if close in time, ~0 if far
          # weighted InfoNCE: temporal neighbors positive, temporal
          # distants negative, REGARDLESS of visual similarity
      L = L_generative + λ · L_temporal_contrastive
  ```

- **Resulting geometry.** A loose tree — really a reticulate DAG (art history is not a tree; movements have multiple parents), with dense *bands by year*, each band internally organized by style/influence, each work sitting adjacent to its plausible predecessors and successors. And, nicely, the *heavily influential ancient sources* (Egyptian friezes, Greek sculpture) **bulge**: they sit near *everything*, because movement after movement reaches back and touches them, so respecting all those later "reaches back to X" relations pulls X toward the center. That's correct behavior, though we'll see it also causes trouble.

- **Why causality falls out for free.** Later works cannot cause earlier ones. So a space that wants to compress "who is near whom in influence" is *forced* to respect the time order — the arrow of time is the cheapest available regularity, and thus the embedding recovers direction without ever being told about causation as such. And around each real work sits a cloud of *counterfactual works that could have been made at that time and weren't*.

- **Selection bias is fine.** Most art is lost, unpreserved, never scanned, never in the training set. Acceptable, because the lost works *weren't available to the contemporary artists either* — so a model of influence built on the survivors is modeling roughly the same information the historical actors had; and holes let you *infer* the missing ones anyway.

- **Time need not be one axis.** It'll be smeared across directions, but you can *extract* a time direction (contrast pairs early-vs-late, or supervised regression of date on latent) and then manipulate it like any GAN latent attribute (à la "add smile vector").

# Simulating art history

Once time is an extractable, manipulable direction, the capabilities are mostly geometry:

**Island hopping.** Walk the extracted time axis monotonically forward and you *replay* history, or *extrapolate* new history by sampling latent points that keep moving forward in extracted-time. Sampling wants **small jumps** (incremental works, the day-to-day of a movement) *plus* occasional **big jumps** (a Lévy-flight[^levy] tail, the rare rupture that starts something).

[^levy]: Heavy-tailed step lengths give you both the local grind and the occasional teleport, which is empirically what innovation-search wants; pure Gaussian steps never leave the neighborhood.

**Forward invention.** Embed Shrimp-Jesus-slop at its date (2023), then *branch out a rooted tree of hypothetical responses*: negations (anti-slop), extensions (slop-maximalism), fellow-travelers (adjacent æsthetics), derive new styles from the clusters, then generate responses *to those*, recursively. History as a growing DAG rather than a filmstrip.

**Backward counterfactual attribution — the thing we actually wanted.** Embed the *Mona Lisa*; identify its *descendants* (its forward light-cone in the latent, everything downstream that its influence flows into); **delete** them; retrain on this impoverished history; measure how much worse the model is. *That* is the counterfactual LOOCV was supposed to compute and couldn't — because now dropping the *Mona Lisa* also drops the parodies (they can't exist without their target), so the model genuinely can't reconstruct it, and the true, large contribution shows.

**Soft deletion when hard deletion is too violent.** Some works are upstream of *so much* (the bulging ancients) that deleting the light-cone deletes half of art, an analogue of **[pedigree collapse](https://en.wikipedia.org/wiki/Pedigree_collapse)** (go back far enough and everyone is everyone's ancestor). So instead of a hard cut, use **soft phylogenetic weights**: sample ancestor→descendant paths through the DAG and weight each downstream work by how much of its lineage passes through the target. A near-copy of the *Mona Lisa* gets weight ~1 (basically all its lineage is the target); a merely-same-medium contemporary gets ~0. This gives a graded, non-catastrophic counterfactual.

**Uncertainty via [bootstrap](https://en.wikipedia.org/wiki/Bootstrapping_(statistics)).** Point estimates of "how important was work X" are false precision. Resample the training set, refit the tree, retrain, repeat → interval estimates: "the model is 12% worse without it (4–19%)." Report the interval.

**Self-consistent refinement (EM-ish).** The embedding and the tree should agree, so iterate:

```
1. train temporal contrastive embedding
2. fit best phylogenetic tree/DAG, with KNOWN DATES as hard constraints
3. compute pairwise tree distances (path lengths through influence)
4. retrain contrastive embedding to match those tree distances
5. goto 2  until self-consistent
```

Step 4 is the key correction: it lets *direct credit assignment* (the tree) discipline the *appearance-driven* embedding, fixing the ill-behaved long-range bulges — an over-central ancient work gets pulled back if the tree says its actual descendant-count is smaller than its visual centrality implied.

**Byproducts of a good tree.** Ancestral reconstruction (interpolate the un-preserved parent work that two children imply); and *discovery of hidden influence*, e.g. the tree posits that some African mask sits on the path to a [Picasso](https://en.wikipedia.org/wiki/Pablo_Picasso), which is either an error to correct or a *research hypothesis* to check (and, famously, sometimes true). The model will be overconfident. Fine — it's a hypothesis generator, and overconfident-but-cheap hypotheses beat no hypotheses.

**Evaluation** of such a counterfactual model: after deleting *Mona Lisa* + descendants, (a) how *hard* is it to embed a held-out real *Mona Lisa* (should be much harder), (b) how much lower the likelihood of generating *Mona-Lisa*-like samples, (c) how much lower end-users value the outputs.

# Use-cases

## Intellectual property

This gives a *philosophically and æsthetically meaningful* attribution of causal influence, including plain copying — of copyrighted works on a given sample, one **not** fooled by surface similarity (the cheap method) and **not** answering the irrelevant is-it-redundant-in-this-dataset question (LOOCV/influence functions). Which is what you'd actually want if copyright were doing its [constitutional job](https://constitution.congress.gov/browse/essay/artI-S8-C8-1/ALDE_00013060/), "to promote the Progress of Science and useful Arts" — i.e. rewarding works that *genuinely influenced others* and not rewarding works that didn't, and it enables sanely-designed schemes (compulsory licensing keyed to measured downstream influence, say) instead of all-or-nothing injunctions.[^ipfoot]

[^ipfoot]: Most AI-copyright commentary rests on moral-desert intuition ("they *took* my work!") or anti-corporate sentiment rather than on copyright's *function*, and the proposals that follow tend to backfire on their own supporters: "style is IP" and "every model is a derivative work of all its training data" both *privilege the incumbents*, Adobe, Disney, Getty, anyone sitting on a huge in-house or already-licensed corpus — while making training *infeasible for everyone else*, because rights-clearing across billions of works one-by-one is not a hard problem, it's an impossible one, and impossibility is a moat. If you want small labs and open models to exist, "all training is derivative" is the last rule you want.

## Measuring novelty, importance, fertility

Truncate the model at a date (train only ≤1850) and ask which later movement it finds most *surprising*, was Cubism or Impressionism the bigger jump given only pre-1850? (This doubles as a diagnostic for the EM loop: after rationalization cycles, later works should get *more* predictable — if they don't, the tree isn't converging.) And keep three distinct objective quantities apart, because people constantly conflate them:

- **novelty**, surprise given the past (how far the jump was);
- **importance** — how many downstream works it caused *in total*;
- **fertility**, how many *distinct known styles* it spawned, even if none got popular (a low-influence, high-fertility work is a seed that keeps sprouting weird things).

A movement can be novel and infertile (dead end), or unoriginal and enormously important (right synthesis at the right time).

## Generating styles

Run history forward to predict *plausible new styles*: branching future trees where each style has *context* and *reacts* to earlier hypothetical styles; search for the most-plausible futures (plausibility correlates weakly with quality, which is enough to prune). Let users curate/backtrack the tree of samples. Convert a cluster into a *named* style (hand it to a VLM: "name and describe the common æsthetic") or a *usable* one (average a LoRA / distill a steering vector over the cluster — now addable and *subtractable*, "more of this, minus that"). Simulate *schools* by learning and *fixing* an artist latent dimension and drawing *time-ordered* sample sequences per fixed artist; you get artists with **careers**, exploring and exhausting a style over a run.

This is the point where the system is meaningfully generating *styles* rather than *images*: the win is in *structuring the latent for historical causality and context*, not in i.i.d. sampling quality.

## Inventing styles (the strong version)

Take a latent *artist* point and inflate it into a simulated artist: feed the summaries/analyses of its work-sequence to an LLM and have it *confabulate a biography, an æsthetic, an ideology* that retrodict the works ("given this trajectory, who would have made it and why"). In the limit these become **policies with memory** — taste, constraints, a rival they're defining themselves against, a preferred medium, a social network, a career arc, a parasocial fandom, virtual artists with net presences who interact with each other and with fans. Artificial-life loop, per timestep:

```
each timestep t:
  for each artist a:
      view recent works within a's attention/social graph
      generate new work(s) conditioned on a's latent + memory + rivalries
  critics/audiences (learned selectors) select/amplify some works
  selected works update the influence graph (edges = who saw/reacted to whom)
  persistent dense clusters get NAMED as movements (VLM)
  update artist memories, alliances, boredom
```

Such a thing generates lineages, schools, rivalries, reactions, and *stale phases* — not isolated images. Which lets us finally *redefine* the term: **style invention** = generating an æsthetic lineage that (a) stays recognizable, (b) develops internal variation, (c) acquires simulated advocates *and opponents*, and (d) survives long enough to be *named*. Concession, stated plainly: this is still only simulation. But it is the *right object* to simulate, a movement, with a history and a social life — rather than a prettier draw from a fixed distribution.

---

The program in one line: **images → embeddings → influence graph → counterfactual histories → virtual artists → adoption dynamics → movements.**

And the sharp end of it: once you can *counterfactually retrain* on histories where the *Mona Lisa* was never painted, "originality" stops being a mystical property of an object and becomes a *measured quantity*, the size of the hole a work leaves in the causal graph when you cut it out — which means the first real casualty of style-aware generation is not the artist's income (that was already lost to the cheap similarity lookup) but the artist's *myth*, because we will finally be able to check, work by work, how much of it was ever actually there.