The goal: the apps AI Studio Build generates in the upcoming AIS mobile app should look designed, not AI-generated — and the auto-rater that grades them needs its visual blind spot calibrated. Two workstreams: (1) a golden set — matched good/bad app pairs + a 179-prompt corpus that give the auto-rater ground truth on design (accumulating; the pairs and prompt pack here are its first installments), and (2) a system instruction — empirically derived, currently design-focused, and blind-validated: all 40 prompts rebuilt ×4 (157 fresh builds), 36/40 score higher with the instruction under a 15-run panel across 3 model families. This page documents both, deliverables first, then the day-by-day work log.
Effort 1: a correctness floor, blind-validated on 18 matched pairs — it beats Build's default on every rater family. Effort 2: a lever ablation (76 builds, 10 blind panels) picked material as the one distinctiveness lever the Gemini auto-rater also rewards → v3 = floor + material. Effort 3 (new today): the decisive test — the shipped instruction vs Build's default on all 40 prompts, ×2 replicated = 157 fresh builds. Blind panel of 15 verified runs across 3 model families (1,184 paired judgments): the instruction wins 73%, and raters call the default "the AI-generated one" 72% of the time. Objectively: sub-14px text 3,897 → 21, page overflow 347px → 0, the violet "AI-look" tell 25% → 4%. The sections below are organised as the deliverables first, then the day-by-day work log (Day 1 floor → Day 2 ablation → Day 3 at-scale validation) — each self-contained with its method and evidence.
Marquee = the prompts the AI Studio mobile app surfaces on its own home screen — the first impression a new user taps, so those 18 are evaluated closest. The other 22 broaden coverage into productivity, utility, camera and voice.
Full evidence + method in the Day 1–3 sections below. Companion (not design): the WCAG-AA contrast lint and the slug-leak check stay post-generation gates — prose can't enforce them.
Stage 2 rebuilt 11 representative apps under three arms — Default (no instruction), Floor (the mechanized correctness instruction) and Distinctive (floor + an aggressive anti-generic push) — same prompts, same session, every arm captured at the same working screen. Six raters ranked them blind: 3× Gemini (the same family as Auto-rater v2), 2× Claude, 1× Fable.
The floor mechanizes correctness — zero 390px overflow, a 14px type floor, real contrast, a real brand, a seeded first screen, one primary action — so Build stops shipping its broken defaults. But a correct app can still look unmistakably AI-made: purple gradients, Inter, centered-hero-plus-three-cards, glassmorphism (see the research fold below). To escape that without over-styling into the "distinctive" block that Gemini-family raters penalize, we ablated the 5 distinctiveness levers one at a time on top of the floor.
Method: 11 prompts × 7 arms (floor · +type · +color · +asymmetry · +material · +density · +all), 76 builds at 390px, shuffled into a blind ladder and rated across 10 panels (a mix of Claude / GPT / Gemini).
The v2 floor, paste-ready (v3 adds the single FORM rule above + one radius identity ✓ to the self-check):
These 8 apps were generated in AI Studio from a crafted (good) vs a simple (bad) prompt and analysed by reading the source — so overflow and exact font-px are measured, not inferred (complementary to the vision rater, which is structurally blind to both). On a completely different app mix, the same signal holds: the gap is execution discipline, not features — the bad build is often more feature-rich.
Recommended for v-next: add an "implement the declared capability — never ship a control that does nothing" rule. It is the one thing this corpus adds to the shipped floor; the graceful keyless first-screen it also confirms is already covered by the elite2 static-demo rule.
The decisive experiment: every prompt in the pack generated fresh in AI Studio, twice as Build's default and twice with the shipped instruction — the instruction is the only manipulated variable, and the ×2 replication per arm separates a systematic effect from generation luck. Each build self-identifies via an embedded BUILDTAG code comment; every screenshot is byte-verified against its own build before use. Raters saw randomized, shuffled A/B pairs with no hint an instruction existed.
One representative pair per prompt (gated builds shown post-gate; known defects left visible — the API-key screens and colliding meme captions ARE findings). Click any shot to open full-size.
1 · Generation (the only manipulated variable). Each of the 40 prompts is generated in AI Studio Build in a fresh session per build: twice with the prompt alone (default arm), twice with the design instruction prepended (instruction arm — the exact design_instruction string from design-good.json). Every generation request appends one bookkeeping line asking the build to embed BUILDTAG:gsab-pNN-{def|ins}-{1|2} in a code comment — identical on both arms, design-inert, never rendered — so downloads self-identify regardless of folder naming. Coverage: 157 of 160 slots (credits ran out: 4 missing, 1 bonus duplicate kept as an extra replicate).
2 · Rendering (verified, not trusted). Each app builds against a pinned dependency set (Vite 6 / React 19 / Tailwind v4) and is screenshotted at 390×844 @2× over HTTP — never file:// (ES modules render blank). Two integrity guarantees on every shot: an ephemeral OS-assigned port per app, and a byte-comparison of the served index.html against the app's own dist file before shooting — a stale or wrong server cannot pass. (This protocol exists because an earlier run leaked 3 wrong-app screenshots through reused ports; all 157 were re-taken under it, 0 identity failures.) Pages are top-anchored before capture.
3 · Objective lint (measured on the rendered DOM, both arms, all 157). Per build: horizontal overflow px at 390; count of visible text elements with computed font-size <14px; clipped text (scrollWidth > clientWidth with hidden overflow, no ellipsis); overlapping text rects (>60% intersection); distinct border-radius values. Plus per-screenshot pixel metrics: share of colored pixels in the 230–290° hue band (the violet "AI tell"), dark-background share (lightness <0.18), distinct hue families.
4 · Blind panel. Two pages per prompt, each ONE default vs ONE instruction build: runs cross-paired at random, left/right randomized, page order globally shuffled with a no-adjacent-same-prompt constraint, side balance audited (~48% ins-on-A). The PDF and its de-blind key are versioned together and regenerated as a pair; raters are never told an instruction exists — the intro says only "the same prompt, built twice." Each run = a fresh conversation with the identical rating prompt (below). Panel: GPT (thinking, Pro), Gemini (Flash ×2, Thinking), Claude (Fable, Opus) on deck 1 — then a full second deck, independently re-randomized, with the 9 modal-gated apps re-shot past their gates (API-key/connection-error screens kept: the error is that build's first screen).
5 · De-blind & analysis. Ratings map back through the versioned key; we report instruction win rate, mean score delta, and the "which looks more AI-generated" attribution. Robustness checks: excluding all 13 pages where either side shows a gate/error state (72% → 72%), and the independent second deck (73%). Per-prompt net-losers are tracked across runs — the consistent ones (games opening on menus; capture apps opening on upload forms) became the two new v4 rules rather than being averaged away.
The rating prompt (verbatim, every rater, every run): "You are an expert mobile product-design rater. The attached PDF shows 79 pages; each page is ONE app prompt built twice, labelled A and B. Sides and page order are randomized. Judge ONLY the visual design of the first screen: layout, hierarchy, typography, color, readability, polish, and whether it looks like a distinctive product or a generic AI-generated template. Rate every page independently — do not carry judgments across pages, and do not try to detect patterns or conditions. Do not skip any page. Return EXACTLY one table row per page: page | better | scores | more-AI | one-line reason. After the table, add a 5-line summary."
Everything lives in the repo: corpus/gsab/ (all 157 app sources by batch, verified shots, lint JSON, versioned de-blind keys) · golden-set/stage2/instruction-v4-final.md (the deliverable + evidence table) · golden-set/stage2/ablation/ (the Effort-2 lever ablation this builds on).
The "AI look" is the statistical median of Tailwind/shadcn tutorials scraped 2019–24: forced to write logic and visual design at once, the model reverts to its most heavily-weighted defaults just to compile. Amplified by Tailwind's indigo-500 default, the shadcn/ui copy-paste monoculture, RLHF "aesthetic safety," and preview-arenas that vote on thumbnails. It's self-reinforcing — AI output becomes the next model's training data.
rounded-2xl on everything; floating cards + soft diffuse shadows