CS 319 · Meeting 8 of 10
The chapter that turns a coder who uses AI into a foreman — and the catalog to read when a loop misbehaves.
Agenda · 100 min
Reading: chapters 16–17 · glossary
Core concept · ch 16
Prompting
Instructing one run. Necessary, transient, and the smallest part of the job.
~20% early, →5%
Coaching
Leaving notes on output so the next run does better. The foreman's default mode.
~60% mid
Codifying
Graduating repeated notes into playbooks, evals, and skills the agents find themselves.
→dominant late
The 30% rule
When roughly 30% of runs hit a coaching note without failing because of it, the note graduates into the playbook. Below that it's noise; above that you're late.
The graduation ladder · ch 16
# coaching note (3rd time this month) → SKILL.md
---
name: idr-currency-rounding
when: any output contains money amounts in IDR
body: round HALF_UP to whole rupiah AFTER window/eligibility checks,
never before. show the pre-round value in a comment.
verify: golden inputs 003, 004 must pass exec-match
---
A note you've given three times and not codified is a choice to keep paying the same tax forever.
Catalog · ch 17 · name the failure to stop guessing at the fix
prompt stuffing — five jobs in one 3000-token prompt; extract the specialists
council groupthink — convergence without debate; anonymize, mix model families
prolonged disagreement — rounds never end; chairman + round cap
forceful agreement — caving to a confident peer; independent first passes
single-agent scope creep — the "agent" grows subheadings; migrate (M5)
hidden cost explosion — no budgets; caps per run and per day
verifier self-deception — maker grades own work; separate checker
context rot — landfill window; 60–80% budget, sub-agent isolation
spec rot — spec drifted from reality; re-baseline, version it
automation theater — runs ≠ value; landed-outcome column
The test underneath the catalog
The documented case: a reply loop ran 140 times, all ok — and had landed nothing for a week, because a CLI bug made every run silently skip. Activity metrics said reliable. The landed column said dead.
Case study · part 8 of 10
Week 9: the digest loop's LLM-judge scores had been climbing for a month — 8.1, 8.4, 8.6. Lovely graph. The foreman (Rasa, now spending mornings on reviews) noticed humans were editing 3 of 5 digests by hand.
Diagnosis against the catalog: verifier self-deception — the judge was the same model family as the writer, shared context, and its calibration set was one example. From month two, the maker and judge had quietly co-drifted. The gate score had come apart from the thing it proxied.
Fix: different model family for the judge, 5-item calibration set (2 bad), version pinned. Scores dropped to 6.9 — and human edits went to zero. Lower score, better loop. The coaching note became the judge-independence skill; the 30% rule fired in three weeks flat.
Diagnosis clinic · 30 minutes
Loop A
Runs daily, 6 weeks. Dashboard green. Nobody has merged its PR in 12 days. The team defends it: "it's 90% reliable."
Loop B
The "architecture advisor" agent is brilliant, but every session costs 8× its budget and takes 40 minutes — it re-reads the whole repo each time.
Loop C
Council of four agents. Outputs are consistently well-written, suspiciously similar, and always agree — decisions feel pre-decided.
For each: name the anti-pattern, cite the chapter, prescribe the fix in one line, and name the observability signal that would have caught it early. (Answers are in the catalog — the discipline is the naming.)
Assignment A8 · due before Meeting 9
Part 1 — post-mortem
Take the worst thing your A6/A7 loop did. Name it against the catalog (or argue it deserves a new entry — write the entry: name, description, example, workaround).
Part 2 — graduation
Find the coaching note you've left most often. Graduate it to an SOP-grade prompt or skill file, with the when and verify fields filled.
Rubric
Correct diagnosis (or a genuinely defensible new entry) 50% · graduation quality: could another agent invoke it cold? 50%.
Next week: the future engineer and the graph — the layer above loops, and your capstone kickoff. Read ch 18–19.