CS 319 · Meeting 2 of 10

The Council
and the Context

Why one brain isn't enough — and why a 200k window doesn't mean you should fill it.

Agenda · 100 min

The council pattern: four variants, three failure modes (30')
Context engineering: the five rules (25')
Case: Kirana's architecture decision (10')
In-class: run a council debate (25')
Assignment A2 briefing (10')

Reading: chapters 4–5

Core concept · ch 4

Why one brain isn't enough

A single call gives you

one perspective, one blind spot
self-agreement: it grades its own homework
confidence indistinguishable from correctness

A council gives you

a debate: specialized agents with clashing incentives
failure isolation: one bad take gets outvoted
research-backed gains on high-stakes calls (SWE-Debate, LLM Council)

The foreman philosophy in one line: a team of disagreeing specialists beats a single confident voice. Not for every call — for the high-stakes ones.

Core concept · ch 4

Four council variants

Voting

N agents answer independently; majority wins. Cheap, parallel, kills outliers. Use for classification, triage, go/no-go.

Specialization

Each agent has a role — architect, security, cost, UX — and reviews through that lens only. Use for design review.

Debate

Two-plus agents argue opposing positions across rounds, then synthesize. Use for architecture decisions, tradeoffs.

Judge + Jury

Specialists produce; a separate judge (different model) scores against a pinned rubric. Karpathy's chairman pattern. Use for quality gates.

Picking rule: if being wrong is cheap → one model. If being wrong is expensive → which variant, and why? (You will defend the choice in A2.)

Failure modes · documented, not hypothetical

How councils fail

Groupthink

All agents converge without real debate; the synthesis is whatever the loudest said.

Fix: anonymize rounds, mix model families.

Forceful agreement

Agents abandon their position the moment they see a peer's confidence.

Fix: independent first passes before any sharing.

Prolonged disagreement

Rounds never converge; tokens burn; nothing lands.

Fix: round cap + chairman with tie-break authority.

Example · copy-pasteable

A specialization council, in one prompt each

# reviewer-security.md — one node of a specialization council
You are the SECURITY reviewer on a council. You review ONLY through
your lens. You do not praise. You do not review style or performance.

CONTEXT (curated, not pasted):
  - spec.md       (task boundary + acceptance criteria)
  - api-contract.yaml

OUTPUT (JSON only):
  { "verdict": "pass" | "fail" | "needs-revision",
    "criterion_ids": ["AC-3"],          # tie to numbered criteria
    "reason": "one paragraph, evidence-based",
    "refusal_conditions_triggered": [] }

RULES: If evidence is missing, verdict is needs-revision — never guess.
Different model family than the producer. Read-only tools.

Note the three disciplines inside one prompt: lens (specialization), numbered criteria (spec, M3), different model + read-only (maker-checker, M5).

Core concept · ch 5

Context engineering ≠ prompt engineering

A 200k-token window is a budget, not a landfill. Context engineering is curating what enters the window, what gets compressed, what is retrieved on demand, and what gets dropped.

5 rules

budget: 60–80% max
memory hierarchy: hot → warm → cold
compact at the limit
scope tools per role
offload reasoning to files

Context rot

Long, stale, irrelevant context degrades output even when the model "fits" it. Utilization is a quality dial, not just a cost dial.

Sub-agent isolation

Give the noisy work (search dumps, raw logs) to a sub-agent that returns a distilled artifact. The main window never sees the swamp.

Example · the audit you'll do in A2

Before / after: one code task's context

The landfill (before)

full 40-file repo pasted (61k tokens)
3 old chat transcripts
yesterday's stack trace
"you are a 10x senior expert…" padding
no acceptance criteria

Utilization 96%. Context rot by minute two.

The bundle (after)

spec.md — 40 lines, numbered criteria
api-contract.yaml — the 2 relevant endpoints
db-schema.sql — 3 tables, not 30
one example of "done" (a past PR)
tools.yaml — 4 scoped tools

Utilization 22%. The agent has everything and nothing else.

Case study · part 2 of 10

Kirana's monolith question goes to a council

The PM wanted "AI to decide" if CekFaktur should split its monolith. One model said yes, confidently — twice. Instead, Kirana ran a debate council of four: Architect (pro-split), SRE (anti-split, cost lens), Security (compliance lens), Chairman (synthesizer, tie-break).

The bundle: one page of traffic data, the two hottest modules, a schema of shared tables, one real incident report. Total context: 9k tokens.

Result: after 2 rounds the council converged on "modularize the invoicing path only" — a position nobody held at round zero, and one the team actually shipped. First pass, alone: the wrong answer with total confidence.

In-class · 25 minutes

Run a council, feel the failure modes

Setup

Groups of 4. Decision: "Our student portal should replace email announcements with an in-app feed." Role cards: PRO (shipping speed), CON (inclusion/accessibility), SECURITY/PRIVACY, CHAIR.

Rules

Round 1: independent positions, written silently (anti-forceful-agreement).
Round 2: anonymized exchange, then revise.
Round 3: chairman synthesizes or deadlocks. 2-round cap before the chair must break it.

Debrief: which documented failure mode did your group hit? Groupthink, forceful agreement, or a chair that decided too early?

Assignment A2 · due before Meeting 3

Council + context audit

Part 1 — council
Take a real decision (yours, your team's, Kirana's). Pick a variant and defend the choice in 3 sentences. Run a 3–4 agent council (any chat LLMs; different models per role if possible). Log each round.

Part 2 — context audit
Take your longest recent AI conversation. Rebuild it as a bundle: what should have entered the window, what was missing, what was redundant. Estimate utilization before/after.

Deliverables
Debate log (rounds + verdict JSON) · 3-sentence variant defense · before/after context table.

Rubric
Variant justification 25% · debate quality (did real disagreement occur?) 35% · context audit honesty 40%.

Next: the spec and the loop — where intent becomes machine-checkable. Read ch 6–7.

The Counciland the Context