Give the advisor and the market agent the same nightly habit — a four-phase dream that maintains a living wiki of everything each one is learning. Design + build spec · 2026-06-26
In one breath: A HomeZero agent earns its keep by getting better every day, and an agent that only ever appends doesn't get better — it just gets longer. The fix has a name in two places already: Claude Code's AutoDream (a nightly four-phase memory-consolidation pass — orient, gather signal, consolidate, prune & index) and Andrej Karpathy's LLM Wiki (raw sources → an LLM-authored, backlinked wiki → a lint that keeps it healthy). They're the same idea, and HomeZero should adopt both wholesale. Each agent gets a private wiki — a lean index pointing to backlinked pages, stored in its own Durable Object's files table — that its working beats ingest into and its nightly dream lints. The advisor's wiki is a person's search; the market's wiki is a place, fed by many sources from day one — Zillow plus county records, Google Maps, water rights, and local news. But the expensive per-property enrichment is gated by attention: only homes someone views, saves, or zeroes get the deep multi-source treatment — the whole market never does, or we'd drown. "Comps," "what fits this buyer," "what's moving" are all just queries against a wiki, never the wiki itself. Everything the public sees — board cards, briefings, the market read — is a projection of a wiki page, not a separate brain. Shared machinery, separate minds: one src/brain/ module, two wikis that never read each other.
Two shipped references describe exactly the beat we want, from two angles. We take both literally.
A background consolidation pass that runs while you rest, in four phases: Orient (read the memory dir), Gather Signal (scan recent transcripts for corrections & patterns), Consolidate (merge findings into memory), Prune & Index (rebuild a lean index — Claude Code keeps MEMORY.md under ~200 lines, the cutoff for what loads at startup). This is the shape of our dream.
Three layers — raw/ (immutable sources the LLM reads, never edits), wiki/ (LLM-authored, backlinked concept pages + an index.md), and a schema doc that "makes the LLM a disciplined wiki maintainer rather than a generic chatbot." Three ops — ingest, query, lint. This is the thing the dream maintains.
They line up one-to-one. Ingest happens on the agent's working beats; query is how it answers; and lint is AutoDream's prune & index — the nightly dream. Working beats append (fast, finite, reactive); the dream is the one pass allowed to rewrite: fold settled pages into summaries, weave missing backlinks, flag contradictions, rebuild the lean index.
One principle resolves where everything lives:
files table — a lean index page that loads at boot, pointing to many small, backlinked pages. The buyer-facing board card, the published briefing, the market read — none of those are the brain. They are renders of the relevant wiki page(s), public-safe views projected outward. The brain stays private and whole; the projections are partial and shareable.
This is also Claude Code's own model — a MEMORY.md index pointing to small topic files — which is precisely how this very session's memory is structured. We adopt it verbatim: a lean index → many small pages, replacing the advisor's current monolithic JOURNAL.md. The files table already keys on path, so index-plus-pages needs zero schema change — only discipline in the schema prompt and a dream that maintains it.
EventStreamStore.readEvents(path, {offset, limit}), SQLite-backed on the DO, served at GET /agents/:name/:id (the same stream the ?offset=&live=sse route reads). The dream's gather-signal phase reads that — the real record of the day — not whatever happens to be in the model's in-context window, and never the internal flue_session_entries table (the undocumented surface whose drift bricked the v6 DOs).files table, exposed as a Flue SandboxApi (makeAdvisorSandboxApi / a new makeMarketSandboxApi) so the runtime's native read/write/edit/grep/glob tools operate on it directly. Consolidation rewrites pages with the native edit tool. Nothing leaves the DO.files row read on demand by a tool call — there is no boot-injection and no ~10 GB-away storage limit. The agent can grep the wiki for just the pages it needs, so a big market wiki costs context tokens only when (and only as much as) it's actually read.| Capability | Advisor (person) | Market (place) |
|---|---|---|
| Nightly dream beat | dispatch only 30 13 fires {kind:'dream'}; the 6-step pass has never been confirmed to run live | none only metrics at 0 11 is scheduled |
| Durable memory | monolithic one JOURNAL.md + IDENTITY.md in the files table | none no learning-state at all |
| Lean index → pages | no single file, no index | no |
| Wiki (backlinked pages) | none | none briefings are standalone one-shots, "grounded ONLY in that record" |
| Yearnings | yes 7 core + onboarding/observed | none |
| Memory / brain skills | yes memory + brain | none |
| Raw sources ingested | reactions, answers, scout finds | Zillow listings, parcels, parcel_facts — Zillow-only today |
| Public projections it writes | board cards, project_read | resting cards, publish_briefing (verdict→Convex), publish_metrics (read→market_reports) |
Interlinking today: zero. Nothing in either agent cross-references anything. Every briefing is authored from a single listing record in isolation. The wiki is net-new for both — but the projection plumbing (cards, briefings, reads) already exists and is reused unchanged, now fed from pages instead of from thin air.
Both agents run the identical four-phase pass. The dispatch mechanism already exists for the advisor and is copied verbatim for the market: a nightly cron fires a named beat method that calls dispatch({agent, id, input:{kind:'dream'}}) into the agent's own session; the runtime renders a <dispatch ... kind:"dream"> signal and the model branches on it.
memory and brain skills. Declare it private — no new homes, no messages.readEvents at agents/<type>/<id>) — the real record, robust to compaction — together with the wiki pages the working beats touched, looking for corrections, new facts, and patterns. (Advisor: the buyer's reactions/answers. Market: what sweep/drain/metrics observed, plus any new raw sources.)brain_write — raise confidence where the day supported a standing question, lower/reframe where it contradicted, resolve what got settled, add ≤1 new. Small defensible moves.pending_resolution rows (the data hygiene, guarded — §6). Sign off with one dated index line tallying what consolidated and what was pruned.files/yearnings tables. So when a session must be wiped, the agent reboots, reads its index, and remembers who it is. The dream doesn't un-brick a session; it makes a wipe survivable.Same machinery, same four phases — two subjects. Crucially, both agents get a wiki, and neither wiki is "comps." Comps is one query against the market's; "what fits this buyer now" is one query against the advisor's.
One household's evolving search. Net-new wiki over an existing brain.
The buyer's reactions and answers; what the scout surfaces; the homes weighed.
Neighborhoods they're weighing · recurring themes · homes considered · open questions — backlinked (a home page links its area + the themes it touches).
"What fits them now?" → the scout reads the wiki to choose what to bring.
Board cards · project_read — rendered from the wiki, public-safe.
A whole market (e.g. North Fork Valley — ~19k parcels). Net-new brain + wiki.
Zillow sweep plus, from day one: county records, Google Maps, Colorado water rights, and local news — broadly where it's cheap, and deeply only where there's signal (§6).
Per listing · per neighborhood · per theme (water, schools, development, what's-moving) · per news event — backlinked. Comps = the edges between listing pages.
Comps · the market read · "what's happening here" — all reads against the same wiki.
Resting cards · publish_briefing (now a render of the listing's page) · publish_metrics read (rendered from the index).
The asymmetry is scale and subject, not "one has a wiki and one has a journal." The market's wiki will dwarf the advisor's and pulls from many sources; the advisor's is intimate and single-household. Same pattern, same dream, same code.
Seeded once at market provisioning at confidence 0.2, status open, source core — the advisor's birth-seed pattern, idempotent on re-provision. These are the standing questions the dream reweights as the wiki grows.
| key | standing question (the market's voice) |
|---|---|
price-drivers | What actually moves price here — acreage, owned water, views, the school zone, walk-to-town? |
moving-areas | Which neighborhoods are selling fast right now, and which are sitting? |
seasonality | How does this market breathe through the year — when do listings and sales peak? |
honest-vs-over | What separates an honestly-priced home from an over-record ask here? |
supply-trajectory | Is inventory tightening or loosening, and at which price bands? |
whats-changing | What's shifting in this place — development, water, jobs, the stories in the local news? |
The market wiki is fed by many sources from day one, not just Zillow. But the load-bearing rule is not "ingest everything" — it's ingest deeply only where there's signal. ~19k parcels × 8 sources is impossible to keep fresh and would hammer egress (the residential-IP rate-limit). So ingest runs in two tiers.
drain, which is already gated to "zeroed in on, no briefing yet" — extend its trigger set to view/save, tune later.Page lifecycle: stub (Tier 1, on sweep) → enriched (Tier 2, on first signal) → maintained (the nightly dream lints & re-indexes — it never triggers Tier-2 en masse). The dream consolidates what signal produced; it doesn't go hunting.
Each source carries a tier (broad market-level vs signal-gated property-level) and a feasibility note grounded in what's reachable today.
| Source | What it gives | Tier | Feasibility |
|---|---|---|---|
| Zillow have | For-sale facts, price, days-on-market, photos | 1 · broad (sweep stub) | Bright Data scrape (market owns it) |
| County records (assessor + clerk) day-one | Real sale prices & dates, owner, assessed value, acreage, zoning, tax | 2 · property (parcels spine already bulk-loaded; sale-history enrich on signal) | Direct Worker fetch (Delta EagleWeb, per egress test) |
| Google Maps / Places day-one | Drive-time to town, distance to schools/trails/grocery, what's nearby | 2 · property (per address) | Maps API (MAPS secret set) |
| Colorado water rights (DWR/CDSS) day-one | Decrees, priority dates, ditch/irrigation rights | 2 · property (per parcel) | CDSS public REST API |
| Local news (DCI, NF Merchant Herald, Montrose) day-one | Headlines; development & civic items; property/street mentions | 1 market-wide + 2 property mentions | RSS/scrape, Browser-Render fallback |
| Civic minutes & agendas (Paonia/Hotchkiss/Delta) day-one | Council/county items — zoning, development, infrastructure | 1 · market | .gov PDFs |
| Schools (CO DOE / GreatSchools) next wave | Ratings, district boundaries | 1 · per neighborhood | CDE open data / scrape |
| Hazard layers (FEMA flood + wildfire) next wave | Flood zone, wildfire risk score | 2 · property (geospatial) | ArcGIS REST (NFHL, CO WRC) |
| USDA soils (NRCS) next wave | Soil class, ag capability, septic suitability | 2 · property | NRCS SDA REST |
| Census/ACS + building permits next wave | Demographics, income, housing stats; new-permit counts | 1 market + 2 property permits | Census API; county permits |
The ingest interface is source-agnostic: a connector declares its kind, its tier, and which wiki pages it enriches. Adding a source — or moving the next wave to day-two — is a new connector, not a re-architecture.
The wiki layers under the existing projection plumbing — it doesn't replace it. Today a briefing is authored "grounded ONLY in that record" and written straight to Convex. With the wiki, the briefing is rendered from the listing's wiki page, which is itself woven from comps-backlinks + parcels + news. Same output table; a brain underneath.
| Today (one-shot) | With the wiki (projection of a page) |
|---|---|
sweep → store_listings → publish_cards (resting cards) | same, plus sweep ingests a stub wiki page per new listing (raw facts, not yet woven) |
drain → read_listing → publish_briefing from the lone record | drain is the Tier-2 trigger (§6): on a zeroed listing it deep-ingests county/water/Maps/news for that property, weaves the wiki page (comps backlinks + parcels), then publish_briefing renders verdict + valueFactors from the page → Convex listings unchanged |
metrics → publish_metrics read from the tiles | same tiles, but the read prose is rendered from the wiki index (the dream's private market take feeds it) |
advisor author_home_card (needs a zpid) | unchanged — the card body is rendered from the advisor's wiki page for that home; zpid dependency stays |
listings.verdict/valueFactors/deepDive shapes, boardCards, and market_reports are untouched. The wiki is purely additive — a private source the existing publishers now render from. publish_briefing's signature doesn't change; only where the agent draws the content does.FOR_SALE listing and never deletes a row a live board card points at.listings where homeStatus != 'FOR_SALE' AND last_seen_at older than 30 days (gone from several sweeps). Active inventory untouched; its wiki page is archived, not deleted.pending_resolution rows whose listing now has a matching parcel_listings link — the resolution already happened.2026-06-26 [dream] wove 6 pages, +14 backlinks; reweighted 3 yearnings; pruned 4 stale, 9 resolved-pending.The yearning store, the files-table sandbox, and the dream's discipline are generic — nothing in them is buyer- or place-specific. Lift them into a new src/brain/ that both agents import; each agent supplies only its own seeds and body prose.
The split is deliberate: src/brain/ holds only what is identical (DDL, CRUD, the sandbox, wiki primitives, the contract prose). Anything that knows it's a person or a place — the seed yearnings, the page taxonomy, what each projection renders — stays in the agent's own folder. Pete's line stays bright: they share the habit and the machinery, never the memory.
Six phases, each independently shippable and testable. The detailed TDD plan (bite-sized tasks) is the follow-on artifact via the writing-plans skill; this is the shape.
Phase A · Shared brain + the index→pages modelsrc/brain/ and the wiki primitivesfiles sandbox, and makeBrainTools() out of src/advisor/; add wiki.ts (page write, backlink, lean-index rebuild) and contract.ts (the 4-phase prose). Re-point advisor imports; advisor tests stay green.JOURNAL.md with a lean index + topic pages; the dream's prune&index maintains it. Test: a seeded brain rebuilds a bounded index from many pages.{kind:'dream'} turn on Joe; confirm pages updated, ≥1 yearning reweighted, index rebuilt, project_read refreshed. Closes the "wired but never run" gap.project_read are rendered from wiki pages. Behavior-preserving: existing card/read shapes unchanged.ensureBrainSchema + seed MARKET_CORE_YEARNINGSfiles + yearnings; seed the 6 place-questions idempotently at provisioning. Add memory + brain skills and the sandbox/tools the market lacks. Tests mirror the advisor's birth-brain suite.ingestSource(kind, tier, payload) interface; each connector declares its tier and which pages it enriches. Zillow is the first; the day-one set follows. Test: a fake connector writes/enriches a page.drainpublish_briefing renders from it. A budget cap bounds Tier-2 calls per pass. Test: a non-signal listing never triggers a county/Maps fetch.dream() + scheduledream() → dispatch({agent:'market', id, input:{kind:'dream'}}); schedule 0 12 * * * under the arm-once guard. Unit test mirrors advisor dream.test.ts.| What | How | Pass criteria |
|---|---|---|
| Shared extract is inert | Full advisor suite after Phase A | All advisor tests green, no behavior diff |
| Index→pages rebuilds lean | Seed many pages, run prune&index | Index bounded; every page reachable; orphans flagged |
| Advisor dream runs E2E | Live trigger on Joe (Phase A) | Pages updated · ≥1 yearning moved · index rebuilt · read refreshed |
| Market brain seeded idempotently | Provision twice; reweight one between | 6 yearnings, no dupes, reweight preserved |
| Deep ingest is gated | Sweep a market; assert no Tier-2 fetch fired | Stubs only; a county/Maps/water call fires only after a signal on that home |
| Briefing renders from page | Drain a zeroed listing with neighbors present | Verdict cites comps backlinks; Convex listings shape unchanged |
| Dream dispatched once | Unit test on market dream() | {agent:'market', id, input:{kind:'dream'}} exactly once |
| Hygiene is safe | SQL unit test on seeded fixture | FOR_SALE kept · 31-day stale pruned · resolved-pending cleared · unresolved kept |
| Market dream runs E2E | Seed beats, trigger live dream | Pages woven · index lean · tally line written · metrics read reflects it next run |
drain gate. (Pete, 2026-06-26.)MEMORY.md model), replacing the monolithic journal. (Pete, 2026-06-26.)| Risk | Mitigation |
|---|---|
| Wiki rewrites the briefing system. Folding the wiki under briefings could destabilize a working path. | The wiki is additive — a private source the existing publishers render from. publish_briefing's signature and the Convex shapes don't change (§6); Phase D is behavior-preserving by construction. |
| "Index bloat" — not actually a risk. (Earlier framing was wrong.) | The index is a files row in the DO's SQLite, read on demand — NOT auto-loaded at boot (that's a Claude Code MEMORY.md property, not ours), and ~10 GB from any storage wall. The agent greps for the pages it needs, so cost is context-tokens-when-read, bounded by a lean catalog. Prune&index keeps the catalog tidy as a nicety, not a necessity. |
| Model over-prunes. Hygiene runs via prompt. | Fixed, unit-tested SQL (Phase E); the prompt invokes, never improvises the WHERE. FOR_SALE + recent rows structurally excluded. |
| Dream can't run on a bricked session. | Out of scope — that's the wipe route. The dream's value is making a future wipe survivable (§4), by living in files/yearnings, not the session log. |
| Deep ingest overwhelms the system. 19k parcels × 8 sources can't be kept fresh and would hammer egress. | The core mitigation, not an afterthought: Tier-2 enrichment is signal-gated (§6) — only attention-bearing homes get a county/Maps/water fetch, with a per-pass budget cap. The long tail stays thin stubs. |
| Messy sources (news scrapes, civic PDFs, paywalls) fail or drift. | Per-connector isolation behind the ingestSource seam: a failing source degrades to "no page enrichment," never breaks the beat. Browser-Render is the fallback for scrape-hostile sites (per the egress playbook). |