246 lines
12 KiB
Markdown
246 lines
12 KiB
Markdown
# Dreaming with Jev: a self-improvement method for the pi configuration
|
||
|
||
*RSI project — proposal. Status: design, ready for staged implementation.*
|
||
|
||
## What "dreaming" is
|
||
|
||
Dreaming is a scheduled batch job that replays the agent's past interactions
|
||
and mines them for improvement opportunities, then turns the best ones into
|
||
configuration changes:
|
||
|
||
- new or updated **skills** (`~/.omp/agent/skills/<name>/SKILL.md`)
|
||
- **context file** edits (`~/.omp/agent/AGENTS.md`, `RULES.md`)
|
||
- **system prompt** updates (`APPEND_SYSTEM.md`, `PERSONALITY.md`)
|
||
- new **custom tools** (`~/.omp/agent/tools/*.ts`)
|
||
|
||
Nothing lands directly. Every output is a reviewable artifact — a Gitea PR
|
||
against the config repo and/or a pm-board proposal — and the user merges.
|
||
|
||
## Why Jev, and why now
|
||
|
||
The harness already has the raw material: every persisted session is a JSONL
|
||
file under `~/.omp/agent/sessions/<encoded-cwd>/`, plus prompt history in
|
||
`~/.omp/agent/history.db` (FTS5). The local memory pipeline (`memory.backend:
|
||
local`) already replays changed sessions — but it runs a `default`-role LLM per
|
||
session, caps at 64 sessions per startup, 300 scanned, with 12-hour idle
|
||
gating. It is an *understanding* pipeline, not an *improvement* pipeline.
|
||
|
||
The bottleneck is triage: deciding which parts of which sessions are worth a
|
||
$0.01+ LLM pass. That is a bounded judgment — "is there a durable improvement
|
||
opportunity in this segment, and what kind?" — exactly Jev's shape:
|
||
|
||
- input-only billing at $0.042/MTok, free output
|
||
- 70-500 ms per call, questions in one call evaluated in parallel
|
||
- calibrated probabilities → thresholds you can tune against outcomes
|
||
- every candidate question is a Noul/Score/Choice, no string parsing
|
||
|
||
At those economics the dreamer can screen **everything** (all sessions, all
|
||
segments) every night and escalate ~1-2% to an LLM. That inverts the usual
|
||
sampling-with-heuristics design: no session is skipped because it was cheap to
|
||
look.
|
||
|
||
## Pipeline
|
||
|
||
```
|
||
S0 harvest → S1 segment → S2 jev triage → S3 cluster → S4 LLM synthesis
|
||
↓
|
||
S6 calibration ← S5 human gate (PR + pm-board proposals)
|
||
```
|
||
|
||
### S0 — Harvest
|
||
|
||
Sources:
|
||
|
||
- `~/.omp/agent/sessions/<bucket>/*.jsonl` — parse per the session format:
|
||
fixed-width 256-byte title slot first, then session header, then entries.
|
||
Keep `message` entries (roles are camelCase: `user`, `assistant`,
|
||
`toolResult`); note `toolCall` blocks live inside assistant content.
|
||
Skip or downweight: `compaction`, `branch_summary`, `custom` (opaque),
|
||
entries after `reset_boundary`.
|
||
- `~/.omp/agent/history.db` (`history` table) — bare user prompts are a cheap
|
||
signal of repeated requests.
|
||
- pm-board `.pm/` state — accepted/rejected proposals tell you which past
|
||
suggestions the user actually wanted.
|
||
- Generated skills and `learned.md` under the memory root — what has already
|
||
been learned, so the dreamer proposes deltas, not duplicates.
|
||
|
||
### S1 — Segment
|
||
|
||
Cut each session into units small enough for one Jev call (state ceiling 64k
|
||
tokens; aim ≤4k). Natural cuts: one user intent → the assistant trajectory
|
||
that resolved it (tool calls + results + verdicts). Attach metadata: project
|
||
cwd, date, exit reason (`session_exit` entries), todo outcomes.
|
||
|
||
### S2 — Jev triage (the speculative fan-out)
|
||
|
||
One call per segment; all questions ride the same call:
|
||
|
||
```python
|
||
response = jev.system_one(
|
||
state={
|
||
"segment": seg.text, # redacted (see Safety)
|
||
"meta": {"cwd": seg.cwd, "date": seg.date, "exit": seg.exit_reason},
|
||
},
|
||
questions={
|
||
"friction": Noul("The user had to correct, repeat, or unblock the agent"),
|
||
"repeated": Noul("This is a routine workflow, done by hand, that recurs"),
|
||
"failure": Noul("The attempt failed or needed several retries"),
|
||
"config_gap": Noul("The failure or friction traces to a missing skill, rule, prompt guidance, or tool"),
|
||
"waste": Noul("The path taken was clearly inefficient vs a simpler one"),
|
||
"already_configured": Noul("The behavior that would fix this is already written in the provided current-config summary"),
|
||
"value": Score("How much lasting improvement is available here",
|
||
criteria=["Nothing actionable",
|
||
"Cosmetic preference",
|
||
"Workflow would improve one class of tasks",
|
||
"Recurring failure class would be prevented",
|
||
"Systemic issue affecting many future sessions"]),
|
||
"opportunity": Choice("What kind of change would fix it",
|
||
criteria={"new_skill": "A repeatable workflow deserves its own SKILL.md",
|
||
"rule_update": "A hard rule (RULES.md) or context guidance (AGENTS.md) is missing or wrong",
|
||
"prompt_update": "System-prompt behavior guidance should change (APPEND_SYSTEM.md)",
|
||
"new_tool": "A deterministic helper tool is missing",
|
||
"memory_only": "Worth remembering, not config-worthy",
|
||
"nothing": "No improvement available"}),
|
||
},
|
||
)
|
||
```
|
||
|
||
Pass a compact summary of the *current* config (skill names + descriptions,
|
||
rules titles, APPEND_SYSTEM.md) in state so `already_configured` can veto
|
||
duplicate proposals — this is the "context rot" safeguard: send only the fields
|
||
each question needs.
|
||
|
||
**Gating:** proceed only where `config_right_side` confidence is real:
|
||
|
||
```python
|
||
keep = (a["config_gap"].noul > 0.7 and a["already_configured"].noul < 0.3
|
||
and a["value"].score >= 2.0 and a["value"].confidence > 0.6)
|
||
```
|
||
|
||
Flat distributions ("value" confidence < 0.5) are a signal the criteria are
|
||
wrong for that segment — log them, do not guess. Thresholds live in one config
|
||
file; the calibration loop (S6) tunes them.
|
||
|
||
### S3 — Cluster and dedupe
|
||
|
||
Ten segments about git-push friction are one proposal, not ten. Group by
|
||
`opportunity_type`, then similarity over segment embeddings (or a second,
|
||
tiny Jev Choice over candidate groups — still bounded, still cheap). Keep the
|
||
top cluster exemplars (highest `value` score), cap proposals per dream run
|
||
(default 5) to bound blast radius.
|
||
|
||
### S4 — Synthesis (this is where the LLM earns its price)
|
||
|
||
Jev cannot write. Hand each surviving cluster — evidence links (session ids,
|
||
timestamps, the exact friction quotes) plus the relevant current-config files —
|
||
to a harness run (`omp -p --no-session`, the same pattern the pm-address timer
|
||
uses to reconcile boards). The LLM drafts one artifact per cluster:
|
||
|
||
| Cluster verdict | Artifact it drafts |
|
||
| --- | --- |
|
||
| `new_skill` | `<name>/SKILL.md` with `name` + `description` frontmatter (description required for native discovery), body as a workflow with evidence-linked examples |
|
||
| `rule_update` | diff for `~/.omp/agent/RULES.md` (hard rules only — short) or `AGENTS.md` (durable background) |
|
||
| `prompt_update` | diff for `APPEND_SYSTEM.md` |
|
||
| `new_tool` | `.omp/tools/<name>.ts` custom-tool module sketch (factory + Zod schema), marked needs-implementation |
|
||
|
||
### S5 — Human gate
|
||
|
||
Artifacts are written to a dream workspace branch:
|
||
|
||
- one Gitea PR per dream run against the config repo (per the standing branch
|
||
workflow: never commit to main; the user merges), and
|
||
- pm-board proposals in `.pm/proposals/*.md` (frontmatter `op:`/`id:`/
|
||
`section:`/`title:`) for board-tracked items.
|
||
|
||
The user's accept/reject is the ground-truth label for S6.
|
||
|
||
### S6 — Calibration loop
|
||
|
||
Log every verdict with its probability and its eventual outcome:
|
||
|
||
```json
|
||
{"segment": "...", "config_gap_noul": 0.83, "value_score": 2.4,
|
||
"cluster": "git-push-friction", "outcome": "accepted", "artifact": "PR #12"}
|
||
```
|
||
|
||
Then measure, per question, whether the 0.8 bucket really lands ~80% accepted.
|
||
This is RLCD's philosophy applied to the dreamer itself: confidence must track
|
||
outcomes, and drift in that curve is the alarm that thresholds (or the criteria
|
||
wording — Jev reads literally) need re-tuning. Calibration data is also what
|
||
justifies loosening the human gate later, if ever.
|
||
|
||
## Wiring into this harness
|
||
|
||
- **Scheduler:** mirror the pm-address pattern (hourly `omp -p --no-session`
|
||
reconciliation in `deploy/README.md`) — a nightly `dream` timer/cron that
|
||
invokes the pipeline, then a synthesis `omp` run over the shortlist.
|
||
- **Secrets:** the memory pipeline redacts secret patterns before writing
|
||
memory artifacts; the dreamer must do the same *before* anything leaves the
|
||
machine as Jev state, and re-redact before synthesis.
|
||
- **Overlap with memory:** memory consolidation produces knowledge summaries;
|
||
dreaming produces config changes. They share S0. If `memory.backend: local`
|
||
is enabled, reuse its per-session extraction cadence markers instead of
|
||
re-parsing unchanged sessions (the pipeline tracks processed-since mtime).
|
||
- **Where outputs land:** skills into `~/.omp/agent/skills/<name>/` (native
|
||
provider, priority 100; beats same-named lower providers), rules per the
|
||
RULES.md contract (short, sticky, native locations only), tools into
|
||
`~/.omp/agent/tools/`.
|
||
|
||
## Cost model
|
||
|
||
Per dream run (rough, at self-reported prices):
|
||
|
||
| Stage | Volume | Cost |
|
||
| --- | --- | --- |
|
||
| S2 Jev: 300 sessions → ~1,200 segments x ~4k tokens | ~5 MTok input | **~$0.21** |
|
||
| S4 LLM synthesis: ~5 clusters x ~15k tokens | ~75k tok | cents–$1 |
|
||
| S0-S3, S6 code + SQLite | — | negligible |
|
||
|
||
A full-history dream costs well under $1. The LLM synthesis spend is the
|
||
larger line item and scales with proposal count, which S3 caps.
|
||
|
||
## Safety: this is self-modification, treat it like it
|
||
|
||
1. **Human gate.** No direct writes to live config. PRs + proposals only; the
|
||
user merges. (Also required by the house rule: no commits to main.)
|
||
2. **Blast-radius cap.** ≤5 proposals per run, each with evidence links; a
|
||
proposal without a session citation is rejected by construction.
|
||
3. **Injection.** Sessions contain untrusted content (web pages, issue text)
|
||
and "state is not hostile" applies — content can argue for its own
|
||
improvement. Mitigations: redaction before Jev sees it, the LLM synthesizer
|
||
independently reviews evidence, and the human gate sees the evidence too.
|
||
4. **Feedback loops.** The dreamer must not grade its own artifacts as
|
||
"accepted" — outcomes come from the human's merge/reject actions only.
|
||
5. **Reversibility.** Config lives in git; every accepted proposal is a
|
||
reviewable commit; `RULES.md` stays short so the sticky layer can't bloat.
|
||
|
||
## Implementation notes
|
||
|
||
- Adapter interface first: `JevClient` with three backends — `jev` (real API,
|
||
needs `TYPESAFE_API_KEY` early access, or Vercel AI gateway), `llm` (a
|
||
frontier LLM constrained to the same schema, e.g. via TypeSafe's published
|
||
`system-one-adapter-python` shim), and `mock` (deterministic, for tests).
|
||
The interface is the design; the backend is swappable, so the pipeline can
|
||
be built and audited before the key exists.
|
||
- Pin `jev-1.13.0` in config, log the `model` field from every response.
|
||
- Dry-run mode writes an audit report (what would have been proposed, with
|
||
Jev's raw probabilities) and nothing else.
|
||
|
||
## Rollout
|
||
|
||
| Milestone | Gate |
|
||
| --- | --- |
|
||
| M0: parser + S2 dry run over last 30 days, human reads the audit report | report is sensibly targeted (eyeball) |
|
||
| M1: full pipeline, thresholds strict (`config_gap > 0.8`), ≤3 proposals/run | user accepts ≥1 real artifact |
|
||
| M2: weekly cadence, thresholds tuned from calibration data | calibration curve within ±15pp of diagonal |
|
||
|
||
## Open questions
|
||
|
||
- Does Jev's calibration hold on *this* corpus (session transcripts are
|
||
long, messy, and self-authored)? M0 exists to answer this before anything
|
||
acts.
|
||
- Cluster-level evidence may need a second Jev pass with the current config in
|
||
state (serial call, justified dependency per the primitives' independence
|
||
rule).
|
||
- Whether S4 synthesis should also run the doc-02 "config-drift guard" on its
|
||
own drafts before proposing them. |