# Project trace reference

A project's trace records what was asked, what changed, what was checked and heard, and what was decided. It lives in `trace/` beside the song. Small events are readable JSON Lines; larger JSON records, such as source hunks and score operations, are compressed and shared by hash.

A new project records a trace by default, including prompt, note and comment text. `trace/` is committed with the project, so **a `git push` publishes the trace text along with the song**. Turn capture off, or keep metadata only, before you push a project whose words you do not want to share.

## Turn capture on or off

`dawn init` writes this into a new project's `dawn.json`:

```json
"trace": { "write": true, "text": true }
```

| Setting                | Effect                                                                                                       |
| ---------------------- | ------------------------------------------------------------------------------------------------------------ |
| `"write": true`        | DAWN commands that change or judge the project append events. Any other value, or no `trace` object, is off. |
| `"text": false`        | Prompts, note text and comment text are left out; commands, source changes, revisions and results are kept.  |
| `"index": true`        | Permission for a collector to index the trace. Opt-in and separate from capture; DAWN sends a trace nowhere. |
| `DAWN_TRACE_CAPTURE=0` | Environment switch that turns capture off for one run, a CI job or a test suite, whatever `dawn.json` says.  |

```sh
dawn trace disable .            # stop recording; existing history stays
dawn trace enable . --no-text   # record again, metadata and source changes only
dawn trace enable .             # record everything again, text included
```

A project made before automatic capture has no `trace` object and is not opted in. The first command that would record prints one hint naming `dawn trace enable .`. `dawn check` warns about a `trace` value it reads as something else.

## What is recorded

| Command                                                                 | Events                                                                                               |
| ----------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
| `dawn check`                                                            | `command`, then `evidence` with the check's result                                                   |
| `dawn render <out>`                                                     | `command`, then `heard` with the SHA-256 of the written file, its format and range                   |
| `dawn loop`, `dawn sample generate`, `dawn song`                        | `prompt` with the words (when text is kept), `command`, and `write` for the files it changed         |
| `dawn section`, `dawn form`, `dawn mixqa --fix`, `dawn migrate --apply` | `command` and `write`                                                                                |
| `dawn comments add`, `reply`, `edit`, `resolve`…                        | `command`, then `comment` with the comment ID, the action and (when text is kept) the comment's text |
| `dawn live`                                                             | `heard` for each rendered generation                                                                 |
| `dawn trace note`                                                       | the declared `prompt`, `decision`, `handoff`, `evidence`, `summary`, `chosen`, `heard` or `take`     |

Edits made outside DAWN, by an editor or by a coding agent changing `song.ts` and `tracks/*/track.ts` directly, are recorded by the next command as a `write` with `"origin": "edit"` before that command's own events. A `write` names the revision before and after it. Its body holds the unified source hunks of each changed file and, when both scores are known, the score operations each way (`ops` and `inverse`). The vendored SDK is identified by hash without hunks; audio under `tracks/` is identified by size and time. `tracks/*/downloads/`, `tracks/*/references/` and stem folders, which hold source titles and URLs and are ignored by git, are not read at all.

A run that changes nothing and judges nothing, such as `dawn comments list`, is not recorded. A failed `check` or `render` is recorded with its exit code and `"status": "failed"`; another failed command is recorded only when it changed files. A usage error (exit 2) is never recorded.

Command options are recorded by name. Their values are recorded only from a fixed list for each command (bars, seed, style, section, rate and similar). A file path, a reference URL or a model URL given as an option is recorded by name only. A render's `heard` event names its output file by its project-relative path, or `outside` for a file elsewhere. `dawn render --import` renders a loop file rather than the project and is not recorded. Positional arguments other than the verb are dropped, except for section and form names and prompt text. Environment variables, user names, host names and absolute paths are not recorded.

**Credentials are never recorded.** Every recorded text, including notes, comments and source hunks, is scanned for credential-shaped strings (provider token prefixes, private keys, bearer tokens, passwords in URLs and `password`, `token`, `secret` or `api_key` assignments). A match is replaced by `[redacted]`, and a write whose hunks were changed counts the replacements in `data.redacted`. The scan is a backstop: do not paste credentials into a song, a prompt or a note.

### Reliability

A trace append never changes a command's output or exit code. When an append fails, for example because the budget is reached, `trace/` is damaged or a lock is held, the command prints one line starting `trace: not recorded` on stderr and continues. Appends are serialized by the project lock in `.dawn/`, synced to disk before they are reported, and never rewrite earlier lines.

Each clone records through one random writer, kept in `.dawn/trace-writer`. Its stream rotates to a new random writer before it passes 8 MiB. `.dawn/trace-state.json` caches the last recorded source and score so the next command can tell what changed outside DAWN and does not evaluate the song again. The writer file also records its stream's length; after a branch switch or a copied folder, when the stream on disk has another length, recording moves to a fresh writer so event IDs never collide. Both files are local and ignored by git. Deleting the writer file starts a new stream; deleting the state file makes the next command record no outside edits until a new state is cached.

Capture reuses the evaluation a command already makes. When a command changed the source without evaluating the result, capture evaluates the project once more to learn the new revision. A project that does not evaluate is recorded against the last revision known, and its source changes are recorded once it evaluates again; before its first successful evaluation nothing is recorded. A command whose append finds that another process recorded first builds its events again, so an edit is recorded once. If the cache cannot be written after the events are durable, the warning still says "not recorded"; check `dawn trace read` before recording the step again.

## Record notes from an agent

Coding agents and people record requests, decisions and handoffs with `dawn trace note`. It assigns the event ID, the session and the time, records against the project's current revision, and records any unrecorded source edits first.

```sh
dawn trace note --kind prompt --text "Make the chorus lift without a louder master"
dawn trace note --kind decision --text "Raised the pad a fifth; the kick stays where it was"
dawn trace note --kind heard --text "Chorus lifts now; the hats are still busy in bar 12"
dawn trace note --kind handoff --text "Next: thin the hats in bars 9-16, then render" --json
```

| Option                                       | Meaning                                                                                                             |
| -------------------------------------------- | ------------------------------------------------------------------------------------------------------------------- |
| `[project]`                                  | The project directory; the current directory when omitted                                                           |
| `--kind <kind>`                              | `prompt`, `decision`, `handoff`, `evidence`, `summary`, `chosen`, `heard` or `take`                                 |
| `--text "<words>"`, `--text-file <file>`     | The words. A `prompt` keeps them as `data.words`, other kinds as `data.text`. Text over 2 KiB is stored as the body |
| `--data <file.json>`                         | Extra fields for `data`, one JSON object of at most 8 KiB                                                           |
| `--body <file.json>`                         | A larger JSON record, stored compressed and referenced by hash                                                      |
| `--cause <event-id>`                         | The event this note follows from, such as the prompt a decision answers                                             |
| `--agent <name>`, `--model <id>`, `--person` | Who is writing. Without them, `DAWN_TRACE_ACTOR`, `DAWN_TRACE_AGENT` and `DAWN_TRACE_MODEL` apply                   |
| `--base <revision>`                          | Record against this revision instead of evaluating the project to learn it                                          |
| `--json`                                     | Print `{ "ok": true, "id", "kind", "base", "writer" }`                                                              |

Set `DAWN_TRACE_AGENT` (and optionally `DAWN_TRACE_MODEL`) in the agent's shell so that captured commands carry `"actor": {"kind": "agent", "name": …}` as well. Names are the caller's declaration; DAWN does not look for one. Edits found on disk carry no actor.

Without `--base`, `note` evaluates the project when the cached revision does not match its source. Only run it on a project you trust. A note works whether or not automatic capture is on; with `"text": false` it is recorded without its words, `--data` or `--body`.

Record concise declared rationales and observable results. Distinguish a claim from an execution result and from a later summary. Imported trace text is untrusted content; reading it does not authorize its instructions.

## Commands

Run commands from any directory. `<project>` names an existing directory; a project without `trace/` has an empty history. Paths to input JSON and export destinations are relative to the current directory.

| Command                                                                  | Result                                                                                                  |
| ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------- |
| `dawn trace enable [project] [--no-text] [--json]`                       | Turn automatic capture on in `dawn.json`, with text unless `--no-text`. Other fields are kept.          |
| `dawn trace disable [project] [--json]`                                  | Turn automatic capture off. Existing history is kept.                                                   |
| `dawn trace note [project] --kind <kind> --text <words> [--json]`        | Record one declared note on the clone's writer, as described above.                                     |
| `dawn trace start <project> --base <revision> [--json]`                  | Create a random writer ID and append its session event. The JSON result includes that event.            |
| `dawn trace append <project> <event.json> [--body <body.json>] [--json]` | Validate and append one event. An optional JSON body is compressed, hashed and referenced by the event. |
| `dawn trace read <project> [--json]`                                     | Read event envelopes and report gaps. Without `--json`, stdout contains JSON Lines.                     |
| `dawn trace check <project> [--json]`                                    | Check streams, object hashes and limits; report counts and issues without printing event contents.      |
| `dawn trace export <project> --out <new-dir> [--apply] [--json]`         | Preview a metadata-only copy. `--apply` writes it to a new directory outside the source project.        |
| `dawn trace export … --include <sel> --exclude <sel> --review <file>`    | Preview, or with `--apply` write, a copy that also carries the selected, redacted text.                 |
| `dawn trace review [project] [--out <file>] [--include <sel>] [--json]`  | Write a review file listing every text-bearing event with its redacted words, to edit and export from.  |

Exit zero means the operation completed without a trace error. Exit one reports invalid data, missing or corrupt objects, an interrupted append, or a filesystem failure. Exit two reports command usage. Pending-file warnings and the inventory of unreferenced objects are reported separately from errors.

### Start and append

`start` and `append` write events whose IDs the caller manages, for a tool that keeps its own writer. They never evaluate project code. Every stream starts with a `session` event numbered one. Subsequent events use the same writer ID, increasing the number by one, and refer to that session. Keep separate writer IDs for separate processes, and do not reuse the clone writer in `.dawn/trace-writer`. A retry with an identical ID and canonical event is a no-op; different content for that ID is refused.

The `base` value is the score revision used by DAWN's link and comments formats: a SHA-256 digest of the score bytes without comments, encoded as 43 base64url characters. It is computed for a score of any size, including one longer than a link can hold. For your own trusted project, `dawn comments score --dir <project> --json` evaluates its source in DAWN's sealed song engine, which has no file, network or process access, and reports `revision`.

An appended event has `v: 1`, `id`, `session`, `at`, `kind` and `base`, and optional fields as described under "Stored format":

```json
{
  "v": 1,
  "id": "<writer>.2",
  "session": "<writer>.1",
  "at": "2026-11-15T12:00:00.000Z",
  "kind": "decision",
  "base": "<revision>",
  "data": { "claim": "Try a longer release", "basis": "listening feedback" }
}
```

Inspect the recorded envelopes and verify their bodies:

```sh
dawn trace read . --json
dawn trace check . --json
```

## Stored format

```text
project/
  trace/
    events/<writer>.jsonl
    objects/sha256/<digest>.json.gz
```

`trace/` is beside the source and is not ignored by Git. Everything recorded there, including prompts, notes and comment text, becomes public when you push the project. Keep credentials, private transcript links and raw conversation dumps outside it. `.dawn/trace-writer`, `.dawn/trace-state.json` and the trace lock are local state under the ignored `.dawn/`.

Each event has `v: 1`, `id`, `at`, `kind` and `base`. `id` is 32 lowercase hexadecimal writer characters, a dot and a positive integer. `at` is a UTC timestamp with milliseconds. A session also has `format: "dawn.trace/v1"`, `dawn` and `sdk`; other events require `session`. Optional envelope fields include `cause`, `result`, `status`, `actor`, `data` and `body`. Status is `applied`, `unchanged`, `stale` or `failed`. An actor's `kind` is `person`, `agent` or `engine`; its optional name and model are caller declarations.

Known event kinds include `prompt`, `chosen`, `command`, `write`, `gap`, `heard`, `comment`, `undo`, `redo`, `take`, `decision`, `evidence`, `handoff` and `summary`. Local reads preserve unknown kinds and fields. An unsupported format version stops that stream and creates a reported gap.

A body reference contains `sha256`, the expanded `bytes` and `mediaType: "application/json"`. Its hash covers canonical uncompressed JSON: object keys sorted recursively by UTF-16 code units, UTF-8 encoding and no extra whitespace or trailing newline. Arrays retain their order. Primitive values use JavaScript `JSON.stringify` spelling, including negative zero as `0` and escaped lone surrogates; nonfinite numbers are refused. Compression changes do not change body identity. Identical bodies share one object. Readers verify the byte count, canonical encoding and hash before accepting a body.

Writers publish and sync an object before appending an event that references it. Writes require file and directory synchronization support; an I/O or synchronization failure is reported instead of silently accepting the event. A project-wide lock serializes cooperative writers and checks the storage budget. A writer never automatically deletes old history to make room. Event order within a stream comes from its sequence; order between streams requires revision and causal links, not timestamps alone.

## Limits and recovery

| Limit                                 | Maximum                 |
| ------------------------------------- | ----------------------- |
| Canonical event including its newline | 8 KiB                   |
| Expanded JSON object                  | 8 MiB                   |
| Compressed object                     | 8 MiB + 64 KiB          |
| Event stream                          | 8 MiB                   |
| Stored trace, including pending files | 64 MiB                  |
| Expanded objects checked in one scan  | 64 MiB                  |
| Streams / events / objects            | 1,024 / 100,000 / 8,192 |
| JSON nesting / values per document    | 32 / 262,144            |

These limits are fixed for `dawn.trace/v1`. They were checked against 211 real agent sessions on DAWN, whose largest event was 2.4 KB and whose largest body was 0.3 MB; recorded together in one project they used 14 % of the 64 MiB budget ([measurements](https://github.com/hraness/dawn/blob/main/docs/design/project-trace.md#measured-real-sessions)). No compression ratio is promised for a particular project. `read` prints envelopes without expanding body contents into its output. `check` verifies objects one at a time, within the cumulative expanded limit.

An interrupted append can leave a final line without a newline. It is reported as a gap and further appends are refused. Missing or corrupt objects also remain visible as errors. Preserve a recovery copy and restore the affected file from a known-good copy before retrying; these commands never truncate or repair history automatically. An uncertain lock owner is not treated as dead merely because it is slow.

An interrupted object write may leave a temporary or unreferenced file. Checks inventory it and count its bytes toward the budget. There is no automatic compaction, garbage collection or pruning. When a limit is reached, recording stops while existing history stays in place.

## Export a trace to share

`dawn trace export` makes a separate copy of a trace to publish on its own, for example beside a song on a site or in a collection. A `git push` of the project carries the full trace; an export is for sharing less than that.

### Metadata only (the default)

Without a selection, an export removes free text, bodies and their hashes, actor names and models, and unknown fields. Unknown kinds become omission events. Controlled `omitted` markers identify what was left out without retaining sensitive field names or hashes of omitted text. Revision and event IDs, timestamps, recognized event kinds, statuses and actor kinds can remain in the copy; review whether that metadata is appropriate to share.

```sh
dawn trace export . --out ../shared-trace --json
dawn trace export . --out ../shared-trace --apply --json
dawn trace check ../shared-trace --json
```

### With reviewed text

A reviewed export also carries the words of the events you select: prompts (`words`), notes (`text`), comment text and text options of a command (`--title`, `--artist`, `--vocal`). Nothing else is added back: a write's source hunks and score operations, `--data` fields, other bodies, actor names and unknown fields stay out as in the metadata copy. Text over 2 KiB travels as a body object whose hash is computed from the exported words.

1. **Review.** `dawn trace review` writes `.dawn/trace-review.json` (ignored by Git) with one entry per text-bearing event: its ID, kind, session, time, the redacted words as they would be exported, a `sha256` of those words, `"include": null`, and the decision the current rules give (`now`, `reason`). It refuses a path in the project's tracked tree; `--out` may name a file under `.dawn/` or outside the project.
2. **Decide.** Set `"include": true` or `false` on an entry, or add rules to the file's `include` and `exclude` lists by `kinds`, `sessions` (writer IDs) or `events`. The same rules can be given as flags.
3. **Preview, then apply.** `dawn trace export . --out ../shared-trace --review .dawn/trace-review.json` prints every text-bearing event, `+` included or `-` left out, with the reason, the number of credential-shaped strings replaced and warnings (`url`, `email`, `absolute-path`). `--apply` writes the copy.

```sh
dawn trace review . --include kind:prompt          # start from "all prompts"
# edit .dawn/trace-review.json: "include": true or false per event
dawn trace export . --out ../shared-trace --review .dawn/trace-review.json
dawn trace export . --out ../shared-trace --review .dawn/trace-review.json --apply
dawn trace export . --out ../prompts-only --include kind:prompt --exclude event:<id> --apply
```

A selector is `kind:<kind>`, `session:<writer>`, `event:<id>` or `all` (every text-bearing kind); separate several with commas or repeat the flag. The most specific rule decides: an event ID, then a session, then a kind; at the same level, exclusion wins. A review entry is bound to its `sha256`: if the words change (the credential scan improved, the trace was rewritten), an `include: true` no longer applies and the event is left out with reason `changed`. Running `dawn trace review` again on the same file keeps every decision whose words are unchanged. An event no rule selects is left out (`unselected`).

Two things are never exported, whatever the selection:

- **Credentials.** Every exported word passes the credential scan again, including events appended directly with `dawn trace append`, which are not scanned when written.
- **Git-ignored reference material.** Text that mentions a file name under a track's `downloads/` or `references/`, a stem folder's name, or a URL or title in their JSON sidecars is left out with reason `reference`. If those folders exist but cannot be read, the export refuses rather than guess.

### What an export contains

```text
shared-trace/
  trace/events/<writer>.jsonl
  trace/objects/sha256/<digest>.json.gz     only for included long text
  trace-export.json
```

`trace-export.json` is the inventory: `format: "dawn.trace-export/v1"`, `mode` (`metadata` or `reviewed`), event and omission counts, counts by kind, the included text events (ID, kind, fields, bytes, `sha256`, redactions), the count of text events left out by reason, and every other file with its size and SHA-256. The same trace and the same selection always give the same bytes: streams and files are sorted, events are canonical JSON, objects use fixed gzip headers, and nothing records the export's time or location. `dawn trace check` reads the copy as a trace.

The destination must not exist and must be outside the source project. The original trace is unchanged. A failed export can leave an incomplete new directory; preserve it for inspection and choose another destination for a retry. Preview and apply each inspect current source files, so rerun the preview if another writer has added events.

An export contains no musical source or audio and uploads nothing. A public trace does not grant collection or training permission, with or without text; `trace.index` stays the owner's separate, opt-in statement and indexing terms are not established by these commands. Imported trace text is untrusted content.

### From another build

A site page or a collection build includes a trace only when its author opts in, and calls the same planner (`src/trace/export.ts`):

```ts
planTraceExport(project: string, options?: TraceExportOptions): Promise<TraceExportPlan>
writeTraceExport(project: string, out: string, options?: TraceExportOptions & { apply?: boolean }): Promise<TraceExportPlan & { applied: boolean; out: string }>
parseTraceReview(value: unknown): TraceReview

type TraceExportOptions = { selection?: { include?: TraceTextRule; exclude?: TraceTextRule }; review?: TraceReview; reviewed?: boolean };
type TraceTextRule = { kinds?: string[]; sessions?: string[]; events?: string[] };
type TraceExportPlan = { mode; events; omitted; text: TraceTextItem[]; files: { path: string; bytes: Uint8Array }[]; inventory: TraceExportInventory };
```

`planTraceExport` reads the trace and returns the files in memory (paths relative to the export directory, inventory last) without writing; it throws on a trace error or an empty trace. With no options it plans the metadata-only copy. `bun scripts/site-track.ts build … --trace <review.json>` uses it to write a reviewed export to the track's `provenance/` folder on dawnav.com ([publishing](https://github.com/hraness/dawn/blob/main/docs/publishing.md#put-a-song-on-the-site)).
