mirror of
https://github.com/bendtherules/opencode-plugin-dsml-fix.git
synced 2026-10-17 16:56:50 +00:00
bendtherules
52b7776d5a
Recovers DeepSeek DSML tool calls that leak as assistant text: streaming middleware (AI-SDK path), SSE rewrite (native path), history sanitization, preventive directive, conditional system nudge, and a capped idle watchdog. 116 tests, synthetic golden fixtures.
113 lines
6.7 KiB
Markdown
113 lines
6.7 KiB
Markdown
# Test Matrix
|
||
|
||
One row per parser behaviour. Each row names a fixture (inline string or file under
|
||
`test/fixtures/`) and the assertion. Tests are numbered `V#`/`reject-*` to match
|
||
`docs/PATTERNS.md`.
|
||
|
||
Fixture convention: a **fixture is raw text built at runtime** so we can inject the real
|
||
`U+FF5C` bytes without this repository ever containing them literally; helpers in
|
||
`test/fixtures.ts` provide `m` (marker), `invoke`, `param`, `block`.
|
||
|
||
## Parser: `parseDsml(text)`
|
||
|
||
| ID | Fixture | Expect |
|
||
|----|---------|--------|
|
||
| `v1-canonical` | `block([invoke("bash", [param("command","ls -la")])])` | 1 call `{name:"bash", args:{command:"ls -la"}}`, 1 range covering block |
|
||
| `v2-doubled-bars` | same with doubled bars `m2` | 1 call |
|
||
| `v3-ascii-pipes` | same with `|DSML|` | 1 call |
|
||
| `v4-marker-space` | `< m " calls>"` (space before `calls`) | 1 call |
|
||
| `v5-truncated-tag` | block tag `calls` instead of `tool_calls` | 1 call |
|
||
| `v6-name-spacing` | `invoke name = "bash"` | 1 call |
|
||
| `v7-tool-call-singular` | block tag `tool_call` | 1 call |
|
||
| `v8-function-calls` | block tag `function_calls` | 1 call |
|
||
| `v9-orphan-invoke` | `invoke("edit",[param("path","a.kt")])` with **no** block | 1 call, range covers invoke |
|
||
| `v10-orphan-parameter` | `param("content","x")` only (no invoke) | 1 call named `unknown`? **NO** — see note |
|
||
| `v11-misspelled-closer` | `param("alpha",..)‹BAR›>‹BAR›parameter name="beta"..` | 2 calls: alpha + beta (not swallowed) |
|
||
| `v12-runaway-name` | `invoke("record_item {\\ncategory")` | 0 calls, range strips it |
|
||
| `v13-undeclared-tool` | `invoke("totally_made_up",[param("x","y")])` | 1 call kept |
|
||
| `v14-empty-name` | `invoke("",[param("x","y")])` | 0 calls, no infinite loop |
|
||
| `v15-unclosed` | `block-open + invoke("bash",..)` with no closes | calls from complete invokes only; remainder as text |
|
||
| `v16-truncated-invoke` | two invokes, 2nd missing `</invoke>` | 1 call (first) |
|
||
| `v18-surrounding` | `"before " + block + " after"` | text before/after preserved, block ranged |
|
||
| `v19-multiple` | two invokes in one block + two blocks | 4 calls |
|
||
| `v20-code-fence` | `"```\n" + block + "\n```"` | 0 calls, 0 ranges |
|
||
| `v21-eos` | `block + eosMarker` | eos in ranges |
|
||
| `v22-entities` | param value `a & b "q"` | decoded `a & b "q"` |
|
||
| `v23-multiline-value` | `param` value with `\n` and indent | value preserved, only hug newlines removed |
|
||
| `v24-json-param` | `param` `string="false"` value `{"a":1}` | args[{a:1}] as object |
|
||
| `v25-raw-json-body` | invoke body `{"a":1}` no params | args `{a:1}` |
|
||
| `v26-wrapped-tool` etc. | markerless/markered outer + complete inners | call X recovered; plain prose / incomplete nesting untouched |
|
||
| `reject-no-marker` | `"hello world"` | 0 calls, 0 ranges |
|
||
| `reject-not-dsml` | long prose, no marker | fast path (looksLikeDsml false) |
|
||
| `reject-empty-args` | `invoke("bash",[])` | 0 calls (nothing to run) |
|
||
|
||
**Note on `v10-orphan-parameter`:** a bare `parameter` with no enclosing invoke has no
|
||
tool name, so it cannot become a call. Decision: **strip the noise, emit no call**
|
||
(not `unknown`). Executing an `unknown` tool is worse than dropping an unusable
|
||
fragment. This row therefore expects `calls: []`, `ranges: [the parameter markup]`.
|
||
|
||
## Streaming: `createDsmlStreamMiddleware()`
|
||
|
||
| ID | Fixture | Expect |
|
||
|----|---------|--------|
|
||
| `stream-text-passthrough` | plain text deltas | all `text-delta` emitted, no tool parts |
|
||
| `stream-v2-recovered` | doubled-bar block split per token | `tool-input-start/delta/end` + `tool-call`, no leaked text |
|
||
| `stream-v17-chunk-split` | opener split `<`/`BAR`/`DSML`/… | recovered, no partial marker leaked |
|
||
| `stream-finish-rewrite` | recovered call, source finish `stop` | finish `tool-calls` |
|
||
| `stream-finish-untouched` | no DSML | finish stays `stop` |
|
||
| `stream-unclosed-fallback` | open block, then `text-end` | buffered text emitted verbatim, finish `stop` |
|
||
| `stream-empty-invoke-fallback` | closed block, unparseable | block re-emitted as text, finish `stop` |
|
||
| `stream-surrounding` | `"a " + block + " b"` | text `"a b"`, 1 call |
|
||
| `stream-multiline-edit` | real captured edit shape (DB V9) | 1 `edit` call with correct args |
|
||
| `stream-orphan-invoke` | V9 degradation mid-stream | 1 call |
|
||
| `stream-buffer-cap` | >64 KiB unclosed | fallback to text, no unbounded growth |
|
||
| `stream-eager-stop` | prose opener, no structure | flushed early as text, nothing held |
|
||
| `stream-transition` | reasoning part mid-text | held text flushed first, order preserved |
|
||
|
||
## Repair: `recoverStoredMessage(message)`
|
||
|
||
| ID | Fixture | Expect |
|
||
|----|---------|--------|
|
||
| `repair-strip-only` | assistant msg with native tool-call + leaked text | text stripped, calls **not** appended |
|
||
| `repair-append` | assistant msg with leaked call, no native | text stripped, call appended, `stopReason:"toolUse"` |
|
||
| `repair-thinking` | leaked block in a `thinking` part | thinking stripped |
|
||
| `repair-idempotent` | run twice | second run is a no-op |
|
||
|
||
> Status: the repair *parser* (`parseDsml` + `stripRanges`) is fully tested. The
|
||
> message-level append wrapper is intentionally **not** built: OpenCode persists
|
||
> assistant messages durably and the plugin has no safe write path for them, so
|
||
> repair happens at request time via history sanitization in the `context` hook
|
||
> (`src/index.ts`) instead of mutating stored messages.
|
||
|
||
## Config / modes
|
||
|
||
| ID | Fixture | Expect |
|
||
|----|---------|--------|
|
||
| `config-defaults` | `resolveConfig()` | both Zen providers, relaxed, resume 3 |
|
||
| `config-strict` | `mode:"strict"` | every rescue off |
|
||
| `config-granular` | one rescue off | others stay on |
|
||
| `strict-block` | complete block + strict flags | parses |
|
||
| `strict-orphan` | orphan invoke + strict flags | no call |
|
||
| `prompt-no-marker` | both directives | no live marker bytes, outer-layer rule stated |
|
||
| `fallback-*` | decide/messageText/attempts | nudge policy + cap |
|
||
|
||
## Wrapper: `wrapDsmlLanguageModel`
|
||
|
||
| ID | Fixture | Expect |
|
||
|----|---------|--------|
|
||
| `wrapper-stream` | fake model, block in stream | 1 tool-call, finish `tool-calls` |
|
||
| `wrapper-passthrough` | plain text | untouched |
|
||
| `wrapper-generate` | block in generate text | tool-call content part, finish `tool-calls` |
|
||
|
||
## Sources / provenance
|
||
|
||
Fixtures marked with a source are byte-faithful captures from that source; others are
|
||
constructed from the grammar. Provenance is recorded in `test/fixtures.ts` beside each.
|
||
|
||
- `[DB]` shapes observed in a real long-context DeepSeek session: wrapped
|
||
edit/shell leaks with O(1000)-char multiline values and shell metachars
|
||
- `[vLLM]` vLLM #54686 leak classes (runaway name, mis-closed param, misspelled opener)
|
||
- `[OpenClaw]` openclaw #128858 doubled full-width bars
|
||
- `[Pi]` pi-dsml grammar (mangled ` calls>` variant)
|
||
- `[CS]` CherryStudio #14747 chunk-split SSE sample
|