From b756508c276fd3d7e7816f096cfa83441c47fb38 Mon Sep 17 00:00:00 2001 From: bendtherules Date: Fri, 13 Mar 2026 00:52:22 +0530 Subject: [PATCH] plan: new testcases --- .../plans/1773342290743-silent-harbor.md | 141 ++++++++++++++++++ 1 file changed, 141 insertions(+) create mode 100644 .opencode/plans/1773342290743-silent-harbor.md diff --git a/.opencode/plans/1773342290743-silent-harbor.md b/.opencode/plans/1773342290743-silent-harbor.md new file mode 100644 index 0000000..1f04dae --- /dev/null +++ b/.opencode/plans/1773342290743-silent-harbor.md @@ -0,0 +1,141 @@ +--- +model: gemini-3.1-pro-preview +--- + +# Plan: Comprehensive Tests for `multi-model` Tool + +## 1. Overview +The goal is to provide complete test coverage for `@.opencode/tools/multi-model.ts` by updating the `@.opencode/multi-model.test.ts` file. Since internal helper functions (like `shellQuote`, `createWindowPlans`, etc.) are not exported, we will test all logic through the `execute` method by mocking shell commands (`Bun.$`). + +A key requirement is to **log all mocks that were actually called and verify that the sequence matches** expectations. + +## 2. Approach + +1. **Enhance the Mock Infrastructure:** + - Update `setupMockBun$` in `multi-model.test.ts` to push each executed command to an `executedCommands` array. + - Clear `executedCommands` in `beforeEach`. + - For every test case, use `expect(executedCommands).toEqual([...])` or `expect(executedCommands).toStrictEqual([...])` to assert the exact sequence of shell commands executed. + +2. **Define Test Cases:** + + ### A. Export & Test Helper Functions + To ensure robust unit coverage, we will export the internal helper functions from `multi-model.ts` without cluttering the main export. + * In `multi-model.ts`, append an export for `__testing_helpers` containing functions like: + * `shellQuote` + * `normalizeModels` + * `findDuplicates` + * `createWindowBaseName` + * `createWindowPlans` + * `levenshtein` + * `suggestModels` + * `formatInvalidModelError` + * `sanitizeName` + * In `multi-model.test.ts`, import `__testing_helpers` from `./tools/multi-model`. + * Add comprehensive unit test suites (`describe("__testing_helpers", ...)`) for each of these functions covering edge cases (e.g. empty inputs, special characters, fuzzy match exact bounds, complex naming collisions). + + ### B. Validation & Pre-flight Checks (Early Returns) + These tests verify that the tool bails out early and doesn't execute unnecessary commands. + * **Empty session name:** Fails validation. `executedCommands` should be `[]`. + * **Empty models array:** Fails validation. `executedCommands` should be `[]`. + * **Duplicate models:** Fails validation. `executedCommands` should be `[]`. + * **Not a git repo:** Mocks `git rev-parse --is-inside-work-tree` to fail. + * **Missing `tmux`:** Mocks `command -v tmux` to fail. + * **Missing `opencode`:** Mocks `command -v opencode` to fail. + * **`opencode models` failure:** Mocks `opencode models` to fail. + * **Invalid model IDs (Fuzzy Matching):** + * Mocks `opencode models` to return a specific list. + * Test with an invalid model that is close to a valid one (verifies "Did you mean?" suggestions). + * Test with multiple invalid models. + * **Tmux session already exists:** Mocks `tmux has-session -t ` to succeed (return exit code 0). + + ### B. First Model Failures (isFirst === true) + The first model sets up the primary tmux session. If it fails, it returns a string immediately instead of collecting errors in `failedModels`. + * **Worktree path exists:** Mock `fs.existsSync` to return `true`. Ensure it fails and `executedCommands` stops after checking tmux session. + * **Git branch exists:** Mock `git show-ref --verify --quiet refs/heads/...` to return exit code 0. + * **Git worktree add fails:** Mock `git worktree add -b ...` to fail. + * **Tmux new-session fails:** Mock `tmux new-session ...` to fail. Verify that `undoWorktree` (`git worktree remove` and `git branch -D`) is called and appended to `executedCommands`. + * **Tmux send-keys fails:** Mock `tmux send-keys ...` to fail. Verify `undoWorktree` commands are called. + + ### C. Subsequent Model Failures (isFirst === false) + Failures on subsequent models do not abort the process; they append to `failedModels`. + * **Worktree path exists:** Mock `fs.existsSync` to return `true` only for the second model's path. Verify first model succeeds, second model is skipped, and result output indicates partial success/failure. + * **Git branch exists:** Mock `git show-ref` to succeed for the second model. Verify it continues and records the failure. + * **Git worktree add fails:** Mock `git worktree add` to fail for the second model. + * **Tmux new-window fails:** Mock `tmux new-window` to fail. Verify `undoWorktree` commands are executed for the second model's branch/path. + * **Tmux send-keys fails:** Mock `tmux send-keys` to fail for the second model. Verify `undoWorktree` commands are executed. + + ### D. Success Cases & Complex Window Names + * **Single model success:** Standard happy path. Verify exact command sequence. + * **Multiple models success:** Verify `tmux new-session` is called for the first, and `tmux new-window` for the rest. + * **Sanitization:** Pass a session name with spaces and special chars. Verify the safe name is used in git branch and worktree path commands. + * **Model name collision:** Pass models like `openai/gpt-4o` and `anthropic/gpt-4o`. Verify the window names in the commands are `gpt-4o` and `gpt-4o-2` respectively. + * **Long model name:** Pass a model name that evaluates to > 24 chars for the base window name. Verify truncation in the executed `tmux` commands. + +3. **Refactor Existing Unit Tests:** + - Update the "unit tests" at the bottom of the existing file to correctly test `__testing_helpers.shellQuote` and `__testing_helpers.normalizeModels` instead of redefining the functions. Expand these unit tests to cover all edge cases mapped out in section A. + +## 3. Implementation Details + +We will modify `setupMockBun$` in `.opencode/multi-model.test.ts`: + +```typescript +let executedCommands: string[] = []; + +function setupMockBun$() { + const mockFn = mock((strings: TemplateStringsArray, ...values: any[]) => { + // Interleave strings and values to reconstruct the command + let commandSignature = strings[0]; + for (let i = 0; i < values.length; i++) { + commandSignature += String(values[i]) + strings[i + 1]; + } + commandSignature = commandSignature.trim().replace(/\s+/g, ' '); + + executedCommands.push(commandSignature); + + // ... (rest of the existing mock logic to return based on mockCommandResponses) + }); + // ... +} +``` + +In `beforeEach`: +```typescript +executedCommands = []; +// ... setup default mockCommandResponses for happy path up to tmux has-session ... +``` + +Example test case asserting on `executedCommands`: +```typescript +test("fails if tmux new-session fails for the first model and triggers undo", async () => { + mockCommandResponses["tmux new-session -d -s test-session -n gpt-4o -c /mock/home/.local/share/opencode/multi-model/test-session/gpt-4o"] = { ok: false, stdout: "", stderr: "tmux error" }; + + // mock undo command success + mockCommandResponses["git worktree remove -f /mock/home/.local/share/opencode/multi-model/test-session/gpt-4o"] = { ok: true, stdout: "", stderr: "" }; + mockCommandResponses["git branch -D opencode/test-session/gpt-4o"] = { ok: true, stdout: "", stderr: "" }; + + const result = await multiModelTool.execute( + { sessionName: "test-session", models: ["openai/gpt-4o"] }, + { metadata: mock() } as any + ); + + expect(result).toContain("Failed to create tmux session"); + + // Verify exact sequence + expect(executedCommands).toEqual([ + "git rev-parse --is-inside-work-tree", + "command -v tmux", + "command -v opencode", + "opencode models", + "tmux has-session -t test-session", + "git show-ref --verify --quiet refs/heads/opencode/test-session/gpt-4o", + "git worktree add -b opencode/test-session/gpt-4o /mock/home/.local/share/opencode/multi-model/test-session/gpt-4o", + "tmux new-session -d -s test-session -n gpt-4o -c /mock/home/.local/share/opencode/multi-model/test-session/gpt-4o", + // Because it failed, it must run cleanup + "git worktree remove -f /mock/home/.local/share/opencode/multi-model/test-session/gpt-4o", + "git branch -D opencode/test-session/gpt-4o" + ]); +}); +``` + +## 4. Next Steps +Once this plan is approved, I will implement all the described test cases in `@.opencode/multi-model.test.ts`, ensuring that `executedCommands` accurately captures and verifies every step of the orchestration logic.