bendtherules/opencode-multi-model
1
0
Fork 0
mirror of https://github.com/bendtherules/opencode-multi-model.git synced 2026-08-18 21:52:11 +00:00
Code Issues Packages Projects Releases Wiki Activity

plan: testcases - add helper cases and mark dones

Browse Source
This commit is contained in:
bendtherules
2026-03-13 01:05:22 +05:30
parent b756508c27
commit 9d950a74e8
1 changed files with 82 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.opencode/plans/1773342290743-silent-harbor.md
+82 -42
View File
@@ -20,56 +20,96 @@ A key requirement is to **log all mocks that were actually called and verify tha
### A. Export & Test Helper Functions
To ensure robust unit coverage, we will export the internal helper functions from `multi-model.ts` without cluttering the main export.
* In `multi-model.ts`, append an export for `__testing_helpers` containing functions like:
* `shellQuote`
* `normalizeModels`
* `findDuplicates`
* `createWindowBaseName`
* `createWindowPlans`
* `levenshtein`
* `suggestModels`
* `formatInvalidModelError`
* `sanitizeName`
* In `multi-model.ts`, append an export for `__testing_helpers` containing functions.
* In `multi-model.test.ts`, import `__testing_helpers` from `./tools/multi-model`.
* Add comprehensive unit test suites (`describe("__testing_helpers", ...)`) for each of these functions covering edge cases (e.g. empty inputs, special characters, fuzzy match exact bounds, complex naming collisions).
* Add comprehensive unit test suites (`describe("__testing_helpers", ...)`) for each function covering:
* `shellQuote`:
* Normal string without special characters.
* String with spaces.
* String with single quotes (verifying `'"'"'` replacement, e.g., `test'value`).
* String with multiple single quotes.
* Empty string.
* `normalizeModels`:
* Empty array.
* `undefined` input.
* Array with whitespace-only strings.
* Array with mixed valid models, padded models, and empty strings (verifying trimming and filtering).
* `findDuplicates`:
* Array with no duplicates.
* Array with one duplicate pair.
* Array with multiple different duplicates.
* Array with a single value repeated more than twice (should only return it once).
* Verifying first-repeated-occurrence order.
* `createWindowBaseName`:
* Normal model name without slashes.
* Model with slashes (verifying extraction of the last part, e.g., `vendor/namespace/model`).
* Model exceeding `WINDOW_NAME_LIMIT` (24 chars) (verifying truncation).
* Model with special characters (verifying lowercase, hyphen replacement, and trimming of leading/trailing hyphens).
* Model made entirely of special characters (fallback to `"model"`).
* `createWindowPlans`:
* Single model (verifying original base name).
* Two models with the same base name (verifying `-2` suffix for the second).
* Multiple collisions (verifying `-3`, `-4`, etc.).
* Verifying truncation of the base name to accommodate the suffix without exceeding `WINDOW_NAME_LIMIT` (e.g., base name length 24 + suffix length 2 -> base name truncated to 22).
* `levenshtein`:
* Identical strings (distance 0).
* One substitution, insertion, or deletion (distance 1).
* Completely different strings.
* One empty string, both empty strings.
* `suggestModels`:
* Exact match (distance 0).
* Fuzzy match/typo (e.g., `gpt5.4` matching `gpt-5.4`).
* Fuzzy match with contains boost (e.g., `gpt4o` requesting `gpt-4o` vs `gpt-4o-mini`).
* Verifying maximum of 3 suggestions returned.
* Sorting logic (score first, then alphabetical fallback).
* `formatInvalidModelError`:
* Single invalid model with available suggestions.
* Single invalid model without suggestions (fallback message to run `opencode models`).
* Multiple invalid models (verifying formatting of comma-separated list and suggestions based on the first invalid model).
* `sanitizeName`:
* Normal alphanumeric name.
* Name with spaces and special characters (verifying replacement with hyphens).
* Name with consecutive hyphens (verifying deduplication).
* Name with leading/trailing hyphens (verifying trimming).
### B. Validation & Pre-flight Checks (Early Returns)
These tests verify that the tool bails out early and doesn't execute unnecessary commands.
* **Empty session name:** Fails validation. `executedCommands` should be `[]`.
* **Empty models array:** Fails validation. `executedCommands` should be `[]`.
* **Duplicate models:** Fails validation. `executedCommands` should be `[]`.
* **Not a git repo:** Mocks `git rev-parse --is-inside-work-tree` to fail.
* **Missing `tmux`:** Mocks `command -v tmux` to fail.
* **Missing `opencode`:** Mocks `command -v opencode` to fail.
* **`opencode models` failure:** Mocks `opencode models` to fail.
* **Invalid model IDs (Fuzzy Matching):**
* Mocks `opencode models` to return a specific list.
* Test with an invalid model that is close to a valid one (verifies "Did you mean?" suggestions).
* Test with multiple invalid models.
* **Tmux session already exists:** Mocks `tmux has-session -t <session>` to succeed (return exit code 0).
These tests verify that the tool bails out early and doesn't execute unnecessary commands. Note: Existing tests marked as `[x]` will be updated to also assert on `executedCommands` being `[]`.
* [x] **Empty session name:** Fails validation. `executedCommands` should be `[]`.
* [x] **Empty models array:** Fails validation. `executedCommands` should be `[]`.
* [x] **Duplicate models:** Fails validation. `executedCommands` should be `[]`.
* [x] **Not a git repo:** Mocks `git rev-parse --is-inside-work-tree` to fail.
* [x] **Missing `tmux`:** Mocks `command -v tmux` to fail.
* [x] **Missing `opencode`:** Mocks `command -v opencode` to fail.
* [ ] **`opencode models` failure:** Mocks `opencode models` to fail.
* [x] **Invalid model IDs (Fuzzy Matching):**
* [x] Mocks `opencode models` to return a specific list.
* [ ] Test with an invalid model that is close to a valid one (verifies "Did you mean?" suggestions).
* [ ] Test with multiple invalid models.
* [x] **Tmux session already exists:** Mocks `tmux has-session -t <session>` to succeed (return exit code 0).
### B. First Model Failures (isFirst === true)
### C. First Model Failures (isFirst === true)
The first model sets up the primary tmux session. If it fails, it returns a string immediately instead of collecting errors in `failedModels`.
* **Worktree path exists:** Mock `fs.existsSync` to return `true`. Ensure it fails and `executedCommands` stops after checking tmux session.
* **Git branch exists:** Mock `git show-ref --verify --quiet refs/heads/...` to return exit code 0.
* **Git worktree add fails:** Mock `git worktree add -b ...` to fail.
* **Tmux new-session fails:** Mock `tmux new-session ...` to fail. Verify that `undoWorktree` (`git worktree remove` and `git branch -D`) is called and appended to `executedCommands`.
* **Tmux send-keys fails:** Mock `tmux send-keys ...` to fail. Verify `undoWorktree` commands are called.
* [ ] **Worktree path exists:** Mock `fs.existsSync` to return `true`. Ensure it fails and `executedCommands` stops after checking tmux session.
* [ ] **Git branch exists:** Mock `git show-ref --verify --quiet refs/heads/...` to return exit code 0.
* [ ] **Git worktree add fails:** Mock `git worktree add -b ...` to fail.
* [ ] **Tmux new-session fails:** Mock `tmux new-session ...` to fail. Verify that `undoWorktree` (`git worktree remove` and `git branch -D`) is called and appended to `executedCommands`.
* [ ] **Tmux send-keys fails:** Mock `tmux send-keys ...` to fail. Verify `undoWorktree` commands are called.
### C. Subsequent Model Failures (isFirst === false)
### D. Subsequent Model Failures (isFirst === false)
Failures on subsequent models do not abort the process; they append to `failedModels`.
* **Worktree path exists:** Mock `fs.existsSync` to return `true` only for the second model's path. Verify first model succeeds, second model is skipped, and result output indicates partial success/failure.
* **Git branch exists:** Mock `git show-ref` to succeed for the second model. Verify it continues and records the failure.
* **Git worktree add fails:** Mock `git worktree add` to fail for the second model.
* **Tmux new-window fails:** Mock `tmux new-window` to fail. Verify `undoWorktree` commands are executed for the second model's branch/path.
* **Tmux send-keys fails:** Mock `tmux send-keys` to fail for the second model. Verify `undoWorktree` commands are executed.
* [ ] **Worktree path exists:** Mock `fs.existsSync` to return `true` only for the second model's path. Verify first model succeeds, second model is skipped, and result output indicates partial success/failure.
* [ ] **Git branch exists:** Mock `git show-ref` to succeed for the second model. Verify it continues and records the failure.
* [ ] **Git worktree add fails:** Mock `git worktree add` to fail for the second model.
* [ ] **Tmux new-window fails:** Mock `tmux new-window` to fail. Verify `undoWorktree` commands are executed for the second model's branch/path.
* [ ] **Tmux send-keys fails:** Mock `tmux send-keys` to fail for the second model. Verify `undoWorktree` commands are executed.
### D. Success Cases & Complex Window Names
* **Single model success:** Standard happy path. Verify exact command sequence.
* **Multiple models success:** Verify `tmux new-session` is called for the first, and `tmux new-window` for the rest.
* **Sanitization:** Pass a session name with spaces and special chars. Verify the safe name is used in git branch and worktree path commands.
* **Model name collision:** Pass models like `openai/gpt-4o` and `anthropic/gpt-4o`. Verify the window names in the commands are `gpt-4o` and `gpt-4o-2` respectively.
* **Long model name:** Pass a model name that evaluates to > 24 chars for the base window name. Verify truncation in the executed `tmux` commands.
### E. Success Cases & Complex Window Names
Note: Existing tests marked as `[x]` will be updated to assert on the exact `executedCommands` sequence.
* [x] **Single model success:** Standard happy path. Verify exact command sequence.
* [x] **Multiple models success:** Verify `tmux new-session` is called for the first, and `tmux new-window` for the rest.
* [x] **Sanitization:** Pass a session name with spaces and special chars. Verify the safe name is used in git branch and worktree path commands.
* [ ] **Model name collision:** Pass models like `openai/gpt-4o` and `anthropic/gpt-4o`. Verify the window names in the commands are `gpt-4o` and `gpt-4o-2` respectively.
* [ ] **Long model name:** Pass a model name that evaluates to > 24 chars for the base window name. Verify truncation in the executed `tmux` commands.
3. **Refactor Existing Unit Tests:**
- Update the "unit tests" at the bottom of the existing file to correctly test `__testing_helpers.shellQuote` and `__testing_helpers.normalizeModels` instead of redefining the functions. Expand these unit tests to cover all edge cases mapped out in section A.