bendtherules/opencode-multi-model
1
0
Fork 0
mirror of https://github.com/bendtherules/opencode-multi-model.git synced 2026-08-18 21:52:11 +00:00
Code Issues Packages Projects Releases Wiki Activity

plan: testcases - add helper cases and mark dones

Browse Source
This commit is contained in:
bendtherules
2026-03-13 01:05:22 +05:30
parent b756508c27
commit 9d950a74e8
1 changed files with 82 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.opencode/plans/1773342290743-silent-harbor.md
+82 -42
View File
@@ -20,56 +20,96 @@ A key requirement is to **log all mocks that were actually called and verify tha
### A. Export & Test Helper Functions ### A. Export & Test Helper Functions
To ensure robust unit coverage, we will export the internal helper functions from `multi-model.ts` without cluttering the main export. To ensure robust unit coverage, we will export the internal helper functions from `multi-model.ts` without cluttering the main export.
* In `multi-model.ts`, append an export for `__testing_helpers` containing functions like: * In `multi-model.ts`, append an export for `__testing_helpers` containing functions.
* `shellQuote`
* `normalizeModels`
* `findDuplicates`
* `createWindowBaseName`
* `createWindowPlans`
* `levenshtein`
* `suggestModels`
* `formatInvalidModelError`
* `sanitizeName`
* In `multi-model.test.ts`, import `__testing_helpers` from `./tools/multi-model`. * In `multi-model.test.ts`, import `__testing_helpers` from `./tools/multi-model`.
* Add comprehensive unit test suites (`describe("__testing_helpers", ...)`) for each of these functions covering edge cases (e.g. empty inputs, special characters, fuzzy match exact bounds, complex naming collisions). * Add comprehensive unit test suites (`describe("__testing_helpers", ...)`) for each function covering:
* `shellQuote`:
* Normal string without special characters.
* String with spaces.
* String with single quotes (verifying `'"'"'` replacement, e.g., `test'value`).
* String with multiple single quotes.
* Empty string.
* `normalizeModels`:
* Empty array.
* `undefined` input.
* Array with whitespace-only strings.
* Array with mixed valid models, padded models, and empty strings (verifying trimming and filtering).
* `findDuplicates`:
* Array with no duplicates.
* Array with one duplicate pair.
* Array with multiple different duplicates.
* Array with a single value repeated more than twice (should only return it once).
* Verifying first-repeated-occurrence order.
* `createWindowBaseName`:
* Normal model name without slashes.
* Model with slashes (verifying extraction of the last part, e.g., `vendor/namespace/model`).
* Model exceeding `WINDOW_NAME_LIMIT` (24 chars) (verifying truncation).
* Model with special characters (verifying lowercase, hyphen replacement, and trimming of leading/trailing hyphens).
* Model made entirely of special characters (fallback to `"model"`).
* `createWindowPlans`:
* Single model (verifying original base name).
* Two models with the same base name (verifying `-2` suffix for the second).
* Multiple collisions (verifying `-3`, `-4`, etc.).
* Verifying truncation of the base name to accommodate the suffix without exceeding `WINDOW_NAME_LIMIT` (e.g., base name length 24 + suffix length 2 -> base name truncated to 22).
* `levenshtein`:
* Identical strings (distance 0).
* One substitution, insertion, or deletion (distance 1).
* Completely different strings.
* One empty string, both empty strings.
* `suggestModels`:
* Exact match (distance 0).
* Fuzzy match/typo (e.g., `gpt5.4` matching `gpt-5.4`).
* Fuzzy match with contains boost (e.g., `gpt4o` requesting `gpt-4o` vs `gpt-4o-mini`).
* Verifying maximum of 3 suggestions returned.
* Sorting logic (score first, then alphabetical fallback).
* `formatInvalidModelError`:
* Single invalid model with available suggestions.
* Single invalid model without suggestions (fallback message to run `opencode models`).
* Multiple invalid models (verifying formatting of comma-separated list and suggestions based on the first invalid model).
* `sanitizeName`:
* Normal alphanumeric name.
* Name with spaces and special characters (verifying replacement with hyphens).
* Name with consecutive hyphens (verifying deduplication).
* Name with leading/trailing hyphens (verifying trimming).
### B. Validation & Pre-flight Checks (Early Returns) ### B. Validation & Pre-flight Checks (Early Returns)
These tests verify that the tool bails out early and doesn't execute unnecessary commands. These tests verify that the tool bails out early and doesn't execute unnecessary commands. Note: Existing tests marked as `[x]` will be updated to also assert on `executedCommands` being `[]`.
* **Empty session name:** Fails validation. `executedCommands` should be `[]`. * [x] **Empty session name:** Fails validation. `executedCommands` should be `[]`.
* **Empty models array:** Fails validation. `executedCommands` should be `[]`. * [x] **Empty models array:** Fails validation. `executedCommands` should be `[]`.
* **Duplicate models:** Fails validation. `executedCommands` should be `[]`. * [x] **Duplicate models:** Fails validation. `executedCommands` should be `[]`.
* **Not a git repo:** Mocks `git rev-parse --is-inside-work-tree` to fail. * [x] **Not a git repo:** Mocks `git rev-parse --is-inside-work-tree` to fail.
* **Missing `tmux`:** Mocks `command -v tmux` to fail. * [x] **Missing `tmux`:** Mocks `command -v tmux` to fail.
* **Missing `opencode`:** Mocks `command -v opencode` to fail. * [x] **Missing `opencode`:** Mocks `command -v opencode` to fail.
* **`opencode models` failure:** Mocks `opencode models` to fail. * [ ] **`opencode models` failure:** Mocks `opencode models` to fail.
* **Invalid model IDs (Fuzzy Matching):** * [x] **Invalid model IDs (Fuzzy Matching):**
* Mocks `opencode models` to return a specific list. * [x] Mocks `opencode models` to return a specific list.
* Test with an invalid model that is close to a valid one (verifies "Did you mean?" suggestions). * [ ] Test with an invalid model that is close to a valid one (verifies "Did you mean?" suggestions).
* Test with multiple invalid models. * [ ] Test with multiple invalid models.
* **Tmux session already exists:** Mocks `tmux has-session -t <session>` to succeed (return exit code 0). * [x] **Tmux session already exists:** Mocks `tmux has-session -t <session>` to succeed (return exit code 0).
### B. First Model Failures (isFirst === true) ### C. First Model Failures (isFirst === true)
The first model sets up the primary tmux session. If it fails, it returns a string immediately instead of collecting errors in `failedModels`. The first model sets up the primary tmux session. If it fails, it returns a string immediately instead of collecting errors in `failedModels`.
* **Worktree path exists:** Mock `fs.existsSync` to return `true`. Ensure it fails and `executedCommands` stops after checking tmux session. * [ ] **Worktree path exists:** Mock `fs.existsSync` to return `true`. Ensure it fails and `executedCommands` stops after checking tmux session.
* **Git branch exists:** Mock `git show-ref --verify --quiet refs/heads/...` to return exit code 0. * [ ] **Git branch exists:** Mock `git show-ref --verify --quiet refs/heads/...` to return exit code 0.
* **Git worktree add fails:** Mock `git worktree add -b ...` to fail. * [ ] **Git worktree add fails:** Mock `git worktree add -b ...` to fail.
* **Tmux new-session fails:** Mock `tmux new-session ...` to fail. Verify that `undoWorktree` (`git worktree remove` and `git branch -D`) is called and appended to `executedCommands`. * [ ] **Tmux new-session fails:** Mock `tmux new-session ...` to fail. Verify that `undoWorktree` (`git worktree remove` and `git branch -D`) is called and appended to `executedCommands`.
* **Tmux send-keys fails:** Mock `tmux send-keys ...` to fail. Verify `undoWorktree` commands are called. * [ ] **Tmux send-keys fails:** Mock `tmux send-keys ...` to fail. Verify `undoWorktree` commands are called.
### C. Subsequent Model Failures (isFirst === false) ### D. Subsequent Model Failures (isFirst === false)
Failures on subsequent models do not abort the process; they append to `failedModels`. Failures on subsequent models do not abort the process; they append to `failedModels`.
* **Worktree path exists:** Mock `fs.existsSync` to return `true` only for the second model's path. Verify first model succeeds, second model is skipped, and result output indicates partial success/failure. * [ ] **Worktree path exists:** Mock `fs.existsSync` to return `true` only for the second model's path. Verify first model succeeds, second model is skipped, and result output indicates partial success/failure.
* **Git branch exists:** Mock `git show-ref` to succeed for the second model. Verify it continues and records the failure. * [ ] **Git branch exists:** Mock `git show-ref` to succeed for the second model. Verify it continues and records the failure.
* **Git worktree add fails:** Mock `git worktree add` to fail for the second model. * [ ] **Git worktree add fails:** Mock `git worktree add` to fail for the second model.
* **Tmux new-window fails:** Mock `tmux new-window` to fail. Verify `undoWorktree` commands are executed for the second model's branch/path. * [ ] **Tmux new-window fails:** Mock `tmux new-window` to fail. Verify `undoWorktree` commands are executed for the second model's branch/path.
* **Tmux send-keys fails:** Mock `tmux send-keys` to fail for the second model. Verify `undoWorktree` commands are executed. * [ ] **Tmux send-keys fails:** Mock `tmux send-keys` to fail for the second model. Verify `undoWorktree` commands are executed.
### D. Success Cases & Complex Window Names ### E. Success Cases & Complex Window Names
* **Single model success:** Standard happy path. Verify exact command sequence. Note: Existing tests marked as `[x]` will be updated to assert on the exact `executedCommands` sequence.
* **Multiple models success:** Verify `tmux new-session` is called for the first, and `tmux new-window` for the rest. * [x] **Single model success:** Standard happy path. Verify exact command sequence.
* **Sanitization:** Pass a session name with spaces and special chars. Verify the safe name is used in git branch and worktree path commands. * [x] **Multiple models success:** Verify `tmux new-session` is called for the first, and `tmux new-window` for the rest.
* **Model name collision:** Pass models like `openai/gpt-4o` and `anthropic/gpt-4o`. Verify the window names in the commands are `gpt-4o` and `gpt-4o-2` respectively. * [x] **Sanitization:** Pass a session name with spaces and special chars. Verify the safe name is used in git branch and worktree path commands.
* **Long model name:** Pass a model name that evaluates to > 24 chars for the base window name. Verify truncation in the executed `tmux` commands. * [ ] **Model name collision:** Pass models like `openai/gpt-4o` and `anthropic/gpt-4o`. Verify the window names in the commands are `gpt-4o` and `gpt-4o-2` respectively.
* [ ] **Long model name:** Pass a model name that evaluates to > 24 chars for the base window name. Verify truncation in the executed `tmux` commands.
3. **Refactor Existing Unit Tests:** 3. **Refactor Existing Unit Tests:**
- Update the "unit tests" at the bottom of the existing file to correctly test `__testing_helpers.shellQuote` and `__testing_helpers.normalizeModels` instead of redefining the functions. Expand these unit tests to cover all edge cases mapped out in section A. - Update the "unit tests" at the bottom of the existing file to correctly test `__testing_helpers.shellQuote` and `__testing_helpers.normalizeModels` instead of redefining the functions. Expand these unit tests to cover all edge cases mapped out in section A.