Changed the `idempotentHint` annotation from `false` to `true` in the MCP server configuration. This enables idempotent behavior hints for relevant operations.
- Replace union output schema with a single combined schema containing optional fields for success and error cases, addressing MCP SDK limitation with union types.
- Add `importantSections`, `otherSections`, and `consoleOutput` as optional fields in the new schema.
- Update exported `outputSchema` and `EvaluateToolOutput` type to use the combined schema.
- Refactor execution logic to handle engine262's `ThrowCompletion_` instead of relying on thrown exceptions, extracting a readable error message.
- Adjust MCP server error detection to check `result.error !== undefined` rather than using the `in` operator.
- Update related comments and documentation to reflect the new schema design and error handling approach.
- Updated tool metadata to describe pure ECMAScript restrictions and new JSON output (importantSections, otherSections, consoleOutput).
- Added `ConsoleEntry` type and logic to capture console method calls during evaluation.
- Exposed a global `console` with `log`, `warn`, `debug`, and `error` methods to the evaluated code.
- Adjusted argument description and example usage accordingly.
- Introduce `recursive` boolean argument (default true) to the getSectionContent tool schema.
- Update descriptions to clarify that recursive fetching occurs only when `recursive=true`.
- Pass the `recursive` parameter from the CLI wrapper to the core tool function.
- Modify the content fetching logic to traverse child sections only when `recursive` is enabled.
Implemented a capture flag in `Ask262Debug` to enable/disable marking.
- Added `_captureEnabled` property and related logic in `mark()`.
- Introduced `startTrace()` and `stopTrace()` methods to toggle capture.
- Updated verification script to start tracing, run tests, stop tracing, and assert that the `Array.prototype.every` section is captured and marked important.
- Moved `verify-ask262-debug.mts` from `scripts` to `lib-src/node`.
- Updated import to use `#self` alias.
- Added npm script `verify-ask262` to run the verification script.
- Adjusted `scripts/tsconfig.json` to include the new file location.
Restore original formatting in engine262 folder:
- unicode.d.ts: revert to 4-space indentation, single quotes
- syntax-error.d.ts: revert to 4-space indentation, remove semicolons
Biome should not format files in engine262/ as it's excluded in biome.json.
Ordered lists switch between `1.` (even depth) and `A.` (odd depth).
Unordered lists switch between `-` (even depth) and `*` (odd depth).
Makes nested list structure visually distinct in extracted text.
- Removed default test root setting.
- Introduced `pathIgnorePatterns` list with multiple directories (engine262, spec-built, node_modules, dist, build, coverage, .idea, docs, storage, graphology, .git) to be excluded from test runs and module resolution.
- Replace per‑chunk sorting with aggregation of total size per section.
- Compute total characters and chunk count for each document.
- Sort sections by total size and display the top five, including part count when applicable.
- Renamed `partIndex`/`totalParts` to `partindex`/`totalparts` in `section_retriever.ts`, `spec_retriever.ts`, and document creation logic.
- Adjusted sorting logic to use the new `partindex` field.
- Updated metadata handling and string interpolation to reference the lowercase keys.
- Fixed import path for `HTMLTextSplitter` and moved the file to `setup/textsplitters/HtmlTextSplitter.ts`.
- Added HTML preprocessing utilities (`addNewlinesAfterBlocks`, `convertTablesToMarkdown`) in ingestion workflow.
- Integrated table-to‑markdown conversion and newline insertion to improve text splitting and document structure.
- Introduced `SectionInfo` interface to store chunk count and individual chunk sizes.
- Replaced simple section count map with a map of `SectionInfo` for richer data.
- Updated logic to aggregate chunk sizes and counts per section.
- Modified summary output to show total characters, top 5 sections with detailed chunk count, total chars, and a list of chunk sizes.
- Refactored comments and variable names for clarity.
add ChunkInfo interface, improve chunk handling typings, and print ingest summary statistics. This introduces typed chunk metadata, refactors related code, and adds a summary report for document sizes, sections, and chunk distribution
- Verify that `splitDocuments` correctly prepends the provided `chunkHeader` to each chunk.
- Ensure `splitDocuments` returns an empty array when the source HTML yields no chunks.
- Add .vscode/extensions.json recommending oven.bun-vscode.
- Add .vscode/launch.json with Bun debug configurations for various scripts.
- Add .vscode/settings.json for TypeScript SDK, debug options, and file exclusions.
- Remove .vscode/ entry from .gitignore.
- Refactor ingest.ts: increase chunk size to 8192 and overlap to 200, raise large document threshold, remove <h1>/<h2> separators, skip sections containing only headings or minimal content, and add warnings for very small chunks.
- Remove construction of `partRef` and parent section annotations from chunk text
- Directly extract and trim text from HTML chunk using cheerio
- Streamline document creation logic without additional references.
Introduce a new manual test file `test/manual/test-spec-retriever.ts` that demonstrates how to use the `spec_retriever` agent tool with a query. The script loads embeddings, connects to the LanceDB storage, creates the tool, executes it, and prints the result. This provides a runnable example for developers.
- Added EMBEDDING_MODEL and RERANKER_MODEL to `constants.ts`.
- Updated imports to use these constants in `agent.ts`, `agent_tools/reranker.ts`, and `setup/ingest.ts`.
- Replaced hard‑coded model strings with the new constants for Ollama embeddings and reranker.
add HTML‑aware text splitter, raise batch size and large‑doc threshold, enhance progress spinner and error logging, rename ingestSpec to buildSpecDocuments and introduce hierarchical section processing.
- Consolidate all structural element breakdowns into a single `breakDownSection` function using an `alwaysBreak` flag.
- Introduce `BREAKDOWN_TAGS` configuration array containing metadata (tag, alwaysBreak, title/id selectors) for each element type.
- Replace previous multi‑phase approach with a unified sequential breakdown flow: `emu-clause` (always extracts children) → `emu-table` → `emu-grammar` → `td` → `p`.
- Add inline markers `[Subsection available: title "X" at sectionid: ID]` where content is removed, enabling parent awareness.
- Update documentation to reflect the new unified breakdown logic, tag table, and hierarchical ID format.
- Adjust threshold handling, recursion depth, and metadata tracking to work with the new unified approach.
Added a detailed markdown plan outlining fixes for parent section awareness, sequential tag‑based breakdown, and recursive processing of large sections. Includes implementation details, configuration constants, and updated ingestion logic. This new plan file is located at `.opencode/plans/1774872666261-parent-breakdown-fix.md`.
- Move full‑section Document creation into an `else` block so it only occurs when no sub‑documents were created.
- Remove premature return that skipped adding the full section.
- Add post‑processing loop to log a warning for any final document exceeding `LARGE_DOC_THRESHOLD`.
Added a comprehensive markdown plan (.opencode/plans/1774872124705-glowing-river.md) outlining the migration from LlamaIndex to LangChain.js with LanceDB. Includes updated dependencies, file modification list, and detailed code changes for `ingest.ts` and `agent.ts`. This serves as documentation for the migration process.
Search for `emu-clause` elements now performed within the `#spec-container` element instead of the entire document, preventing accidental matches outside the spec container.
- Log the number of specification HTML files found in `SPEC_DIR`.
- Emit a warning if no specification HTML files are present.
- Log the number of code files found in `CODE_DIR`.
- Emit a warning if no code files are present.
Update init_engine262.sh to add the engine262 repository as a squashed subtree instead of a shallow submodule. Adjust usage comments, add existence check for the target directory, and update echo messages accordingly.
Introduces `setup/init_engine262.sh`, a Bash utility that clones the `engine262` repository as a shallow submodule (depth 1) into `./engine262`. The script adds the submodule, initializes it, and provides a usage comment. It ensures robust execution with `set -euo pipefail`.
- Added `constants.mjs` defining `STORAGE_DIR` and `GRAPH_FILE`.
- Replaced hard‑coded path strings in `agent.mjs`, `build_graph.mjs`, and `ingest.mjs` with imports from the new constants module.
- Updated console log in `ingest.mjs` to reference `STORAGE_DIR`.
- Adjusted `GRAPH_FILE` path to `./graphology/graph.json` via the constant.
bendtherules