build: basic setup (gemini-3-flash)

.opencode/plans/1774523669864-glowing-wizard.md

@@ -0,0 +1,83 @@
+# Plan for building a RAG-like pipeline for language specification exploration
+
+**Technology Stack:**
+- **Framework:** LlamaIndex.js
+- **Graph Storage:** LlamaIndex `SimpleGraphStore` (file-based)
+- **Embeddings:** Local Ollama
+
+## High-Level Architecture
+
+The system will have four main components:
+
+1.  **Data Ingestion:** A pipeline to process the HTML specification and the Javascript code.
+2.  **Data Storage:** A combination of a vector store and a graph store, both managed by LlamaIndex.
+3.  **RAG Pipeline:** A retrieval-augmented generation pipeline built with LlamaIndex.
+4.  **Agentic Chat Interface:** An interactive chat interface for the user to interact with the system.
+
+## Phase 1: Data Ingestion and Basic RAG
+
+This phase focuses on getting the basic RAG pipeline working with LlamaIndex.
+
+*   **Task 1: Setup Project and Dependencies:**
+    *   Initialize a Node.js project.
+    *   Install `llamaindex`.
+    *   Set up Ollama for local embeddings.
+    *   Clone the `engine262` repository from `https://github.com/engine262/engine262` into the `./engine262` folder.
+
+*   **Task 2: HTML Specification Parsing:**
+    *   Write a script to recursively read all HTML files from the `./spec-built/multipage` folder.
+    *   Use a library like `cheerio` to parse the HTML and extract the main text content.
+    *   Use LlamaIndex's `SimpleNodeParser` to break down the content into meaningful chunks (nodes).
+
+*   **Task 3: Javascript Code Parsing:**
+    *   Write a script to read the Javascript files from the `./engine262/src` folder.
+    *   Initially, treat the code as plain text and parse it into nodes.
+
+*   **Task 4: Vector Database Integration with LlamaIndex:**
+    *   Configure LlamaIndex to use your local Ollama instance for generating embeddings by using the `OllamaEmbedding` class.
+    *   LlamaIndex will manage the documents and embeddings in a vector store. You can start with an in-memory store.
+
+*   **Task 5: Basic RAG Pipeline:**
+    *   Use LlamaIndex's `VectorStoreIndex` to build an index over your parsed documents.
+    *   Create a `QueryEngine` from the index to ask questions.
+
+*   **Task 6: Simple CLI Interface:**
+    *   Create a simple command-line interface to ask questions and see the results from the LlamaIndex query engine.
+
+## Phase 2: Graph Integration with Graphology
+
+This phase will enhance the retrieval process by modeling the structure of the specification and the code using Graphology. LlamaIndex.js's native graph store is currently limited, so we'll use a custom mapping.
+
+*   **Task 1: Graph Storage Setup:**
+    *   Use `graphology` to build an in-memory graph.
+    *   Persist the graph to a `graph.json` file for persistence.
+
+*   **Task 2: Enhance Parsers to Extract Relationships:**
+    *   **HTML Parser:** When parsing the spec, extract section titles (e.g., "If Statement") and their IDs.
+    *   **JS Parser:** Use `acorn` or `@babel/parser` to create an AST of the Javascript code and extract function names. Create a mapping from function names to specification section titles (e.g., `Evaluate_IfStatement` -> `If Statement`).
+
+*   **Task 3: Populate Graph Store:**
+    *   Write a script to build the graph:
+        *   Create nodes for specification sections (`SpecSection`) and Javascript functions (`JSFunction`).
+        *   Create relationships for links between spec sections (`LINKS_TO`) and for functions implementing a spec section (`IMPLEMENTS`).
+    *   Save this graph to `graph.json`.
+
+*   **Task 4: Enhance Retriever with Custom Graph Lookup:**
+    *   Create a custom retriever that first looks up the relevant code/spec in the graph and then uses the vector store for detailed retrieval.
+
+## Phase 3: Agentic Chat and Tool Use
+
+This phase focuses on building the interactive and "smart" agent using LlamaIndex's capabilities.
+
+*   **Task 1: Agent Framework:**
+    *   Use LlamaIndex's agent classes, like `ChatEngine` or `QueryEngine` with tools, to create the agent.
+    *   Configure the agent to use your OpenAI-compatible service for generation.
+
+*   **Task 2: Define Tools:**
+    *   LlamaIndex allows you to define tools. The primary tool will be a `QueryEngineTool` that uses the combined vector and graph index from the previous phases.
+    *   You could also create more specific tools, like one to directly query the graph for structural information.
+
+*   **Task 3: Build the Chat Interface:**
+    *   Create a web-based chat interface (e.g., using React or Vue).
+    *   This interface will interact with the LlamaIndex agent endpoint.
+