mirror of
https://github.com/bendtherules/ask262.git
synced 2026-08-18 21:31:46 +00:00
84 lines
4.4 KiB
Markdown
84 lines
4.4 KiB
Markdown
# Plan for building a RAG-like pipeline for language specification exploration
|
|
|
|
**Technology Stack:**
|
|
- **Framework:** LlamaIndex.js
|
|
- **Graph Storage:** LlamaIndex `SimpleGraphStore` (file-based)
|
|
- **Embeddings:** Local Ollama
|
|
|
|
## High-Level Architecture
|
|
|
|
The system will have four main components:
|
|
|
|
1. **Data Ingestion:** A pipeline to process the HTML specification and the Javascript code.
|
|
2. **Data Storage:** A combination of a vector store and a graph store, both managed by LlamaIndex.
|
|
3. **RAG Pipeline:** A retrieval-augmented generation pipeline built with LlamaIndex.
|
|
4. **Agentic Chat Interface:** An interactive chat interface for the user to interact with the system.
|
|
|
|
## Phase 1: Data Ingestion and Basic RAG
|
|
|
|
This phase focuses on getting the basic RAG pipeline working with LlamaIndex.
|
|
|
|
* **Task 1: Setup Project and Dependencies:**
|
|
* Initialize a Node.js project.
|
|
* Install `llamaindex`.
|
|
* Set up Ollama for local embeddings.
|
|
* Clone the `engine262` repository from `https://github.com/engine262/engine262` into the `./engine262` folder.
|
|
|
|
* **Task 2: HTML Specification Parsing:**
|
|
* Write a script to recursively read all HTML files from the `./spec-built/multipage` folder.
|
|
* Use a library like `cheerio` to parse the HTML and extract the main text content.
|
|
* Use LlamaIndex's `SimpleNodeParser` to break down the content into meaningful chunks (nodes).
|
|
|
|
* **Task 3: Javascript Code Parsing:**
|
|
* Write a script to read the Javascript files from the `./engine262/src` folder.
|
|
* Initially, treat the code as plain text and parse it into nodes.
|
|
|
|
* **Task 4: Vector Database Integration with LlamaIndex:**
|
|
* Configure LlamaIndex to use your local Ollama instance for generating embeddings by using the `OllamaEmbedding` class.
|
|
* LlamaIndex will manage the documents and embeddings in a vector store. You can start with an in-memory store.
|
|
|
|
* **Task 5: Basic RAG Pipeline:**
|
|
* Use LlamaIndex's `VectorStoreIndex` to build an index over your parsed documents.
|
|
* Create a `QueryEngine` from the index to ask questions.
|
|
|
|
* **Task 6: Simple CLI Interface:**
|
|
* Create a simple command-line interface to ask questions and see the results from the LlamaIndex query engine.
|
|
|
|
## Phase 2: Graph Integration with Graphology
|
|
|
|
This phase will enhance the retrieval process by modeling the structure of the specification and the code using Graphology. LlamaIndex.js's native graph store is currently limited, so we'll use a custom mapping.
|
|
|
|
* **Task 1: Graph Storage Setup:**
|
|
* Use `graphology` to build an in-memory graph.
|
|
* Persist the graph to a `graph.json` file for persistence.
|
|
|
|
* **Task 2: Enhance Parsers to Extract Relationships:**
|
|
* **HTML Parser:** When parsing the spec, extract section titles (e.g., "If Statement") and their IDs.
|
|
* **JS Parser:** Use `acorn` or `@babel/parser` to create an AST of the Javascript code and extract function names. Create a mapping from function names to specification section titles (e.g., `Evaluate_IfStatement` -> `If Statement`).
|
|
|
|
* **Task 3: Populate Graph Store:**
|
|
* Write a script to build the graph:
|
|
* Create nodes for specification sections (`SpecSection`) and Javascript functions (`JSFunction`).
|
|
* Create relationships for links between spec sections (`LINKS_TO`) and for functions implementing a spec section (`IMPLEMENTS`).
|
|
* Save this graph to `graph.json`.
|
|
|
|
* **Task 4: Enhance Retriever with Custom Graph Lookup:**
|
|
* Create a custom retriever that first looks up the relevant code/spec in the graph and then uses the vector store for detailed retrieval.
|
|
|
|
## Phase 3: Agentic Chat and Tool Use
|
|
|
|
This phase focuses on building the interactive and "smart" agent using LlamaIndex's capabilities.
|
|
|
|
* **Task 1: Agent Framework:**
|
|
* Use LlamaIndex's agent classes, like `ChatEngine` or `QueryEngine` with tools, to create the agent.
|
|
* Configure the agent to use your OpenAI-compatible service for generation.
|
|
|
|
* **Task 2: Define Tools:**
|
|
* LlamaIndex allows you to define tools. The primary tool will be a `QueryEngineTool` that uses the combined vector and graph index from the previous phases.
|
|
* You could also create more specific tools, like one to directly query the graph for structural information.
|
|
|
|
* **Task 3: Build the Chat Interface:**
|
|
* Create a web-based chat interface (e.g., using React or Vue).
|
|
* This interface will interact with the LlamaIndex agent endpoint.
|
|
|