ask262/.opencode/plans/1774523669864-glowing-wizard.md

Plan for building a RAG-like pipeline for language specification exploration

Technology Stack:

• Framework: LlamaIndex.js
• Graph Storage: LlamaIndex SimpleGraphStore (file-based)
• Embeddings: Local Ollama

High-Level Architecture

The system will have four main components:

1. Data Ingestion: A pipeline to process the HTML specification and the Javascript code.
2. Data Storage: A combination of a vector store and a graph store, both managed by LlamaIndex.
3. RAG Pipeline: A retrieval-augmented generation pipeline built with LlamaIndex.
4. Agentic Chat Interface: An interactive chat interface for the user to interact with the system.

Phase 1: Data Ingestion and Basic RAG

This phase focuses on getting the basic RAG pipeline working with LlamaIndex.

• Task 1: Setup Project and Dependencies:

  • Initialize a Node.js project.
  • Install llamaindex.
  • Set up Ollama for local embeddings.
  • Clone the engine262 repository from https://github.com/engine262/engine262 into the ./engine262 folder.

• Task 2: HTML Specification Parsing:

  • Write a script to recursively read all HTML files from the ./spec-built/multipage folder.
  • Use a library like cheerio to parse the HTML and extract the main text content.
  • Use LlamaIndex's SimpleNodeParser to break down the content into meaningful chunks (nodes).

• Task 3: Javascript Code Parsing:

  • Write a script to read the Javascript files from the ./engine262/src folder.
  • Initially, treat the code as plain text and parse it into nodes.

• Task 4: Vector Database Integration with LlamaIndex:

  • Configure LlamaIndex to use your local Ollama instance for generating embeddings by using the OllamaEmbedding class.
  • LlamaIndex will manage the documents and embeddings in a vector store. You can start with an in-memory store.

• Task 5: Basic RAG Pipeline:

  • Use LlamaIndex's VectorStoreIndex to build an index over your parsed documents.
  • Create a QueryEngine from the index to ask questions.

• Task 6: Simple CLI Interface:

  • Create a simple command-line interface to ask questions and see the results from the LlamaIndex query engine.

Phase 2: Graph Integration with Graphology

This phase will enhance the retrieval process by modeling the structure of the specification and the code using Graphology. LlamaIndex.js's native graph store is currently limited, so we'll use a custom mapping.

• Task 1: Graph Storage Setup:

  • Use graphology to build an in-memory graph.
  • Persist the graph to a graph.json file for persistence.

• Task 2: Enhance Parsers to Extract Relationships:

  • HTML Parser: When parsing the spec, extract section titles (e.g., "If Statement") and their IDs.
  • JS Parser: Use acorn or @babel/parser to create an AST of the Javascript code and extract function names. Create a mapping from function names to specification section titles (e.g., Evaluate_IfStatement -> If Statement).

• Task 3: Populate Graph Store:

  • Write a script to build the graph:
    • Create nodes for specification sections (SpecSection) and Javascript functions (JSFunction).
    • Create relationships for links between spec sections (LINKS_TO) and for functions implementing a spec section (IMPLEMENTS).
  • Save this graph to graph.json.

• Task 4: Enhance Retriever with Custom Graph Lookup:

  • Create a custom retriever that first looks up the relevant code/spec in the graph and then uses the vector store for detailed retrieval.

Phase 3: Agentic Chat and Tool Use

This phase focuses on building the interactive and "smart" agent using LlamaIndex's capabilities.

• Task 1: Agent Framework:

  • Use LlamaIndex's agent classes, like ChatEngine or QueryEngine with tools, to create the agent.
  • Configure the agent to use your OpenAI-compatible service for generation.

• Task 2: Define Tools:

  • LlamaIndex allows you to define tools. The primary tool will be a QueryEngineTool that uses the combined vector and graph index from the previous phases.
  • You could also create more specific tools, like one to directly query the graph for structural information.

• Task 3: Build the Chat Interface:

  • Create a web-based chat interface (e.g., using React or Vue).
  • This interface will interact with the LlamaIndex agent endpoint.