ask262/.opencode/plans/1774872124705-glowing-river.md

Migration Plan: LlamaIndex → LangChain.js + LanceDB

Model: fireworks-ai/accounts/fireworks/routers/kimi-k2p5-turbo
Date: 2026-03-31

Migrate ask262 from LlamaIndex to LangChain.js with LanceDB for superior metadata pre-filtering.

Files to Modify

File	Changes
package.json	Replace LlamaIndex with LangChain.js + LanceDB
agent.ts	ReActAgent → LangChain agent
setup/ingest.ts	Text splitter + LanceDB storage
setup/build_graph.ts	NO CHANGES

Dependencies

Remove:

["@llamaindex/*", "llamaindex"]

Add:

{
  "@langchain/core": "^0.2.0",
  "@langchain/ollama": "^0.1.0", 
  "@langchain/openai": "^0.1.0",
  "@lancedb/lancedb": "^0.5.0",
  "langchain": "^0.2.0"
}

Phase 1: ingest.ts Changes

1. Imports

// OLD
import { OllamaEmbedding } from "@llamaindex/ollama";
import { Document, SentenceSplitter, Settings, storageContextFromDefaults, VectorStoreIndex } from "llamaindex";

// NEW
import { OllamaEmbeddings } from "@langchain/ollama";
import { Document } from "@langchain/core/documents";
import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters";
import { LanceDB } from "@langchain/community/vectorstores/lancedb";
import * as lancedb from "@lancedb/lancedb";

2. Text Splitting

// OLD
const sentenceSplitter = new SentenceSplitter({ chunkSize: 2048, chunkOverlap: 50 });
const rawNodes = sentenceSplitter.getNodesFromDocuments(specDocs);

// NEW
const textSplitter = new RecursiveCharacterTextSplitter({
  chunkSize: 2048,
  chunkOverlap: 50,
  separators: ["\n\n", "\n", ". ", " ", ""]
});
const splitDocs = await textSplitter.splitDocuments(specDocs);

3. Document Creation (LOWERCASE KEYS!)

// OLD
new Document({ text, metadata: { sectionId: id, sectionTitle: title, ... } })

// NEW
new Document({
  pageContent: text,
  metadata: {
    sectionid: id,        // lowercase!
    sectiontitle: title,  // lowercase!
    source: file,
    type: "specification",
    parentsectionid: null,
    breakdowntag: null
  }
})

4. Storage

// OLD
const storageContext = await storageContextFromDefaults({ persistDir: STORAGE_DIR });
const index = await VectorStoreIndex.init({ storageContext });
await index.insertNodes(batch);

// NEW
const db = await lancedb.connect(STORAGE_DIR);

// Check/prompt for existing table
let table;
try {
  table = await db.openTable("spec_vectors");
  // Prompt: overwrite?
} catch {
  table = await db.createTable("spec_vectors", []);
}

// Add scalar indexes (REQUIRED!)
await table.createScalarIndex("sectionid");
await table.createScalarIndex("breakdowntag");
await table.createScalarIndex("type");

// Store documents
const vectorStore = new LanceDB(
  new OllamaEmbeddings({ model: "nomic-embed-text-v2-moe" }), 
  { table }
);
await vectorStore.addDocuments(documents);

Phase 2: agent.ts Changes

1. Imports

import { OllamaEmbeddings } from "@langchain/ollama";
import { ChatOpenAI } from "@langchain/openai";
import { LanceDB } from "@langchain/community/vectorstores/lancedb";
import { createReactAgent } from "@langchain/agents";
import { DynamicTool } from "@langchain/core/tools";
import * as lancedb from "@lancedb/lancedb";

2. LLM Setup

// OLD
Settings.embedModel = new OllamaEmbedding({ model: "nomic-embed-text-v2-moe" });
const llm = new OpenAI({ model: "openai/gpt-oss-120b", apiKey, baseURL, temperature: 0 });
Settings.llm = llm;

// NEW
const embeddings = new OllamaEmbeddings({ model: "nomic-embed-text-v2-moe" });
const llm = new ChatOpenAI({
  modelName: "openai/gpt-oss-120b",
  openAIApiKey: apiKey,
  configuration: { baseURL },
  temperature: 0
});

3. Vector Store

const db = await lancedb.connect(STORAGE_DIR);
const table = await db.openTable("spec_vectors");
const vectorStore = new LanceDB(embeddings, { table });

4. Tools

// spec_retriever - semantic search
const specRetrieverTool = new DynamicTool({
  name: "spec_retriever",
  description: "Queries the language specification for text content about specific sections or topics.",
  func: async (query) => {
    const results = await vectorStore.similaritySearch(query, 3);
    return results.map(r => r.pageContent).join("\n\n");
  }
});

// fetch_section_chunks - get all chunks from a section
const sectionRetrieverTool = new DynamicTool({
  name: "fetch_section_chunks",
  description: "Retrieves all text chunks from a specific specification section by sectionId.",
  func: async (sectionId) => {
    const results = await table
      .query()  // No vector search - pure metadata query
      .where(`sectionid = '${sectionId}'`)
      .limit(100)
      .toArray();
    return results.map(r => r.text).join("\n\n");
  }
});

// graph_explorer - NO CHANGES (wrap in DynamicTool)
const graphTool = new DynamicTool({
  name: "graph_explorer",
  description: "Explores structural relationships between specification sections and implementation code.",
  func: async (query) => {
    // Existing Graphology logic preserved
  }
});

5. Agent

// OLD
const agent = new ReActAgent({ tools: [queryEngineTool, graphTool], llm, verbose: true, systemPrompt });
const response = await agent.chat({ message });

// NEW
const agent = createReactAgent({
  llm,
  tools: [specRetrieverTool, sectionRetrieverTool, graphTool],
  prompt: systemPrompt // Keep existing system prompt
});

const response = await agent.invoke({
  messages: [{ role: "user", content: message }]
});

LanceDB Critical Requirements

Column Naming (⚠️ IMPORTANT)

• ✅ Use lowercase: sectionid, sectiontitle, breakdowntag
• ❌ NO periods: metadata.sectionid will NOT work
• ❌ NO uppercase without backticks: `sectionId`
• Keep names simple: letters, numbers, underscores

SQL Examples

// Simple equality
.where(`sectionid = 'sec-if-statement'`)

// Pattern matching
.where(`sectionid LIKE 'sec-if-%'`)

Note: Use backticks for uppercase columns: .where("sectionId = 'value'")

Performance: Scalar Indexes

Required for columns used in WHERE clauses:

await table.createScalarIndex("sectionid");     // REQUIRED
await table.createScalarIndex("breakdowntag");  // Recommended
await table.createScalarIndex("type");            // Recommended

Highly Selective Filters

If filter returns few rows, increase nprobes:

const results = await table
  .search(queryEmbedding)
  .where(`sectionid = 'sec-rare-section'`)
  .nprobes(20)  // Default is 1-5
  .limit(10)
  .toArray();

Query Methods

• .search(vector) - Vector similarity with optional pre-filtering
• .query() - Pure metadata query (no vector search) - use for section retrieval

Testing Checklist

• bun install completes without errors
• bun run setup/ingest.ts creates LanceDB table with lowercase metadata
• Scalar indexes created for sectionid, breakdowntag, type
• bun run setup/build_graph.ts works unchanged
• Agent queries work: bun run agent.ts "How does if statement work?"
• Section retrieval works: fetches all chunks from specific sectionid
• TypeScript compiles: bun run type-check
• Linting passes: bun run lint

Migration Strategy

Recommended: Clean re-ingest

1. Delete storage/ directory
2. Run bun run setup/ingest.ts
3. Run bun run setup/build_graph.ts
4. Test agent queries

Timeline: ~2 hours