ask262/.opencode/plans/1774872124705-glowing-river.md

# Migration Plan: LlamaIndex → LangChain.js + LanceDB

**Model:** fireworks-ai/accounts/fireworks/routers/kimi-k2p5-turbo  
**Date:** 2026-03-31

Migrate ask262 from LlamaIndex to LangChain.js with LanceDB for superior metadata pre-filtering.

## Files to Modify

| File | Changes |
|------|---------|
| `package.json` | Replace LlamaIndex with LangChain.js + LanceDB |
| `agent.ts` | ReActAgent → LangChain agent |
| `setup/ingest.ts` | Text splitter + LanceDB storage |
| `setup/build_graph.ts` | **NO CHANGES** |

## Dependencies

**Remove:**
```json
["@llamaindex/*", "llamaindex"]
```

**Add:**
```json
{
  "@langchain/core": "^0.2.0",
  "@langchain/ollama": "^0.1.0", 
  "@langchain/openai": "^0.1.0",
  "@lancedb/lancedb": "^0.5.0",
  "langchain": "^0.2.0"
}
```

## Phase 1: ingest.ts Changes

### 1. Imports
```typescript
// OLD
import { OllamaEmbedding } from "@llamaindex/ollama";
import { Document, SentenceSplitter, Settings, storageContextFromDefaults, VectorStoreIndex } from "llamaindex";

// NEW
import { OllamaEmbeddings } from "@langchain/ollama";
import { Document } from "@langchain/core/documents";
import { RecursiveCharacterTextSplitter } from "@langchain/textsplitters";
import { LanceDB } from "@langchain/community/vectorstores/lancedb";
import * as lancedb from "@lancedb/lancedb";
```

### 2. Text Splitting
```typescript
// OLD
const sentenceSplitter = new SentenceSplitter({ chunkSize: 2048, chunkOverlap: 50 });
const rawNodes = sentenceSplitter.getNodesFromDocuments(specDocs);

// NEW
const textSplitter = new RecursiveCharacterTextSplitter({
  chunkSize: 2048,
  chunkOverlap: 50,
  separators: ["\n\n", "\n", ". ", " ", ""]
});
const splitDocs = await textSplitter.splitDocuments(specDocs);
```

### 3. Document Creation (LOWERCASE KEYS!)
```typescript
// OLD
new Document({ text, metadata: { sectionId: id, sectionTitle: title, ... } })

// NEW
new Document({
  pageContent: text,
  metadata: {
    sectionid: id,        // lowercase!
    sectiontitle: title,  // lowercase!
    source: file,
    type: "specification",
    parentsectionid: null,
    breakdowntag: null
  }
})
```

### 4. Storage
```typescript
// OLD
const storageContext = await storageContextFromDefaults({ persistDir: STORAGE_DIR });
const index = await VectorStoreIndex.init({ storageContext });
await index.insertNodes(batch);

// NEW
const db = await lancedb.connect(STORAGE_DIR);

// Check/prompt for existing table
let table;
try {
  table = await db.openTable("spec_vectors");
  // Prompt: overwrite?
} catch {
  table = await db.createTable("spec_vectors", []);
}

// Add scalar indexes (REQUIRED!)
await table.createScalarIndex("sectionid");
await table.createScalarIndex("breakdowntag");
await table.createScalarIndex("type");

// Store documents
const vectorStore = new LanceDB(
  new OllamaEmbeddings({ model: "nomic-embed-text-v2-moe" }), 
  { table }
);
await vectorStore.addDocuments(documents);
```

## Phase 2: agent.ts Changes

### 1. Imports
```typescript
import { OllamaEmbeddings } from "@langchain/ollama";
import { ChatOpenAI } from "@langchain/openai";
import { LanceDB } from "@langchain/community/vectorstores/lancedb";
import { createReactAgent } from "@langchain/agents";
import { DynamicTool } from "@langchain/core/tools";
import * as lancedb from "@lancedb/lancedb";
```

### 2. LLM Setup
```typescript
// OLD
Settings.embedModel = new OllamaEmbedding({ model: "nomic-embed-text-v2-moe" });
const llm = new OpenAI({ model: "openai/gpt-oss-120b", apiKey, baseURL, temperature: 0 });
Settings.llm = llm;

// NEW
const embeddings = new OllamaEmbeddings({ model: "nomic-embed-text-v2-moe" });
const llm = new ChatOpenAI({
  modelName: "openai/gpt-oss-120b",
  openAIApiKey: apiKey,
  configuration: { baseURL },
  temperature: 0
});
```

### 3. Vector Store
```typescript
const db = await lancedb.connect(STORAGE_DIR);
const table = await db.openTable("spec_vectors");
const vectorStore = new LanceDB(embeddings, { table });
```

### 4. Tools
```typescript
// spec_retriever - semantic search
const specRetrieverTool = new DynamicTool({
  name: "spec_retriever",
  description: "Queries the language specification for text content about specific sections or topics.",
  func: async (query) => {
    const results = await vectorStore.similaritySearch(query, 3);
    return results.map(r => r.pageContent).join("\n\n");
  }
});

// fetch_section_chunks - get all chunks from a section
const sectionRetrieverTool = new DynamicTool({
  name: "fetch_section_chunks",
  description: "Retrieves all text chunks from a specific specification section by sectionId.",
  func: async (sectionId) => {
    const results = await table
      .query()  // No vector search - pure metadata query
      .where(`sectionid = '${sectionId}'`)
      .limit(100)
      .toArray();
    return results.map(r => r.text).join("\n\n");
  }
});

// graph_explorer - NO CHANGES (wrap in DynamicTool)
const graphTool = new DynamicTool({
  name: "graph_explorer",
  description: "Explores structural relationships between specification sections and implementation code.",
  func: async (query) => {
    // Existing Graphology logic preserved
  }
});
```

### 5. Agent
```typescript
// OLD
const agent = new ReActAgent({ tools: [queryEngineTool, graphTool], llm, verbose: true, systemPrompt });
const response = await agent.chat({ message });

// NEW
const agent = createReactAgent({
  llm,
  tools: [specRetrieverTool, sectionRetrieverTool, graphTool],
  prompt: systemPrompt // Keep existing system prompt
});

const response = await agent.invoke({
  messages: [{ role: "user", content: message }]
});
```

## LanceDB Critical Requirements

### Column Naming (⚠️ IMPORTANT)
- ✅ Use lowercase: `sectionid`, `sectiontitle`, `breakdowntag`
- ❌ **NO periods**: `metadata.sectionid` will NOT work
- ❌ **NO uppercase** without backticks: `` `sectionId` ``
- Keep names simple: letters, numbers, underscores

### SQL Examples
```typescript
// Simple equality
.where(`sectionid = 'sec-if-statement'`)

// Pattern matching
.where(`sectionid LIKE 'sec-if-%'`)
```

**Note:** Use backticks for uppercase columns: `.where("`sectionId` = 'value'")`

### Performance: Scalar Indexes
**Required** for columns used in WHERE clauses:
```typescript
await table.createScalarIndex("sectionid");     // REQUIRED
await table.createScalarIndex("breakdowntag");  // Recommended
await table.createScalarIndex("type");            // Recommended
```

### Highly Selective Filters
If filter returns few rows, increase `nprobes`:
```typescript
const results = await table
  .search(queryEmbedding)
  .where(`sectionid = 'sec-rare-section'`)
  .nprobes(20)  // Default is 1-5
  .limit(10)
  .toArray();
```

### Query Methods
- `.search(vector)` - Vector similarity with optional pre-filtering
- `.query()` - Pure metadata query (no vector search) - use for section retrieval

## Testing Checklist

- [ ] `bun install` completes without errors
- [ ] `bun run setup/ingest.ts` creates LanceDB table with lowercase metadata
- [ ] Scalar indexes created for `sectionid`, `breakdowntag`, `type`
- [ ] `bun run setup/build_graph.ts` works unchanged
- [ ] Agent queries work: `bun run agent.ts "How does if statement work?"`
- [ ] Section retrieval works: fetches all chunks from specific sectionid
- [ ] TypeScript compiles: `bun run type-check`
- [ ] Linting passes: `bun run lint`

## Migration Strategy

**Recommended:** Clean re-ingest
1. Delete `storage/` directory
2. Run `bun run setup/ingest.ts`
3. Run `bun run setup/build_graph.ts`
4. Test agent queries

**Timeline:** ~2 hours