Key Concepts
RAG Architecture
Three independent, swappable components: Search → Prompt Builder → LLM. Your model is only as good as your retrieval.
Search with minsearch
Lightweight in-memory search. Text fields are ranked by TF-IDF; keyword fields are exact-match filters only.
Prompt Engineering
Fixed instructions separate from dynamic context. Instructions reduce hallucinations by constraining the LLM to retrieved content.
Chunking
Sliding window splits long pages into overlapping pieces. Overlap ensures passages near boundaries aren't lost.
Function Calling
LLM requests tool execution via JSON schemas. It can retry with reformulated queries — behaviour you didn't explicitly program.
The Agentic Loop
while True: call LLM → if function_call → execute → loop. Stops when the model returns no tool calls. Instructions, tools, and memory are the three pillars.
Agent Frameworks
toyaikit auto-generates tool schemas from type hints and docstrings. Same pattern across OpenAI SDK, PydanticAI, LangChain.
Data Ingestion
Separate ingestion from querying. Ingest once to persistent storage; query as often as needed. Avoids re-indexing on every restart.
RAG vs Agentic RAG
Plain RAG
User Query
↓
Search (once, fixed)
↓
Build Prompt
↓
LLM
↓
Answer
Agentic RAG
User Query
↓
LLM ──→ Search
↑ ↓
results ←────┘
↓
Final Answer