AI Agents & Agentic Engineering
  • Context Engineering
  • Building AI Agents
  • Agentic Engineering Best Practices
  • Auto-Research and Self-Improving Agents
  • Self-Maintaining Agents
  • Scaling Long-Running Autonomous Coding
  • Data Agents
  • Agent Memory and Harness Ownership
LLM Internals & Training
  • Reinforcement Learning for LLMs
  • Cursor Composer 2
  • Training-Free GRPO
  • Prompt Repetition for Non-Reasoning LLMs
  • LLM Quantization
  • LLM Knowledge Bases
AI Safety & Security
  • Constitutional Classifiers
  • Project Glasswing
Graphics & Low-Level Programming
  • Shaders and GPU Programming
Personal Development & Mindset
  • Deep Focus and Productivity
  • Identity and Life Design
  • Self-Competition and Entrepreneurship

LLM Knowledge Bases

Andrej Karpathy described a pattern of using LLMs to build and maintain personal knowledge bases: a structured, interlinked collection of markdown files that the LLM owns and evolves. Instead of retrieving from raw documents at query time, the LLM incrementally compiles and maintains a persistent wiki.

Source: Karpathy (@karpathy)

Architecture

Data Ingest

Source documents (articles, papers, repos, datasets, images) go into a raw/ directory. The LLM then incrementally "compiles" them into a wiki: a collection of .md files in a directory structure.

The wiki includes:

  • Summaries of all raw data
  • Backlinks between related pages
  • Categorized concepts
  • Articles for each concept
  • Cross-references throughout

Web clipping: Obsidian Web Clipper converts web articles to .md files. A hotkey downloads related images to local storage for LLM reference.

IDE

Obsidian serves as the frontend for viewing raw data, the compiled wiki, and derived visualizations. The LLM writes and maintains all wiki data; the human rarely touches it directly. Obsidian plugins like Marp handle slides and other formats.

Q&A

Once the wiki reaches sufficient scale (Karpathy's research wiki: ~100 articles, ~400K words), you can ask complex questions against it. The LLM agent researches answers by navigating the wiki. Karpathy expected to need fancy RAG but found the LLM handles it well at this scale through auto-maintained index files and brief summaries.

Output

Instead of terminal text, the LLM renders:

  • Markdown files
  • Slide shows (Marp format)
  • Matplotlib images

All viewable in Obsidian. Outputs often get "filed" back into the wiki to enhance it for further queries. Explorations compound in the knowledge base.

Linting

LLM "health checks" over the wiki:

  • Find inconsistent data
  • Impute missing data (using web searches)
  • Find interesting connections for new article candidates
  • Incrementally clean up data integrity

LLMs are good at suggesting further questions to ask and investigate.

Extra Tools

Karpathy vibe-coded a small search engine over the wiki, used both directly via web UI and as a CLI tool handed off to the LLM for larger queries.

The Key Idea

The difference from traditional RAG: the wiki is a persistent, compounding artifact. Cross-references are already there. Contradictions have already been flagged. Synthesis already reflects everything ingested. The wiki gets richer with every source added and every question asked. Knowledge is compiled once and kept current, not re-derived on every query.

Future Direction

As the repo grows, the natural desire is synthetic data generation + fine-tuning to have the LLM "know" the data in its weights instead of just context windows.

"There is room here for an incredible new product instead of a hacky collection of scripts."

This Vault

This knowledge base follows Karpathy's pattern: raw sources in raw/, an LLM-compiled wiki in wiki/, with cross-references and index maintenance.

Related Topics

  • context-engineering - Knowledge bases as a form of persistent context
  • data-agents - Data agents that also build persistent knowledge through use
  • auto-research - Self-improving knowledge through experimentation
  • agent-memory-and-harnesses - Agent memory as persistent, compounding knowledge