AI Agents & Agentic Engineering
  • Context Engineering
  • Building AI Agents
  • Agentic Engineering Best Practices
  • Auto-Research and Self-Improving Agents
  • Self-Maintaining Agents
  • Scaling Long-Running Autonomous Coding
  • Data Agents
  • Agent Memory and Harness Ownership
LLM Internals & Training
  • Reinforcement Learning for LLMs
  • Cursor Composer 2
  • Training-Free GRPO
  • Prompt Repetition for Non-Reasoning LLMs
  • LLM Quantization
  • LLM Knowledge Bases
AI Safety & Security
  • Constitutional Classifiers
  • Project Glasswing
Graphics & Low-Level Programming
  • Shaders and GPU Programming
Personal Development & Mindset
  • Deep Focus and Productivity
  • Identity and Life Design
  • Self-Competition and Entrepreneurship

Auto-Research and Self-Improving Agents

Auto-research is the paradigm where agents run their own experimentation loops (trying things, evaluating results, and iterating) to improve their own performance. This is a convergence of Karpathy's "autoresearch" concept (agents editing training code to reduce loss) and "harness engineering" (shaping agent capabilities through environment and feedback loops rather than weight updates).

Harvey: Auto-Research for Legal Agents

Harvey (an AI legal platform) demonstrated auto-research applied to agent harness optimization rather than model training. Instead of editing training code, the agent edits its own non-parametric levers (skills, hooks, scripts, sub-agents, output styles) to maximize rubric scores on legal tasks.

Setup:

  • Start with a generic agent harness (basic tools: read document, list directory, run Python, write file)
  • No few-shot examples or legal playbook, just the assignment, documents, tools, and a grading function
  • An LLM judge scores output against detailed rubrics with written feedback

The loop:

  1. Agent attempts task
  2. LLM judge scores against rubric and provides feedback
  3. A coding agent reads feedback, clusters failures, hypothesizes harness improvements
  4. Agent edits relevant components and reruns the task
  5. Keep changes that improve scores, discard those that don't

Results: Across 12 complex legal tasks, average scores went from 40.8% to 87.7%. Seven tasks finished above 90%. One reached 100%. Five tasks started between 2-7% baseline.

What the agent learned to build:

  • Cross-document review playbooks
  • Stop hooks that validate deliverables before runs end
  • Structured fact sheets for drafting
  • File-conversion pipelines for correct output formats

The loop can self-correct. One early iteration fixed several baseline failures but introduced new failure modes. The next iteration learned from both fixes and regressions. However, some complex tasks showed RL-like instability, oscillating rather than converging.

Baseline vs optimized scores across 12 tasks

Score progression over iterations per task

Source: Niko Grupen / Harvey

Parameter Golf: Agent Swarms on Competitive Optimization

The RLLM Project deployed a swarm of Karpathy's autoresearch agents on their Hive platform to compete in OpenAI's Parameter Golf challenge: train the best language model that fits in 16MB, in 10 minutes on 8xH100 GPUs.

How it worked:

  • Each agent gets an isolated fork of the task repo
  • Agents run autoresearch in a loop: read code, make change, train, evaluate, submit if improved
  • Agents share insights via a feed and build on each other's work
  • 129 runs over 3 days, producing 52 improvements

Two #1 submissions with different strategies:

  1. Go Deep (1.1428 val_bpb): Noticed that int6 quantization was treated as fixed cost. Asked: what if MLP weights could be int5 instead? Int5 leaves more zero bits, which yields better zstd compression (1.88x vs 1.51x) and saves ~1.86 MB in a 16MB budget. Human set the direction, agent found the optimal allocation (scaling BigramHash table to 10,240 rows).

  2. Go Wide (1.1228 val_bpb): Adopted a community 11-layer architecture, then stacked 4 independent techniques: GPTQ-lite quantization, EMA averaging, extended warmdown, and later QAT onset. Each technique was orthogonal to the stack, so they composed predictably.

"The agent saved our attention. We gave it direction." Left autonomous, the swarm does shallow random exploration. With human steering toward promising directions, it excels at parameter sweeps and ablations. See also llm-quantization for the quantization techniques.

OpenAI Parameter Golf on Hive

Final stack: 4 independent gains

Source: RLLM Project

The Broader Pattern

Auto-research connects several threads:

  • Karpathy's autoresearch: Agents editing training code to reduce validation loss
  • Harness engineering (OpenAI): Agent capabilities shaped by environment and feedback loops, not weight updates
  • The "loopy era" of AI: Systems that try, evaluate, and improve on their own
  • context-engineering: ACE is essentially auto-research applied to prompts, evolving the context instead of the code or weights

The common principle: "Humans steer. Agents execute." The human provides direction and evaluation criteria. The agent handles the mechanical work (sweeps, ablations, trial and error). Neither alone produces the best results.

Related Topics

  • context-engineering - ACE as auto-research applied to prompts
  • self-maintaining-agents - Ramp's agents that auto-generate monitors and fixes
  • scaling-autonomous-coding - Multi-agent coordination for large projects
  • llm-quantization - Quantization techniques discovered through auto-research
  • reinforcement-learning-for-llms - RL as formalized self-improvement