AI Agents & Agentic Engineering
  • Context Engineering
  • Building AI Agents
  • Agentic Engineering Best Practices
  • Auto-Research and Self-Improving Agents
  • Self-Maintaining Agents
  • Scaling Long-Running Autonomous Coding
  • Data Agents
  • Agent Memory and Harness Ownership
LLM Internals & Training
  • Reinforcement Learning for LLMs
  • Cursor Composer 2
  • Training-Free GRPO
  • Prompt Repetition for Non-Reasoning LLMs
  • LLM Quantization
  • LLM Knowledge Bases
AI Safety & Security
  • Constitutional Classifiers
  • Project Glasswing
Graphics & Low-Level Programming
  • Shaders and GPU Programming
Personal Development & Mindset
  • Deep Focus and Productivity
  • Identity and Life Design
  • Self-Competition and Entrepreneurship

Self-Maintaining Agents

Ramp built an agentic system to maintain their Sheets product that continuously monitors production, triages alerts, and proposes fixes, with no code merged without engineer review. The system runs on a thousand AI-generated monitors, one for every 75 lines of code.

Source: Ramp Labs / Alex Levinson

Why Agents for Maintenance

  • Agents are cheap, infinitely patient, and easy to parallelize. Exhaustive monitoring becomes feasible at production scale.
  • Teams dislike owning observability. Offloading grunt work lets engineers ship faster.
  • Strong observability and QA mean fewer bugs, less downtime, better customer experience.

Ramp Inspect

The system uses Ramp Inspect, their internal background coding agent. Each session spins up a full sandboxed dev environment, allowing real API requests, tests, and end-to-end bug reproduction against live code. This interactivity is critical because subtle failure modes are rarely apparent from static code review.

Evolution of the Approach

Attempt 1: Scheduled Nightly QA

An agent ran every night with broad instructions: sanity-test core features, stress-test recent PRs, probe for latent bugs. It found real bugs daily and even pushed fixes before engineers noticed regressions.

QA agent finds a race condition and puts up a fix

Limitations: With no specific mission, the agent followed the same paths every run: same files, same features, same tests. It caught high-radius issues but missed narrow, situational bugs. Frontier models cannot yet synthesize a large codebase with a large observability surface and determine what needs attention.

Attempt 2: Monitor-Driven Maintenance

Switched to Datadog monitors directing agents at specific issues:

  1. On PR merge: An agent reads the diff and generates monitors instrumenting the new code
  2. When a monitor fires: A Datadog webhook kicks off a new agent with alert context. The agent reproduces the issue in its sandbox, pushes a fix, and notifies on Slack.

Results in the first week: Caught 40 real bugs, each within minutes of a user triggering the issue. In one case, the system flagged a bug before an internal user's Slack message about it even landed.

Filtering Noise

Auto-generated monitors have bad thresholds. The solution: a triage step where the agent first assesses scope:

  • Real issue: push a fix and post to Slack
  • Noise: tune or delete the monitor

Agent-generated composite monitor

To prevent duplicate alerts, state is stored on the monitor itself. When an agent pushes a fix, it appends the PR link to the monitor description. Subsequent agents see the link and stand down.

Key Learnings

  • "Detect everything, notify selectively." Watch every signal, but each alert reaching a human must mean something. Teams ignore noisy monitors and noisy agents.
  • Delegate to the agent. Let it scope the problem, judge impact, make changes, and filter noise.
  • Sandboxed reproduction improves results. The agent reproduces the failure against live code and only pushes a fix once that reproduction test passes.
  • Model choice matters. GPT-5 models are thorough debuggers, but Opus 4.6 was a more accurate triage evaluator, specifically better at filtering noisy alerts.
  • Tight observability breeds customer empathy. When every slow load or bad output fires a notification, the team feels the product the same way users do.
  • Keep your existing stack. Auto-generated monitors are not yet reliable enough as your only line of defense. As models improve, that will change.

Scaling from 10 to 1000+ monitors

The Pattern

This is auto-research applied to production monitoring: agents generate their own instrumentation, evaluate their own alerts, and refine their own thresholds. The system scales from 10 hand-written monitors to 1,000+ AI-generated ones, creating "a tight mesh over the exact shape of the code."

Related Topics

  • auto-research - The broader paradigm of self-improving agent systems
  • scaling-autonomous-coding - Multi-agent coordination at larger scale
  • agentic-engineering-best-practices - Principles for effective agent use