AI Agents & Agentic Engineering
  • Context Engineering
  • Building AI Agents
  • Agentic Engineering Best Practices
  • Auto-Research and Self-Improving Agents
  • Self-Maintaining Agents
  • Scaling Long-Running Autonomous Coding
  • Data Agents
  • Agent Memory and Harness Ownership
LLM Internals & Training
  • Reinforcement Learning for LLMs
  • Cursor Composer 2
  • Training-Free GRPO
  • Prompt Repetition for Non-Reasoning LLMs
  • LLM Quantization
  • LLM Knowledge Bases
AI Safety & Security
  • Constitutional Classifiers
  • Project Glasswing
Graphics & Low-Level Programming
  • Shaders and GPU Programming
Personal Development & Mindset
  • Deep Focus and Productivity
  • Identity and Life Design
  • Self-Competition and Entrepreneurship

Training-Free GRPO

Tencent researchers introduced Training-Free Group Relative Policy Optimization (TF-GRPO), a method that achieves RL-like improvements in LLM reasoning without any gradient updates or fine-tuning. The model learns from its own experiences stored as evolving memory.

Training-Free GRPO architecture

Source: Thread by @rryssf_
Attachment: Training-Free Group Relative Policy Optimization.pdf

How It Works

Instead of updating model weights, TF-GRPO:

  1. Has the model generate multiple rollouts for a problem
  2. Introspects on its own rollouts, analyzing what worked and what didn't
  3. Extracts successful strategies as "semantic advantage": natural language descriptions of effective reasoning patterns
  4. Stores these as evolving memory that refines how the model thinks on future problems

The model is teaching itself how to think, not just what to output. It is a form of reinforcement expressed entirely in language rather than gradients.

Key Results

  • No fine-tuning, no gradients. Zero parameter updates.
  • Only 100 examples needed
  • Outperforms $10,000+ RL setups on reasoning benchmarks
  • Total cost: $18

Connection to Broader Trends

TF-GRPO sits at the intersection of several converging ideas:

  • context-engineering: Like ACE (Agentic Context Engineering), TF-GRPO improves the model through context manipulation rather than weight updates. ACE evolves prompts; TF-GRPO evolves a reasoning memory.
  • auto-research: The self-improvement loop is the same pattern (try, evaluate, store what works) but without any training infrastructure.
  • reinforcement-learning-for-llms: TF-GRPO achieves RL-like effects at a fraction of the cost and complexity. If this scales, it could make traditional RL post-training less necessary for certain tasks.

The paper suggests we may be entering a "training-free era of AI optimization" where inference-time strategies replace expensive training loops.

Related Topics

  • context-engineering - Self-improving context as an alternative to weight updates
  • reinforcement-learning-for-llms - Traditional RL approach that TF-GRPO aims to replace
  • prompt-repetition - Another training-free technique for improving LLM performance
  • auto-research - The broader paradigm of self-improving AI systems