Learning About Recursive Self-improvement — Looking For Feedback On My Experimental Framework

Hi everyone!

I recently started learning about recursive self-improvement (RSI), and I’ve been trying to understand the ideas by building an experimental open-source framework called Gear. I’m still learning, so I’d really appreciate feedback on the approach and what I could improve.

The basic idea is to let an agent learn from its task failures through an iterative loop:

1. Run the agent on a set of tasks with clear evaluation criteria.
2. Have a “meta agent” review the execution traces and evaluation results.
3. Propose changes to the task agent’s instructions, tools, or workflows.
4. Evaluate the changes and keep the versions that perform better.

The current implementation focuses on improving the agent’s harness—the instructions, tools, and workflows around the model. I’m also exploring model training from task feedback, but that part is still experimental.

Here’s the project: Gear on GitHub

A few things I’d love advice on:

• Evaluation: How would you check whether improvements generalize to new tasks, and avoid overfitting to the evaluation set?
• Design: Are there parts of this loop you would simplify or approach differently?
• Learning resources: Are there papers or existing projects I should study to better understand this direction?

I’m particularly interested in where my understanding of RSI might be incomplete, and whether “iterative agent optimization” is a more accurate description of what I’ve built so far.

Even feedback on one small part would be helpful. Thanks for taking a look!

4 posts - 2 participants

Read full topic