Loop agent

AI agent · Data & analytics agent · Data & analytics by Braintrust

AI that helps you improve agents by optimizing prompts, scorers, and datasets automatically.

Loop agent is Braintrust's built-in AI agent designed to help engineering and product teams improve their AI agents. It acts as an autonomous analyst that can query logs, run SQL and code, and investigate production traces to identify recurring failure patterns. Loop can automatically generate better prompts, scorers, and datasets based on natural language descriptions of optimization goals. By connecting synthesized findings to specific trace evidence, it allows teams to turn production failures into regression tests with minimal manual effort. The agent can also be configured to run investig

Languages
English
Investigation Speed
39% faster than manual MCP setups
Insight Accuracy
19 of 22 insight checks passed in benchmarks
Supported Actions
SQL, code execution, dataset creation, prompt iteration

Overview

Loop agent is Braintrust's built-in AI agent designed to help engineering and product teams improve their AI agents. It acts as an autonomous analyst that can query logs, run SQL and code, and investigate production traces to identify recurring failure patterns. Loop can automatically generate better prompts, scorers, and datasets based on natural language descriptions of optimization goals. By connecting synthesized findings to specific trace evidence, it allows teams to turn production failures into regression tests with minimal manual effort. The agent can also be configured to run investigations on a schedule and take approved actions to maintain agent quality.

Analysis and Optimization

  • Data Investigation: The agent can run SQL and code to investigate production data, traces, and logs.
  • Pattern Discovery: Loop identifies recurring issues and opportunities across production traces automatically.
  • Eval Optimization: The agent generates better prompts, scorers, and datasets based on optimization goals.
  • Automated Actions: Loop can repeat investigations on a schedule and take approved actions.

User Interaction

  • Natural Language Interface: Users can ask open-ended questions about logs and experiments in natural language.
  • Evidence-Backed Citations: The agent provides citations that link synthesized findings directly to supporting evidence.
  • Contextual Commands: Loop can be invoked using the @ symbol for context and / for specific commands.

Evaluation Workflow

  • Scorer Generation: Loop builds and validates regression scorers to monitor for specific failure signatures.
  • Dataset Creation: The agent turns production failures and edge cases into versioned eval datasets.
  • SME Calibration: Loop calibrates automated scoring with subject matter expert feedback.

Specifications

Investigation Speed
39% faster than manual MCP setups
Insight Accuracy
19 of 22 insight checks passed in benchmarks
Supported Actions
SQL, code execution, dataset creation, prompt iteration

Added by AntSpace from public sources (braintrust.dev).

Related

More AI agents by Braintrust