Discover
Product · SaaS / cloud software by Braintrust
Find behavior that needs attention and turn it into measurable improvements.
Discover is Braintrust's intelligence layer that automatically surfaces and investigates agent behaviors in production. It helps AI teams turn recurring production failures into measurable improvements through automated pattern discovery and natural language trace analysis.
- Price
- From 0 USD
Category: Observability & Monitoring
- Deployment
- SaaS, BYOC, and Self-hosted options available
- Platforms
- Web, API, CLI
- APIs and SDKs
- Native SDKs for Python, TypeScript, Go, Ruby, C#, and.NET
- Sign-in and administration
- SSO/SAML, RBAC, and granular project-level permissions
- Included Processed Data (Starter)
- 1 GB / month
- Included Scores (Starter)
- 10,000 / month
- Automatically surface important agent behavior from production data
- Turn production traces into eval datasets with one click
- Natural language investigation across traces, logs, and experiments
- Identify recurring issues with evidence-backed Pattern discovery
Overview
Discover is Braintrust's active observability and intelligence layer designed for teams running AI agents in production. It solves the problem of manual log review by automatically surfacing important agent behaviors and recurring failure patterns from millions of production traces. The platform uses the Loop AI agent to perform natural language investigations, allowing users to query logs, run SQL, and execute code to diagnose issues. Discover enables a closed-loop workflow where production failures are converted into eval datasets and regression tests with a single click. By organizing broad themes through topic classification, it helps engineering and product teams identify high-priority quality issues at scale. The system integrates directly with the Braintrust evaluation suite to turn discovered insights into measurable improvements.
Intelligence and Discovery
- Pattern Discovery: The platform continuously applies intelligence to production data to identify recurring issues and opportunities across agent traces.
- Topic Classification: Discover uses topic classification to organize broad themes across millions of logs, such as refund requests or order tracking.
- Natural Language Investigation: Users can ask open-ended questions across traces, logs, experiments, and datasets using natural language.
- Evidence-backed Analysis: The system identifies high-confidence examples of specific behaviors, such as work completed in tools but not communicated to the user.
Workflow Integration
- Trace to Dataset Conversion: Production traces and identified failures can be converted into versioned eval datasets with a single click.
- Automated Scorer Generation: The platform helps build and validate regression scorers based on discovered production patterns.
- Behavioral Monitoring: Users can configure ongoing monitoring and automations for specific behaviors identified during discovery.
- Evaluator Calibration: The system allows for testing evaluators against representative traces before saving them to the project.
Investigation Tools
- AI-Powered Analysis: The Loop AI agent can run SQL and code to verify findings against additional evidence.
- Scheduled InvestigationsOps: Investigations can be set up to repeat on a schedule to track emerging trends or recurring problems.
- Failure Diagnosis: Discover provides natural language trace investigation to diagnose failures directly from production logs.
- IDE Integration: The platform supports querying logs and updating prompts directly from the IDE via the MCP server.
Use cases
- Engineering teams identify recurring production failures and turn them into regression datasets for CI/CD gates.
- Product managers use topic classification to understand broad themes in customer interactions and prioritize feature improvements.
- AI developers use natural language queries to investigate why agents are making specific tool-call errors in production.
- Teams track the cost and token usage of coding agents across different models and branches to optimize spend.
Specifications
- Deployment
- SaaS, BYOC, and Self-hosted options available
- Platforms
- Web, API, CLI
- Integrations
- OpenAI, Anthropic, Google, AWS, Cloudflare, GitHub, Slack, S3, MCP
- APIs and SDKs
- Native SDKs for Python, TypeScript, Go, Ruby, C#, and.NET
- Sign-in and administration
- SSO/SAML, RBAC, and granular project-level permissions
- Limits
- Starter includes 1 GB processed data and 10k scores per month; Pro includes 5 GB and 50k scores
- Languages
- Supports multilingual workspaces including Japanese, Korean, and English
- Included Processed Data (Starter)
- 1 GB / month
- Included Scores (Starter)
- 10,000 / month
- Data Retention (Starter)
- 14 days
- Included Processed Data (Pro)
- 5 GB / month
- Included Scores (Pro)
- 50,000 / month
- Data Retention (Pro)
- 30 days
- Query Engine
- Nitro (Brainstore)
- AI Analysis Agent
- Loop agent
How it works
- Surface Patterns: Braintrust continuously applies intelligence to production traces to identify recurring issues and opportunities.
- Investigate Evidence: Use the Loop AI agent to ask open-ended questions and verify findings against additional evidence.
- Build Evals: Convert high-confidence examples of production failures into regression datasets and scorers.
- Monitor Improvements: Configure ongoing monitoring and automations to ensure improvements are maintained in production.
Plans
- Starter: 0 month
- 1 GB Processed Data: Includes 1 GB of processed data per month for ingestion.
- 10k Scores: Includes 10,000 scored outputs from LLM-as-a-judge or code scorers per month.
- 14-day Retention: Provides 14-day data retention for traces and logs.
- Unlimited Users & Projects: Supports unlimited users, projects, datasets, and experiments.
- Pro: 249 month
- 5 GB Processed Data Add-on: Includes 5 GB of processed data per month for ingestion.
- 50k Scores: Includes 50,000 scored outputs per month.
- 30-day Retention: Provides 30-day data retention, with options for longer storage at $0.50/GB/mo.
- Custom Charts & RBAC: Enables custom charts, dashboards, and role-based access control.
- Enterprise
- Custom Retention & ExportBase: Custom data retention and export policies to meet compliance requirements.
- On-prem Deployment: Support for on-prem or hosted deployment for high volume or privacy-sensitive data.
FAQ
What does processed data mean?
Processed data refers to the data ingested when you send it to Braintrust. The Starter plan includes 1 GB per month, while the Pro plan includes 5 GB per month.
Added by AntSpace from public sources (braintrust.dev).