Crosby RedlineBench
AI model by Crosby
The standard for evaluating frontier models in live contract negotiations.
It provides a standard for measuring legal judgment and surgical redlining across MSAs, DPAs, and NDAs.
Model specs
The full spec card (context, pricing per token, modalities and more) on AntSpace. Sign up free to see the full spec card.
- Launched
- 2026-06-17
- Languages
- English
Category: Legal Software
- Released
- 2026-06-17
- Focus Area
- Commercial Contract Redlining
- Evaluation Metric
- Preference-based scoring
- Contract Types
- MSAs, DPAs, NDAs
- Release Date
- June 17, 2026
- Standard for evaluating frontier models in live contract negotiations
- Measures performance of senior commercial lawyer workflows
- Evaluates surgicality and legal judgment in redlining
- GPT-5.5 score: 50.5% (June 2026)
Overview
Crosby RedlineBench is a specialized evaluation framework and benchmark designed to measure the performance of frontier AI models on commercial legal workflows. It serves as a standard for assessing how well models perform the tasks of senior commercial lawyers during live contract negotiations, particularly for high-stakes documents like MSAs, DPAs, and NDAs. The benchmark moves beyond simple deterministic checks by using preference data to evaluate legal judgment and "surgicality"—the precise, targeted nature of edits. As of June 2026, the benchmark has been used to rank leading models, with GPT-5.5 scoring 50.5% and Claude Fable 5 scoring 47.3%. It is developed by Crosby Intelligence, the research arm of the Crosby law firm.
Evaluation Framework
- Commercial Lawyer Workflow: The benchmark evaluates how frontier models perform the specific workflow of senior commercial lawyers in live contract negotiations.
- Surgicality Evaluation: The framework provides a standard for evaluating surgicality and legal judgment where multiple valid answers may exist.
Model Benchmarks
- GPT-5.5: 50.5%: GPT-5.5 achieved a score of 50.5% on the benchmark as of June 2026.
- Claude Fable 5: 47.3%: Claude Fable 5 recorded a score of 47.3% on the evaluation.
- Gemini 3.5 Flash: 45.1%: Gemini 3.5 Flash achieved a score of 45.1% on the benchmark.
- Claude Opus 4.8: 44.4%: Claude Opus 4.8 scored 44.4% on the evaluation.
Benchmarks
- GPT-5.5: 50.5%; Claude Fable 5: 47.3%; Gemini 3.5 Flash: 45.1%; Claude Opus 4.8: 44.4%
Use cases
- AI researchers use the benchmark to measure how well frontier models handle the nuanced task of redlining commercial contracts.
- Legal technology developers evaluate model performance on high-stakes documents like MSAs and NDAs using preference-based data.
- Model providers compare their systems against senior commercial lawyer workflows to identify gaps in legal reasoning and surgicality.
Specifications
- Released
- 2026-06-17
- Focus Area
- Commercial Contract Redlining
- Evaluation Metric
- Preference-based scoring
- Contract Types
- MSAs, DPAs, NDAs
- Release Date
- June 17, 2026
FAQ
What types of contracts does it cover?
The benchmark measures performance across high-volume commercial contracts including MSAs, DPAs, and NDAs.
Added by AntSpace from public sources (intelligence.crosby.ai).