Lynx: An Open Source Hallucination Evaluation Model
AI model · Safety / moderation · Text → Tables / structured data by Patronus AI
State-of-the-art open-source hallucination detection for RAG reliability.
Lynx is an open-source hallucination detection model by Patronus AI that identifies unsupported or contradictory information in RAG systems.
Model specs
The full spec card (context, pricing per token, modalities and more) on AntSpace. Sign up free to see the full spec card.
- Launched
- 2024-07-11
- Languages
- English
Category: Machine Learning
- Modalities
- Text → Text
- Parameters
- 8B, 70B
- Sizes and variants
- Llama-3-Patronus-Lynx-70B-Instruct, 8B model
- License
- Open Source
- Access
- Hugging Face download, GitHub repository
- Released
- 2024-07-11
- Validated on 15,000 real-world domain samples
- Available in 8B and 70B Llama-3 based variants
- Open-source for public research and access
Overview
Lynx is an open-source hallucination detection model developed by Patronus AI to address the limitations of Large Language Models in Retrieval Augmented Generation (RAG) tasks. It is designed for businesses and researchers who need to identify when an LLM produces information that is unsupported by or contradictory to its retrieved context. The model is capable of advanced reasoning on complex, real-world hallucination scenarios and has been validated against HaluBench, a 15,000-sample benchmark. Lynx is available in multiple sizes, including 8B and 70B versions based on the Llama-3 architecture. It serves as a state-of-the-art LLM-as-a-judge for maintaining AI reliability and correctness.
Hallucination Detection
- RAG Verification: Lynx identifies when LLMs produce information that is unsupported by or contradictory to the retrieved contexts in RAG systems.
- LLM-as-a-Judge: The model serves as a state-of-the-art LLM-as-a-judge for maintaining AI reliability and correctness.
Benchmarks and Evaluation
- HaluBench Validation: The model was evaluated against HaluBench, a comprehensive benchmark consisting of 15,000 samples from various real-world domains.
Benchmarks
- HaluBench: Outperforms GPT-4o and Claude-3-Sonnet
Use cases
- Researchers use Lynx to detect hallucinations in RAG systems by identifying information unsupported by retrieved contexts.
- Developers implement Lynx as an LLM-as-a-judge to evaluate the reliability and correctness of AI-generated outputs.
- Teams benchmark their own models against HaluBench using Lynx to measure performance on real-world hallucination scenarios.
Specifications
- Modalities
- Text → Text
- Parameters
- 8B, 70B
- Sizes and variants
- Llama-3-Patronus-Lynx-70B-Instruct, 8B model
- License
- Open Source
- Access
- Hugging Face download, GitHub repository
- Released
- 2024-07-11
- Base Architecture
- Llama-3
- Model Sizes
- 8B, 70B
- Input Modality
- Text
- Output Modality
- Text (Evaluation/Reasoning)
- Benchmark Samples
- 15,000 (HaluBench)
- License Type
- Open Source
FAQ
What is Lynx?
Lynx is an open-source hallucination detection LLM capable of advanced reasoning on challenging real-world hallucination scenarios.
How does Lynx improve RAG systems?
Lynx is designed to identify when LLMs produce information that is unsupported or contradictory to the retrieved contexts in Retrieval Augmented Generation (RAG) systems.
What model sizes are available for Lynx?
Lynx is available in 8B and 70B variants based on the Llama-3 architecture.
Added by AntSpace from public sources (patronus.ai).