Lynx: An Open Source Hallucination Evaluation Model

AI model · Safety / moderation · Text → Tables / structured data by Patronus AI

State-of-the-art open-source hallucination detection for RAG reliability.

Lynx is an open-source hallucination detection model by Patronus AI that identifies unsupported or contradictory information in RAG systems.

Model specs

The full spec card (context, pricing per token, modalities and more) on AntSpace. Sign up free to see the full spec card.

Launched
2024-07-11
Languages
English

Category: Machine Learning

Modalities
Text → Text
Parameters
8B, 70B
Sizes and variants
Llama-3-Patronus-Lynx-70B-Instruct, 8B model
License
Open Source
Access
Hugging Face download, GitHub repository
Released
2024-07-11
  • Validated on 15,000 real-world domain samples
  • Available in 8B and 70B Llama-3 based variants
  • Open-source for public research and access

Overview

Lynx is an open-source hallucination detection model developed by Patronus AI to address the limitations of Large Language Models in Retrieval Augmented Generation (RAG) tasks. It is designed for businesses and researchers who need to identify when an LLM produces information that is unsupported by or contradictory to its retrieved context. The model is capable of advanced reasoning on complex, real-world hallucination scenarios and has been validated against HaluBench, a 15,000-sample benchmark. Lynx is available in multiple sizes, including 8B and 70B versions based on the Llama-3 architecture. It serves as a state-of-the-art LLM-as-a-judge for maintaining AI reliability and correctness.

Hallucination Detection

  • RAG Verification: Lynx identifies when LLMs produce information that is unsupported by or contradictory to the retrieved contexts in RAG systems.
  • LLM-as-a-Judge: The model serves as a state-of-the-art LLM-as-a-judge for maintaining AI reliability and correctness.

Benchmarks and Evaluation

  • HaluBench Validation: The model was evaluated against HaluBench, a comprehensive benchmark consisting of 15,000 samples from various real-world domains.

Benchmarks

  • HaluBench: Outperforms GPT-4o and Claude-3-Sonnet

Use cases

  • Researchers use Lynx to detect hallucinations in RAG systems by identifying information unsupported by retrieved contexts.
  • Developers implement Lynx as an LLM-as-a-judge to evaluate the reliability and correctness of AI-generated outputs.
  • Teams benchmark their own models against HaluBench using Lynx to measure performance on real-world hallucination scenarios.

Specifications

Modalities
Text → Text
Parameters
8B, 70B
Sizes and variants
Llama-3-Patronus-Lynx-70B-Instruct, 8B model
License
Open Source
Access
Hugging Face download, GitHub repository
Released
2024-07-11
Base Architecture
Llama-3
Model Sizes
8B, 70B
Input Modality
Text
Output Modality
Text (Evaluation/Reasoning)
Benchmark Samples
15,000 (HaluBench)
License Type
Open Source

FAQ

What is Lynx?

Lynx is an open-source hallucination detection LLM capable of advanced reasoning on challenging real-world hallucination scenarios.

How does Lynx improve RAG systems?

Lynx is designed to identify when LLMs produce information that is unsupported or contradictory to the retrieved contexts in Retrieval Augmented Generation (RAG) systems.

What model sizes are available for Lynx?

Lynx is available in 8B and 70B variants based on the Llama-3 architecture.

Added by AntSpace from public sources (patronus.ai).

Related

More AI models by Patronus AI