Luminous Base Sparse
AI model · Language model (LLM) · Text → Text by Aleph Alpha
High-efficiency 80% sparsified AI model for sustainable compute.
Luminous Base Sparse is a 2.6B parameter AI model optimized for extreme efficiency on Graphcore IPU hardware. It delivers a 80% reduction in compute FLOPs and 38% lower energy use compared to dense 13B models while retaining core capabilities.
Model specs
The full spec card (context, pricing per token, modalities and more) on AntSpace. Sign up free to see the full spec card.
- Launched
- 2022-10-17
Category: Natural Language Processing
- Parameters
- 2.6B active (sparsified from 13B)
- Released
- 2022-10-17
- Base Parameters
- 13B
- Sparsified Parameters
- 2.6B
- Weight Reduction
- 80%
- Compute FLOPs Reduction
- 80%
- 80% weight reduction from 13B to 2.6B parameters
- Uses only 20% of processing FLOPs for inference
- 38% lower energy requirement than dense models
- Fits entirely in Graphcore IPU on-chip memory
Overview
Luminous Base Sparse is a highly efficient, sparsified variant of Aleph Alpha's 13 billion parameter Luminous model, developed in partnership with Graphcore. By removing approximately 80% of the model's weights, the parameter count is reduced to 2.6 billion while retaining most of the original model's capabilities. This technical advance allows the model to operate using only 20% of the processing FLOPs and 44% of the memory required by its dense equivalent. It is specifically optimized for Graphcore's IPU architecture, enabling the entire model to be held in ultra-high-speed on-chip memory. The model solves the problem of exponential growth in AI compute demands by providing a sustainable, energy-efficient alternative that requires 38% less power. It is intended for organizations seeking to deploy highly capable AI solutions with minimal compute requirements and lower operational costs.
Efficiency and Sparsification
- 80% Sparsification: The model utilizes parameter pruning to remove approximately 80% of weights from a 13 billion parameter base model.
- CSR Format Representation: Weights are represented using the Compressed Sparse Row (CSR) format to optimize storage and processing.
- Capability Retention: The sparsification process retains most of the capabilities of the original dense model while reducing the parameter count to 2.6 billion.
- Specialized Knowledge Mastery: The model is designed to enable highly specialized sub-models to master dedicated parts of knowledge efficiently.
Performance Gains
- 80% Compute Reduction: The model requires only 20% of the processing FLOPs compared to its dense equivalent for inference.
- 56% Memory Savings: Memory usage is reduced to 44% of the dense model, accounting for the storage of both location and value information for non-zero parameters.
- 38% Lower Energy Use: Luminous Base Sparse has a 38% lower energy requirement than its dense counterpart.
Hardware Integration
- Graphcore IPU Optimization: The model is specifically optimized for the Graphcore IPU architecture and its support for point sparse matrix multiplications.
- Sustainable Scaling: The architecture allows for sustainable linear growth in compute as model capabilities advance.
Use cases
- Developers deploy highly capable models on specialized hardware with minimal compute requirements to reduce operational costs.
- Research teams use sparsification to create specialized sub-models that master dedicated parts of knowledge efficiently.
- Organizations prioritize sustainability by running AI inference with 38% lower energy requirements than standard dense models.
Specifications
- Parameters
- 2.6B active (sparsified from 13B)
- Released
- 2022-10-17
- Base Parameters
- 13B
- Sparsified Parameters
- 2.6B
- Weight Reduction
- 80%
- Compute FLOPs Reduction
- 80%
- Energy Savings
- 38%
- Sparse Format
- Compressed Sparse Row (CSR)
- Target Hardware
- Graphcore IPU-POD16 Classic
How it works
- Parameter Pruning: Aleph Alpha identifies and prunes 80% of the less relevant weights from a dense 13 billion parameter Luminous model.
- Sparse Representation: The remaining important weights are represented using the Compressed Sparse Row (CSR) format to maintain efficiency.
- Model Re-training: The model is re-trained to ensure it retains most of its original capabilities despite the significant reduction in parameters.
- Hardware Deployment: The sparsified model is deployed on specialized hardware like the Graphcore IPU-POD16, utilizing on-chip memory for high-speed inference.
FAQ
What is Luminous Base Sparse?
Luminous Base Sparse is a sparsified version of Aleph Alpha's 13 billion parameter model, reduced to 2.6 billion parameters to improve efficiency. It uses only 20% of the compute FLOPs and 44% of the memory compared to the original dense model.
What hardware does it run on?
The model is specifically optimized for Graphcore's IPU architecture, which supports point sparse matrix multiplications. It is designed to fit entirely within the on-chip memory of an IPU-POD16 Classic for maximum performance.
What are the benefits of using a sparsified model?
Sparsification allows Aleph Alpha to deploy highly capable models with minimal compute requirements. This results in a 38% lower energy requirement and significantly lower operational costs for customers.
How is the model sparsified?
Aleph Alpha uses parameter pruning to remove 80% of the less relevant weights. The model is then re-trained using only the important weights represented in the Compressed Sparse Row (CSR) format.
Added by AntSpace from public sources (aleph-alpha.com).