TinySwallow-1.5B

AI model · Language model (LLM) · Text → Text by Sakana AI

State-of-the-art small-scale Japanese language model

A compact 1.5B parameter Japanese language model achieving state-of-the-art performance for its size through the TAID training method.

Model specs

The full spec card (context, pricing per token, modalities and more) on AntSpace. Sign up free to see the full spec card.

Languages
Japanese

Category: Natural Language Processing

Parameters
1.5B
Languages
Japanese
Model Size
1.5B parameters
Primary Language
Japanese
Training Method
TAID
  • 1.5B parameter Japanese language model
  • State-of-the-art performance for its size class
  • Developed using the TAID training method
  • ICLR Spotlight Paper research foundation

Overview

TinySwallow-1.5B is a small-scale Japanese language model developed by Sakana AI. It is designed for developers and businesses that require high-performance Japanese language capabilities in a compact 1.5B parameter format. Its small footprint makes it suitable for efficient deployment on hardware with limited resources. The technical significance of the model's development was recognized as a Spotlight Paper at ICLR. TinySwallow-1.5B aims to provide a balance of speed and linguistic accuracy for Japanese-specific applications.

Efficiency and Localization

  • SOTA Performance: The model achieves state-of-the-art performance among Japanese language models of similar size.
  • Japanese Optimization: TinySwallow-1.5B is specifically optimized for the Japanese language and context.
  • Small Footprint: The model uses a compact 1.5B parameter format designed for efficient deployment on limited hardware.

Technical Innovation

  • TAID Training Method: The model was developed using the Task-Agnostic Model Injection and Distillation (TAID) method.
  • ICLR Spotlight Research: The research behind the model was accepted as a Spotlight Paper at ICLR.

Benchmarks

  • SOTA performance among models of similar size

Use cases

  • Developers deploy the model on limited hardware to provide efficient Japanese language processing.
  • Businesses use the model for Japanese-specific applications requiring a balance of speed and accuracy.

Specifications

Parameters
1.5B
Languages
Japanese
Model Size
1.5B parameters
Primary Language
Japanese
Training Method
TAID

Added by AntSpace from public sources (sakana.ai).

Related

More AI models by Sakana AI