liquid-finetune
Product · Developer tool by Liquid AI
Official fine-tuning framework for Liquid Foundation Models.
liquid-finetune is Liquid AI's official framework for customizing Liquid Foundation Models on specialized datasets. It allows developers to perform multi-task fine-tuning for vision and language tasks using serverless GPU infrastructure.
- Languages
- English
Category: Machine Learning
- Programming languages
- Python
- Editors and tools
- Terminal / CLI
- Integrations
- Modal, Hugging Face, Weights & Biases (WandB), GitHub
- Deployment
- Cloud (serverless GPU) and local development environments
- Supported Models
- Liquid Foundation Models (e.g., LFM2.5-VL-450M)
- Primary Data Format
- JSONL
- Official fine-tuning framework for Liquid Foundation Models (LFMs)
- Supports multi-task VQA, grounding, and captioning training
- Integrated with serverless GPU clouds for infrastructure-free training
- Automated benchmarking with CIDEr, BLEU, and IoU metrics
Overview
liquid-finetune is Liquid AI's official fine-tuning framework designed for developers to customize Liquid Foundation Models (LFMs) for specialized tasks. It enables organizations to adapt models to proprietary data for applications such as satellite imagery analysis, visual question answering, and object localization. The framework is built to integrate with serverless GPU infrastructure, specifically Modal, allowing users to execute heavy training jobs on NVIDIA H100 GPUs without managing their own hardware. It supports structured data formats like JSONL and provides automated benchmarking during the training process to track metrics like IoU, CIDEr, and BLEU. By using liquid-finetune, developers can create private, high-performance versions of LFMs tailored to specific domain constraints.
Training and Customization
- Model Customization: The framework allows developers to fine-tune Liquid Foundation Models on specialized datasets for tasks like satellite imagery analysis.
- Multimodal Fine-Tuning: It supports multi-task training for vision-language models, including visual question answering, visual grounding, and captioning.
- Cloud Infrastructure Integration: The tool integrates with serverless GPU clouds like Modal to run heavy data preparation and training without managing local infrastructure.
- High-Performance Training: Training jobs can be executed on high-performance hardware such as NVIDIA H100 GPUs.
Developer Workflow
- JSONL Data Support: The framework uses JSONL as the primary data format for training and evaluation datasets.
- Normalized Bounding Boxes: Grounding tasks utilize a JSON bounding box format with 0-1 normalized coordinates to match LFM pretraining.
- Command-Line Interface: It includes a CLI for submitting jobs and streaming logs from cloud training environments to a local terminal.
- Checkpoint Management: Checkpoints are automatically saved to remote volumes and can be retrieved to local machines for deployment.
Evaluation and Monitoring
- Automated Benchmarking: Benchmarks run automatically during training at specified evaluation steps to track model progress.
- Standardized Metrics: It supports standard metrics such as short_answer for VQA, grounding_iou for object detection, and CIDEr or BLEU for captioning.
- Experiment Tracking: Developers can enable experiment tracking by integrating with tools like Weights & Biases (WandB).
- Standalone Evaluation: A full standalone evaluation mode allows for testing checkpoints on complete datasets without retraining.
Use cases
- Researchers fine-tune vision-language models on satellite imagery for automated object detection and scene captioning.
- Developers adapt LFMs to specialized domain data to improve visual question answering accuracy for industrial inspections.
- Organizations create private model checkpoints by training on proprietary datasets within secure, serverless GPU environments.
- Engineers use the framework to localize specific objects in images by training models with normalized JSON bounding boxes.
Specifications
- Programming languages
- Python
- Editors and tools
- Terminal / CLI
- Integrations
- Modal, Hugging Face, Weights & Biases (WandB), GitHub
- Deployment
- Cloud (serverless GPU) and local development environments
- Supported Models
- Liquid Foundation Models (e.g., LFM2.5-VL-450M)
- Primary Data Format
- JSONL
- Training Hardware
- NVIDIA H100
- VQA Metric
- short_answer (case-insensitive substring match)
- Grounding Metric
- grounding_iou (IoU@0.5 threshold)
- Captioning Metrics
- CIDEr, BLEU
- Coordinate System
- 0-1 normalized bounding boxes
- Experiment Tracking
- Weights & Biases (WandB) support
How it works
- Data Preparation: Developers prepare their datasets, such as VRSBench, by converting them into the required JSONL format using provided scripts.
- Training Execution: The framework clones from GitHub and initiates training jobs on cloud-based H100 GPUs via a YAML configuration file.
- Automated Evaluation: During training, the system automatically runs benchmarks and logs metrics to track task quality.
- Checkpoint Retrieval: Once training is complete, developers download the resulting model checkpoints from the cloud volume to their local environment for deployment.
FAQ
What data formats does liquid-finetune support?
The framework requires data to be in JSONL format. For grounding tasks, it uses a JSON bounding box format with coordinates normalized to a 0-1 range.
What hardware is required to use liquid-finetune?
It is designed to work with serverless GPU clouds like Modal, which allows users to run data preparation and training on H100 GPUs without managing their own hardware. It also supports local data preparation for development.
Can I track model performance during fine-tuning?
Yes, benchmarks run automatically during the training process at every evaluation step. Supported metrics include short_answer for VQA, grounding_iou for object detection, and CIDEr or BLEU for captioning.
How do I retrieve my fine-tuned model checkpoints?
Checkpoints are saved to a remote volume (such as a Modal volume) during training. They can be listed and downloaded to a local machine using command-line tools like 'modal volume get'.
Added by AntSpace from public sources (docs.liquid.ai).