TT-Forge

Product · Developer tool by Tenstorrent

Compile, optimize, and extend AI models for Tenstorrent hardware.

An open-source MLIR-based compiler stack for Tenstorrent hardware that enables developers to compile and optimize models from PyTorch, JAX, and ONNX. It provides hardware-aware compilation and performance analysis tools to maximize inference efficiency on Tensix cores.

Languages
Python, C++

Category: Developer Tools

Programming languages
Python, C++
Integrations
PyTorch, JAX, ONNX, TensorFlow, OpenXLA, TVM, StableHLO
License
Open source
Compiler Framework
MLIR
Supported Frameworks
PyTorch, JAX, ONNX, TensorFlow, TVM, OpenXLA
Hardware Compatibility
Blackhole, Wormhole
  • MLIR-based open-source compiler stack
  • Supports PyTorch, JAX, ONNX, and TensorFlow
  • Hardware-aware compilation for Tensix cores
  • Includes TT-Explorer visual performance analyzer
  • Enables multi-chip execution via TT-XLA bridge

Overview

TT-Forge is Tenstorrent’s open-source, MLIR-based compiler stack designed for developers building AI applications on Tenstorrent hardware. It serves as a bridge between high-level machine learning frameworks—including PyTorch, JAX, ONNX, and TensorFlow—and the underlying Tensix core architecture. The software allows users to compile, optimize, debug, and extend models while ensuring high hardware utilization and efficient memory access. Currently in public beta, TT-Forge includes specialized components like TT-XLA for multi-chip JAX projects and TT-Forge-ONNX for framework-agnostic graph optimization. It integrates with Tenstorrent's broader toolchain, including the TT-Metalium SDK and performance analyzers like TT-Explorer, to provide a complete environment for custom silicon AI development.

Compiler Infrastructure

  • MLIR-Based Architecture: The compiler is built on the MLIR framework, providing a modular and extensible foundation that handles workloads ranging from AI to HPC.
  • Optimized IR Generation: It lowers models into optimized Intermediate Representations (IRs) specifically for execution on Tenstorrent's hardware.
  • Custom Dialect Support: The stack supports custom dialects including TTIR, TTNN, and TTKernel to enable efficient execution and high utilization.
  • Hardware-Aware Compilation: It bridges high-level machine learning frameworks with low-level execution on Tensix cores.

Frameworks and Integration

  • Framework Support: Developers can compile models directly from PyTorch, JAX, and ONNX frameworks.
  • TT-XLA Integration: The PJRT-based bridge allows for compiling and running models from JAX and PyTorch with minimal code changes.
  • TT-Forge-ONNX Frontend: A framework-agnostic frontend is available for optimizing and transforming computational graphs from ONNX and TensorFlow.
  • Multi-Chip Scaling: The compiler supports multi-chip execution for projects using JAX and PyTorch.

Developer Tools and Optimization

  • TT-Explorer Performance Analyzer: A visual performance analyzer allows developers to inspect and optimize model execution on the hardware.
  • TT-Blacksmith Training Recipes: This tool provides ready-to-run training recipes and examples specifically for Tenstorrent hardware.
  • TT-NPE Simulator: A network-on-chip (NoC) simulator and profiler helps developers analyze architectural performance.
  • SDK Compatibility: The compiler is designed to work with the TT-Metalium low-level AI hardware SDK for advanced kernel development.

Use cases

  • Compiler engineers use TT-Forge to lower PyTorch models into optimized IRs for execution on Blackhole processors.
  • Data scientists deploy JAX-based models across multi-chip Tenstorrent configurations using the TT-XLA bridge.
  • AI developers optimize ONNX computational graphs for high-utilization inference on Wormhole-based workstations.
  • Systems engineers profile model execution using TT-Explorer to identify and resolve memory access bottlenecks.

Specifications

Programming languages
Python, C++
Integrations
PyTorch, JAX, ONNX, TensorFlow, OpenXLA, TVM, StableHLO
License
Open source
Compiler Framework
MLIR
Supported Frameworks
PyTorch, JAX, ONNX, TensorFlow, TVM, OpenXLA
Hardware Compatibility
Blackhole, Wormhole
Intermediate Representations
TTIR, TTNN, TTKernel
Compilation Type
Just-in-Time (JIT) via StableHLO for TT-XLA
Development Access
Public Beta
Software Stack Level
High-level AI application development

How it works

  1. Model Ingestion: Developers bring models from frameworks like PyTorch, JAX, or ONNX into the TT-Forge environment.
  2. Compilation and Optimization: The compiler lowers the model into optimized Intermediate Representations (IRs) using custom dialects like TTIR and TTNN.
  3. Hardware Execution: The compiled model is executed on Tenstorrent hardware, such as Wormhole or Blackhole processors, utilizing Tensix cores.
  4. Performance Profiling: Developers use tools like TT-Explorer to analyze performance and refine the model for higher utilization.

FAQ

Which machine learning frameworks does TT-Forge support?

It connects with OpenXLA, MLIR, ONNX, TVM, PyTorch, and TensorFlow to provide a modular foundation for AI workloads.

How does TT-Forge handle multi-chip projects?

TT-XLA is a PJRT-based bridge within the stack that enables multi-chip execution for JAX and PyTorch models with minimal code changes.

What optimization tools are included with TT-Forge?

The toolchain includes TT-Blacksmith for training examples, TT-Explorer for visual performance analysis, and TT-NPE for network-on-chip simulation.

Added by AntSpace from public sources (tenstorrent.com).

Related

More products by Tenstorrent