Aerospike Connect

Product · Developer tool by Aerospike

Integrate Aerospike with Spark, Kafka, Trino, and more for real-time data pipelines.

Aerospike Connect integrates the Aerospike Database with frameworks like Spark, Kafka, and Trino for real-time streaming and analytics. It is designed for developers building high-throughput data pipelines for AI, machine learning, and real-time decisioning.

Languages
English

Category: Data Integration

Deployment
Cloud, self-hosted, on-premises
Supported Streaming Frameworks
Kafka, Pulsar, JMS, Event Stream Processing
Supported Query Engines
Trino, Spark SQL
Supported AI/ML Frameworks
Spark
Supported Search Engines
Elasticsearch
JMS Provider Compatibility
IBM MQ, Solace, ActiveMQ Artemis, RabbitMQ
  • Massively parallel data loading into Spark DataFrames
  • ANSI SQL query support via Trino connector
  • Real-time streaming with Kafka and Pulsar connectors
  • Low-latency search integration with Elasticsearch

Overview

Aerospike Connect is a suite of integrations designed to connect the Aerospike Database with popular open-source data frameworks and messaging systems. It enables businesses to build data-intensive applications by leveraging Aerospike's high-performance engine alongside tools like Apache Spark, Kafka, and Trino. The product line supports various data processing patterns, including real-time streaming, ETL processing, and on-demand SQL analytics. Developers use these connectors to move data between Aerospike and external systems with low latency and high throughput. Aerospike Connect is particularly suited for organizations building modern data pipelines for AI, machine learning, and real-time decisioning. By integrating with established ecosystems, it allows teams to use familiar BI and analytics tools on Aerospike's multi-model NoSQL data.

Streaming and Messaging

  • Kafka Connector: The Kafka connector enables inbound and outbound data exchange between Aerospike and external systems using Apache Kafka.
  • Pulsar Connector: The Pulsar connector supports Apache Pulsar for exchanging data between Aerospike and external messaging systems.
  • JMS Connector: The JMS connector enables data exchange with JMS providers including IBM MQ, Solace, ActiveMQ Artemis, and RabbitMQ.
  • ESP Connector: The Event Stream Processing (ESP) connector publishes database changes into message brokers via HTTP requests.

Query and Analytics

  • Spark Connector: The Spark connector loads data from Aerospike into Spark streaming or batch DataFrames in a massively parallel manner.
  • Trino Connector: The Trino connector allows users to run ANSI SQL queries using BI tools for on-demand analytics on Aerospike data.
  • Elasticsearch Connector: The Elasticsearch connector integrates Aerospike with Elasticsearch using XDR and Change Notification services for low-latency search.

Use cases

  • Data engineers load Aerospike data into Spark DataFrames for parallel processing in AI and machine learning workflows.
  • Developers use the Kafka connector to exchange inbound and outbound data streams between Aerospike and external messaging systems.
  • Analysts run ANSI SQL queries on Aerospike data using Trino and standard BI tools for on-demand analytics.
  • Architects integrate Aerospike with Elasticsearch to enable low-latency search capabilities across large datasets.

Specifications

Integrations
Spark, Kafka, Pulsar, JMS, Trino, Elasticsearch, Event Stream Processing (ESP)
Deployment
Cloud, self-hosted, on-premises
Supported Streaming Frameworks
Kafka, Pulsar, JMS, Event Stream Processing
Supported Query Engines
Trino, Spark SQL
Supported AI/ML Frameworks
Spark
Supported Search Engines
Elasticsearch
JMS Provider Compatibility
IBM MQ, Solace, ActiveMQ Artemis, RabbitMQ

FAQ

How does Aerospike Connect work with Spark?

Aerospike Connect for Spark loads data from Aerospike into Spark streaming or batch DataFrames in a massively parallel manner, making it suitable for AI/ML and big data analytics.

Which JMS providers are supported?

The JMS connector supports various providers including IBM MQ, Solace, ActiveMQ Artemis, and RabbitMQ for exchanging data between Aerospike and external systems.

Can I run SQL queries on Aerospike data?

Aerospike Connect for Trino allows users to run ANSI SQL queries quickly using BI tools for on-demand analytics on large amounts of Aerospike data.

How does the Elasticsearch integration work?

The Elasticsearch connector uses Aerospike's XDR and Change Notification services to provide low-latency search capabilities by integrating the two systems.

Added by AntSpace from public sources (aerospike.com).

Related

More products by Aerospike