Product Features
Apache Ignite in-memory computing platform comprises the following set of components:
Ignite Feature Summary
The following is a summary of list of features currently available in Apache Ignite.
Architecture
Durable Memory | |
---|---|
Memory Architecture |
Apache Ignite is based on the Durable Memory architecture that allows storing and processing data and indexes both in memory and on disk. |
Persistence |
Ignite Persistent Store is a distributed ACID and SQL-compliant disk store that transparently integrates with Ignite's durable memory as an optional disk layer storing data and indexes on SSD, Flash, 3D XPoint, and other types of non-volatile storages. |
MARSHALLING | |
Binary Protocol |
Starting from v1.5 Ignite introduced a new concept of storing data in caches, called BinaryObjects. It allows you to:
|
HIGH AVAILABILITY | |
Self-Healing Cluster |
Ignite cluster can self-heal, where clients automatically reconnect in case of failures. |
Distributed SQL
DISTRIBUTED QUERIES | |
---|---|
SQL Queries |
Ignite supports the standard SQL syntax (ANSI 99) to query the cache. You can use any SQL function, aggregation, or grouping. |
Distributed DDL |
Apache Ignite supports using Data Definition Language (DDL) statements for creating and removing SQL indexes at runtime. |
Distributed DML |
Ignite supports DML statements like INSERT, MERGE, UPDATE, and DELETE to modify cached data. |
Collocated Joins |
Ignite supports SQL joins on collocated data stored across multiple caches. |
Non-Collocated Joins |
Ignite offers non-collocated SQL joins for cases where it is extremely difficult to collocate all the data. |
Query Indexing |
For SQL queries, ignites supports in-memory indexing, so all the data lookups are extremely fast. |
Query Consistency |
Ignite provides full query consistency. Updates that happened after the query execution started do not affect the query result. |
Query Fault-Tolerance |
Ignite queries are fault-tolerant, i.e. query result is always consistent and is not affected by cluster topology changes. |
Geospacial Support |
Ignite supports querying and indexing geometry data types such as points, lines, and polygons. |
SQL DRIVERS | |
JDBC Driver |
Ignite JDBC driver allows users to retrieve and modify data, stored in Ignite caches, using standard SQL/DML commands and JDBC API. |
ODBC Driver |
Ignite ODBC driver allows users to retrieve and modify data, stored in Ignite caches, using standard SQL/DML comands and ODBC API. |
Tooling | |
SQL Tooling |
You can process data stored in the Ignite cluster using a SQL tool of your choice - DBeaver, Pentaho, Tableau, Web Console, Apache Zeppelin. |
STANDARDS | |
SQL, JDBC, ODBC |
Ignite supports retrieving distributed data from cache using standard SQL queries, JDBC and ODBC API. |
INTEGRATIONS | |
Apache Zeppelin |
Apache Ignite SQL Grid defines a set of APIs that allows integrating with Apache Zeppelin. |
Tableau |
Using Ignite ODBC driver, you can connect to the Ignite cluster from Tableau to analyze cached data. |
PHP Data Objects |
Using Ignite ODBC driver and PHP Data Objects, you can connect to the Ignite cluster from PHP side. |
Data Grid
DISTRIBUTED CACHING | |
---|---|
Key-Value Store |
Ignite data grid is an in-memory key-value store which can be viewed as a distributed partitioned hash map. |
SQL Support |
In addition to key-value API, Ignite data grid has complete SQL support, including primary and secondary indexes and distributed JOINs. |
Partitioning & Replication |
Ignite can be configured to store the data in memory in either partitioned or replicated manner. |
Client-Side (Near) Cache |
Near cache is local client-side cache that stores the most recently and most frequently accessed data. |
Collocated Processing |
Ignite allows executing any native Java, C++, and .NET/C# code directly on the server-side, close to the data, in collocated fashion. |
DISTRIBUTED QUERIES | |
Continuous Queries |
Ignite allows you to execute a query and continue to get notified about the data changes that fall into your query filter. |
Query Consistency |
Ignite provides full query consistency. Updates that happened after the query execution started do not affect the query result. |
Query Fault-Tolerance |
Ignite queries are fault-tolerant, i.e. query result is always consistent and is not affected by cluster topology changes. |
PERSISTENCE | |
Write-Through |
Write-Through mode allows updating the data in the database. |
Read-Through |
Read-Through mode allows reading the data from the database. |
Write-Behind Caching |
Ignite provides an option to asynchronously perform updates to the database via Write-Behind Caching. |
Automatic Persistence |
Automatically connect to the underlying database and generate XML OR-mapping configuration and Java domain model POJOs. |
STANDARDS | |
JCache |
Ignite is a 100% compliant implementation of JCache (JSR 107) specification. |
XA/JTA |
Ignite can be configured with a Java Transaction API (JTA) transaction manager lookup class. |
OSGI Support | |
INTEGRATIONS | |
Database Integration |
Ignite can automatically integrate with external databases - RDBMS, NoSQL, and HDFS. |
Hibernate L2 Cache |
Ignite In-Memory Data Grid can be easily plugged in as a second level(L2) cache which is visible to all sessions. |
Web Session Clustering |
Ignite is capable of caching web sessions of all Java Servlet containers that follow Java Servlet 3.0 Specification. |
Spring Caching |
Ignite provides an implementation of SpringCacheAbstraction - an annotation-based way to enable caching for Java methods. |
Spring Data |
Ignite implements Spring Data |
Compute Grid
DISTRIBUTED COMPUTE | |
---|---|
Distributed Closure Execution |
Ignite compute grid allows to broadcast and load-balance any closure within the cluster or a cluster group. |
Clustered Executor Service |
Ignite provides a cluster-enabled implementation of ExecutorService. |
MapReduce and ForkJoin |
Ignite provides a ComputeTask API, which is a light-weight MapReduce (or ForkJoin) implementation. |
Collocation of Compute & Data |
Collocation of computations with data allow for minimizing data serialization within the network. |
Checkpointing |
Checkpointing can be useful when long running jobs need to store some intermediate state to protect from node failures. |
Fault-Tolerance |
In case of a node crash, Ignite jobs are automatically transferred to other available nodes for re-execution. |
Load Balancing |
Ignite balances job distribution among cluster nodes, making sure that every node in the cluster is equally loaded. |
Job Scheduling |
Ignite provides a fine-grained control over jobs ordering once they arrive to a specific cluster node for execution. |
Streaming Features
STREAMING AND CEP | |
---|---|
Data Streamers |
Ignite allows you to process continuous never-ending streams of data into Ignite caches, in scalable and fault-tolerant fashion. |
Data Loading |
Data streamers can be used to load large amounts of data into Ignite cahes. They can be used for initial data loading from a 3rd party database or another source. |
Collocated Processing |
Ignite stream receivers allow you to react to the streamed data in collocated fashion, directly on the nodes where it will be cached. |
Continuous Queries |
Ignite allows you to execute a query and continue to get notified about the data changes that fall into your query filter. |
JMS Data Streamer |
Ignite JMS Data Streamer consumes messages from JMS brokers and inserts them into Ignite caches. |
Apache Flume Sink |
IgniteSink is a Flume sink that extracts events from an associated Flume channel and injects into an Ignite cache. |
MQTT Streamer |
Ignite MQTT Streamer consumes messages from a MQTT topic and feeds transformed key-value pairs into an Ignite cache. |
Twitter Streamer |
Ignite Twitter Streamer consumes messages from a Twitter Streaming API and inserts them into an Ignite cache. |
Apache Kafka Streamer |
Ignite Kafka Data Streamer consumes messages for a given Kafka Topic from Kafka Broker and inserts them into an Ignite cache. |
Apache Camel streamer |
Ignite Camel streamer consumes messages from an Apache Camel consumer endpoint and feeds them into an Ignite cache. |
Apache Storm Streamer |
Ignite Storm Streamer consumes messages from an Apache Storm consumer endpoint and feeds them into an Ignite cache. |
Apache Flink Streamer |
Ignite Flink Streamer consumes messages from an Apache Flink consumer endpoint and feeds them into an Ignite cache. |
Apache RocketMQ Streamer |
Ignite RocketMQ Streamer consumes messages from an Apache RocketMQ consumer endpoint and feeds them into an Ignite cache. |
ZeroMQ Streamer |
Ignite ZeroMQ Streamer consumes messages from a ZeroMQ consumer endpoint and feeds them into an Ignite cache. |
Data Structures Features
DISTRIBUTED DATA STRUCTURES | |
---|---|
Queue and Set |
Ignite provides an implementation of a fast Distributed Blocking Queue and Distributed Set. |
Collocated vs. Non-Collocated |
Queues and Sets in Ignite can be deployed in a collocated or non-collocated mode. |
Bounded Queues |
Bounded queues allow users to have queues with predefined maximum size which gives a better control over the overall cache capacity. |
Atomic Types |
Ignite supports distributed AtomicLong and AtomicReference. |
CountDownLatch |
Ignite CountDownLatch allows you to synchronize jobs on all Ignite nodes. |
IdGenerator |
Ignite distributed atomic sequence helps you to sequentially generate unique Ids across the cluster. |
Semaphore |
Ignite's distributed semaphore implementation and behavior is similar to java.util.concurrent.Semaphore. |
Messaging & Events
MESSAGING AND EVENTS | |
---|---|
Topic Based Messaging |
Ignite distributed messaging allows for topic based cluster-wide communication between all nodes. |
Point-to-Point Messaging |
Ignite messages can be sent to either a group of nodes or to an individual node. |
Ordered vs. Unordered |
Ignite supports receiving messages in an ordered as well as unordered manner. |
Event Notifications |
Ignite distributed events functionality allows applications to receive notifications when a variety of events occur within the cluster. |
Local vs. Remote Events |
Applications can get notified for task executions, read, write or query operations occurring on local or remote nodes within the cluster. |
Automatic Batching |
In Ignite, event notifications can be grouped together and sent in batches or timely intervals. |
Service Grid Features
SERVICE GRID | |
---|---|
User Defined Services |
Ignite Service Grid allows for deployments of arbitrary user-defined services on the cluster. |
Cluster Singletons |
Ignite allows to deploy any number of services on any of the grid nodes, including singleton services. |
Fault Tolerance |
Ignite always guarantees that services are continuously available regardless of any topology changes. |
Load Balancing |
In all cases, other than singleton service deployment, Ignite deploys an equal number of services on each node within the cluster. |
Spark Shared RDD
IGNITE FOR SPARK | |
---|---|
Ignite RDD |
Apache Ignite provides an implementation of Spark RDD abstraction which allows to easily share state in memory across multiple Spark jobs. |
SQL Queries |
Running SQL queries using IgniteRDD is orders of magnitude faster than running SQL queries using Spark native RDDs or Data Frame APIs. |
In-Memory File System
IN-MEMORY FILE SYSTEM | |
---|---|
IGFS as Hadoop FileSystem |
Ignite File System (IGFS) delivers similar functionality to Hadoop HDFS, but only in memory. |
Hadoop FileSystem Cache |
IGFS can also be deployed as a caching layer over another Hadoop File System. |
On-Heap and Off-Heap |
IGFS allows to store files either on-heap or off-heap. |
INTEGRATION | |
Stand-Alone |
Ignite comes with Hadoop Accelerator distribution which can be easily plugged into existing Hadoop installations. |
Hortonworks HDP |
Ignite comes with Hadoop Accelerator distribution which can be easily plugged into existing Hortonworks installations |
Cloudera CDH |
Ignite comes with Hadoop Accelerator distribution which can be easily plugged into existing Cloudera installations. |
In-Memory MapReduce
IN-MEMORY MAPREDUCE | |
---|---|
Native Hadoop MapReduce |
Apache Ignite comes with in-memory implementation of Hadoop MapReduce APIs. |
Hadoop Accelerator |
Accelerate existing Hadoop-based systems and products using a dual-mode, high performance in-memory file system. |
Platform Independence
CROSS-PLATFORM INTEROPERABILITY | |
---|---|
Java |
Ignite In-Memory Data Fabric is developed predominantly in Java, and provides native support for other languages - .NET and C++. |
.NET |
Ignite.NET is built on top of Ignite. This allows you to perform almost all the in-memory data grid operations supported in Ignite. |
C++ |
Ignite C++ is built on top of Ignite. This allows you to perform almost all the in-memory data grid operations supported in Ignite. |
Supported Clients
CLIENT-SERVER PROTOCOLS | |
---|---|
Ignite Native Client |
Ignite Native Clients provide full functionality of Ignite In-Memory Data Fabric to the remote clients connecting to the data fabric. |
Memcached Support |
Ignite allows users to store and retrieve data from Ignite caches using any Memcached compatible clients. |
REST/HTTP |
Ignite HTTP REST client gives you the ability to communicate with the grid over HTTP and HTTPS protocols using REST approach. |
SSL/TLS |
Ignite allows you to use SSL socket communication among all Ignite client and server nodes. |
Deployment
DEPLOYMENT OPTIONS | |
---|---|
Zero Deployment |
Ignite nodes automatically become aware of custom classes, eliminating the need to explicitly deploy them. |
Dynamic Schema Changes |
Ignite stores objects in binary format eliminating the need to deploy classes on server nodes. |
Stand-Alone Clusters |
Ignite nodes can automatically discover each other by using DiscoverySpi. |
Public Clouds Support |
Ignite integrates with Amazon Web Services, Google Compute Engine, Apache Mesos, and Kubernetes. |
Docker Container |
Docker automates downloading the Ignite release, deploying users' library into Ignite, and configuring nodes. |
Apache Mesos |
Apache Ignite Framework supports scheduling and running Apache Ignite nodes in a Mesos cluster. |
Hadoop Yarn |
Integration with YARN supports scheduling and running Apache Ignite nodes in a YARN cluster. |
OSGI Support |