Distributed Database

Apache Ignite can be used as all-in-one distributed database that supports SQL, key-value, compute, machine learning and other data processing APIs.

Regardless of the API you decide to use, the data is always stored off-heap in the cluster-aware page-based storage which tries to evenly distribute the data, guaranteeing that every cluster node will own a portion of the overall data set.

The data and indexes can be stored both in RAM and, optionally, on disk if Ignite Native Persistence is enabled.

Ignite native persistent store utilizes disk to allow Ignite users store and process more data than can fit in memory. It also adds extra resiliency and crash recovery capabilities and supports full cluster restarts without loosing data. Moreover, when persistence is turned on, the cluster restarts are almost instantaneous, as Ignite becomes immediately operational from disk without having to preload all the data in memory.

Ignite is ANSI-99 compliant, supporting most SQL and DML commands, including SELECT, UPDATE, INSERT, MERGE, DELETE statements and distributed joins. It also provides support for a subset of DDL commands, including CREATE and ALTER commands for tables and indexes.

Ignite can be used ad a distributed key-value store, supporting all map-like key-value operations, including concurrent CAS operations. It also provides rich support for distributed ACID transactions, continuous queries, collocated processing and more. Ignite key-value store is fully compliant with JCache (JSR 107) specification.

Most disk-centric databases, like RDBMS or NoSQL, generally utilize the classic client-server approach, where the data is brought from the server to the client where it gets processed and then discarded. This approach does not scale well as moving the data over the network is the most expensive operation in a distributed system.

A much more scalable approach is collocated processing that reverses the flow by bringing the computations to the servers where the data actually resides. This approach allows you to execute advanced logic or distributed SQL with JOINs exactly where the data is stored, avoiding expensive serialization and network trips.

Depending on the configuration, Ignite can either partition or replicate data. Unlike REPLICATED mode, where data is fully replicated across all nodes in the cluster, in PARTITIONED mode Ignite will equally split the data across multiple cluster nodes, allowing for storing Terabytes of data in memory and on disk.

Ignite also allows to configure multiple backup copies guaranteeing data resiliency in case of individual node or server failures.

Regardless of which replication scheme is used, Ignite guarantees data consistency across all cluster members.

Check SQL, Data Grid, and Compute Grid features sections for more technical details.