Apache Ignite Multi-Tier Storage
Apache Ignite® is designed to work with memory, disk, and Intel Optane as active storage tiers. The memory tier allows using DRAM and Intel® Optane™ operating in the Memory Mode for data storage and processing needs. The disk tier is optional with the support of two options -- you can persist data in an external database or keep it in the Ignite native persistence. SSD, Flash, HDD, or Intel Optane operating in the AppDirect Mode can be used as a storage device.
Ignite takes full control of its memory tier by allocating and managing off-heap regions. Each Ignite server node allocates memory regions during bootstrap, splits the regions into pages, and keeps data records with indexes in those pages. Java heap is used to keep temporary objects such as query result sets, metrics samples, and objects generated by your application code. All these objects are garbage collected eventually.
If you select native persistence as the disk tier, then most of the processing will still take place in memory on cached data, but with a full copy stored on disk. If any record is missing in memory, Ignite will read it from disk, allowing you to persist much larger data sets than you can cache in memory. This also eliminates the need for time-consuming memory warm-ups on restarts. As soon as your cluster reconnects after a restart, Ignite will serve most of the data from disk warming up the memory tier in the background.
Multi-Tier Storage Usage Modes
| Mode | Description |
|---|---|
| In-Memory |
The whole data set is available in memory only. In order to survive node failures, we recommend keeping at least one backup copy of the data in the cluster. DRAM or Intel® Optane™ operating in the Memory Mode can be used as a storage device. Use cases: General in-memory caching, high-performance computing, web-session caching, real-time processing of continuous data streams. |
| In-Memory + External Database |
Ignite is deployed as a distributed caching layer on top of an existing external database. This mode is for accelerating disk-based databases and your services with APIs that interact with them. Use cases: Acceleration of services and APIs with write-through and write-behind capability, to an external database. |
| In-Memory Cache + Native Persistence |
100% of data is persisted to disk, and the same or smaller amount is cached in memory. The more data is cached, the faster is the performance. The disk serves as the primary storage that survives any cluster failures and restarts. There is no need for memory warm-ups on restarts since Ignite can serve data from disk. SSD, Flash, HDD or Intel® Optane™ operating in the AppDirect Mode can be used as a storage device. Use cases: Ignite as an in-memory database or digital integration hub with the active persistence layer. |
Partitioning & Replication
Depending on the configuration, Ignite can both partition or replicate data across the cluster. In the replicated mode, each cluster node keeps a full copy of the data, but the size of a replicated cache is limited by the amount of memory available on the node. In the partitioned mode, Ignite spreads the data across all the cluster nodes evenly, allowing you to store much more than what can fit in a single machine.
Durability and Consistency
Ignite provides the following ACID guarantees across the cluster:
- Committed transactions always survive failures.
- The cluster can always be recovered to the latest successfully committed transaction.
Write-Ahead Logging and Checkpointing
If Ignite native persistence is selected as a disk tier, then every time a record is updated in memory, the change is added to the write-ahead log (WAL). The purpose of the WAL is to propagate updates to disk in the fastest way possible and provide a consistent recovery mechanism that supports full cluster failures.
As the WAL grows, it periodically gets checkpointed to the main storage. Checkpointing is the process of copying dirty pages from the memory tier to the partition files on disk. A dirty page is a page that was updated in memory, was appended to WAL, but was not written to the respective partition file on disk yet.