Apache Ignite Native Persistence
Even though Apache Ignite® is broadly used as a caching layer on top of external databases, it comes with its native persistence - a distributed, ACID, and SQL-compliant disk-based store. The native persistence integrates into the Ignite multi-tier storage as a disk tier that can be turned on to let Ignite store more data on disk than it can cache in memory and to enable fast cluster restarts.
When the native persistence enabled, Ignite stores a superset of data on disk and caches as much as it can in memory. For example, if your application needs to store 200 records in an Ignite cluster and the memory capacity allows caching only 150 records, then all 200 will be stored on disk, out of which 150 will be served from memory while the rest 50 from disk whenever the application requests them.
Ignite Persistence vs. External Databases
The native persistence has the following advantages over external databases:
- The ability to cache a subset of the data - Ignite stores 100% of data on disk and lets you cache as much as required in memory.
- The ability to query data from disk - if any record is missing in memory, then Ignite reads it from disk. This is supported for all the APIs including SQL.
- Instantaneous cluster restarts - Ignite becomes fully operational from disk upon a cluster startup or restarts without requiring to preload or warm up the memory tier.
Write-Ahead Logging and Checkpointing
If Ignite native persistence is selected as a disk tier, then every time a record is updated in memory, the change is added to the write-ahead log (WAL). The purpose of the WAL is to propagate updates to disk in the fastest way possible and provide a consistent recovery mechanism that supports full cluster failures. As the WAL grows, it periodically gets checkpointed to the main storage. Checkpointing is the process of copying dirty pages from the memory tier to the partition files on disk. A dirty page is a page that was updated in memory, was appended to the WAL, but was not written to the respective partition file on disk yet.
Durability
Ignite native persistence provides the following ACID guarantees across the cluster:
- Committed transactions always survive failures.
- The cluster can always be recovered to the latest successfully committed transaction.