In-Memory Database With Apache Ignite
Apache Ignite® is a distributed in-memory database that scales horizontally across memory and disk tiers. Ignite supports ACID transactions, ANSI-99 SQL, key-value, compute, machine learning, and other data processing APIs. As a database, Ignite uses memory, disk or Intel® Optane™ as active storage tiers and removes the need to cache all the data and the need for memory warm-ups.
Multi-Tier Storage
Apache Ignite works with memory, disk, and Intel Optane as active storage tiers. This architecture combines, in one system, the advantages of in-memory computing with disk durability and strong consistency.
When the native persistence is enabled, Ignite allows you to control the amount of memory it should consume. Depending on the memory space available, Ignite either caches the full data set in memory or keeps only the most frequently used data there and retrieves missing records from disk when needed. For instance, if there are 100 records and the memory of your system can accommodate only 20 of them, then all 100 records will be stored on disk and only 20 records will be cached in memory for better performance.
The following are the primary advantages of Ignite memory management architecture:
- Multi-tiered storage - Ignite treats disk as an active storage layer allowing to cache a subset of the data in memory and query both in-memory and disk-only records with SQL and all other available APIs.
- Instantaneous cluster restarts - Ignite becomes fully operational from disk upon a cluster startup or restarts without requiring to preload or warm up the memory tier.
Better High-Availability With Instantaneous Cluster Restarts
Ignite native persistence takes away the trouble of time-consuming memory warm-ups on cluster restarts. When the native persistence is enabled, Ignite stores a superset of data on disk treats it as one of the storage layers. Hence, Ignite starts reading data from the persistence layer as soon as the cluster becomes active. As you begin to run the queries, the memory tier is warmed up in the background with the data Ignite accesses from the disk.
Avoiding Network Impact on Performance With Co-located Processing
Disk-centric systems, like RDBMS or NoSQL, generally use the classic client-server approach when transferring data from the server to the client-side where it gets processed and then discarded. This approach does not scale very well because moving data over the network is the most expensive operation in a distributed system.
Many distributed databases, including Apache Ignite, support a more scalable approach called co-located processing, which eliminates or significantly reduces network traffic by running application logic right on the cluster nodes. This approach executes data or compute-intensive queries, including distributed SQL with JOINs, exactly where the data resides.