This guide contains detailed information about using BookKeeper for write ahead logging. It discusses the basic operations BookKeeper supports, and how to create logs and perform basic read and write operations on these logs. The main classes used by BookKeeper client are BookKeeper and LedgerHandle.
BookKeeper is the main client used to create, open and delete ledgers. A ledger is a log file in BookKeeper, which contains a sequence of entries. Only the client which creates a ledger can write to it. A LedgerHandle represents the ledger to the client, and allows the client to read and write entries. When the client is finished writing they can close the LedgerHandle. Once a ledger has been closed, all client who read from it are guaranteed to read the exact same entries in the exact same order. All methods of BookKeeper and LedgerHandle have synchronous and asynchronous versions. Internally the synchronous versions are implemented using the asynchronous.
To create a BookKeeper client, you need to create a configuration object and set the address of the ZooKeeper ensemble in use. For example, if you were using zk1.example.com:2181,zk2.example.com:2181,zk3.example.com:2181 as your ensemble, you would create the BookKeeper client as follows.
ClientConfiguration conf = new ClientConfiguration();
conf.setZkServers("zk1.example.com:2181,zk2.example.com:2181,zk3.example.com:2181");
BookKeeper client = new BookKeeper(conf);
It is important to close the client once you are finished working with it. The set calls on ClientConfiguration are chainable, so instead of putting a set* call on a new line as above, it is possible to make a number of calls on the one line. For example;
ClientConfiguration conf = new ClientConfiguration().setZkServers("localhost:2181").setZkTimeout(5000);
There is also a useful shortcut constructor which allows you to pass the zookeeper ensemble string directly to BookKeeper.
BookKeeper client = new BookKeeper("localhost:2181");
See BookKeeper for the full api.
Before writing entries to BookKeeper, it is necessary to create a ledger. Before creating the ledger you must decide the ensemble size and the quorum size.
The ensemble size is the number of Bookies over which entries will be striped. The quorum size is the number of bookies which an entry will be written to. Striping is done in a round robin fashion. For example, if you have an ensemble size of 3 (consisting of bk1, bk2 & bk3), and a quorum of 2, entry 1 will be written to bk1 & bk2, entry 2 will be written to bk2 & bk3, entry 3 will be written to bk3 & bk1 and so on.
Ledgers are also created with a digest type and password. The digest type is used to generate a checksum so that when reading entries we can ensure that the content is the same as what was written. The password is used as an access control mechanism.
To create a ledger, with ensemble size 3, quorum size 2, using a CRC to checksum and "foobar" as the password, do the following:
LedgerHandle lh = client.createLedger(3, 2, DigestType.CRC32, "foobar");
You can now write to this ledger handle. As you probably plan to read the ledger at some stage, now is a good time to store the id of the ledger somewhere. The ledger id is a long, and can be obtained with lh.getId().
Once you have obtained a ledger handle, you can start adding entries to it. Entries are simply arrays of bytes. As such, adding entries to the ledger is rather simple.
lh.addEntry("Hello World!".getBytes());
Once a client is done writing, it can closes the ledger. Closing the ledger is a very important step in BookKeeper, as once a ledger is closed, all reading clients are guaranteed to read the same sequence of entries in the same order. Closing takes no parameters.
lh.close();
To read from a ledger, a client must open it first. To open a ledger you must know its ID, which digest type was used when creating it, and its password. To open the ledger we created above, assuming it has ID 1;
LedgerHandle lh2 = client.openLedger(1, DigestType.CRC32, "foobar");
You can now read entries from the ledger. Any attempt to write to this handle will throw an exception.
NOTE: Opening a ledger, which another client already has open for writing will prevent that client from writing any new entries to it. If you do not wish this to happen, you should use the openLedgerNoRecovery method. However, keep in mind that without recovery, you lose the guarantees of what entries are in the ledger. You should only use openLedgerNoRecovery if you know what you are doing.
Now that you have an open ledger, you can read entries from it. You can use getLastAddConfirmed to get the id of the last entry in the ledger.
long lastEntry = lh2.getLastAddConfirmed();
Enumeration<LedgerEntry> entries = lh2.readEntries(0, 9);
while (entries.hasMoreElements()) {
byte[] bytes = entries.nextElement().getEntry();
System.out.println(new String(bytes));
}