Title: BookKeeper Documentation
Notice: Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
.
http://www.apache.org/licenses/LICENSE-2.0
.
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
Release Notes - Bookkeeper - Version 4.3.0
NOTE: After support protobuf in [BOOKKEEPER-582] in 4.3.0, a 4.3.0 client will not work with a 4.2.3 server, while a 4.3.0 server still keeps backward compatability with 4.2.3 clients. Please upgrade servers in prior to clients.
Sub-task
Bug
- [BOOKKEEPER-313] - Bookkeeper shutdown call from Bookie thread is not shutting down server
- [BOOKKEEPER-363] - Re-distributing topics among newly added hubs.
- [BOOKKEEPER-446] - BookKeeper.createLedger(..) should not mask the error with ZKException
- [BOOKKEEPER-544] - Modify hedwig server tests to allow client testcases to start/stop them as part of their tests
- [BOOKKEEPER-548] - Document about periodic ledger checker configuration
- [BOOKKEEPER-549] - Documentation missed for readOnlyMode support
- [BOOKKEEPER-554] - fd leaking when move ledger index file
- [BOOKKEEPER-556] - BookieServerMXBean#getServerState makes no sense
- [BOOKKEEPER-557] - Compiler error showing up badly with jdk 7
- [BOOKKEEPER-559] - Fix occasional failure in AuditorBookieTest
- [BOOKKEEPER-561] - Findbugs reports errors with openjdk
- [BOOKKEEPER-567] - ReadOnlyBookieTest hangs on shutdown
- [BOOKKEEPER-568] - NPE during GC with HierarchicalLedgerManager
- [BOOKKEEPER-569] - Critical performance bug in InterleavedLedgerStorage
- [BOOKKEEPER-577] - BookieFailureTest uses sync/wait()/notify() incorrectly
- [BOOKKEEPER-579] - TestSubAfterCloseSub was put in a wrong package
- [BOOKKEEPER-580] - improve close logic
- [BOOKKEEPER-581] - Ledger recovery doesn't work correctly when recovery adds force changing ensembles.
- [BOOKKEEPER-583] - Read from a ReadOnlyBookie fails if index fileinfo is not in ledger cache
- [BOOKKEEPER-584] - Data loss when ledger metadata is overwritten
- [BOOKKEEPER-585] - Auditor logs noisily when a ledger has been deleted
- [BOOKKEEPER-586] - Remove recursive call in delivery manager
- [BOOKKEEPER-596] - Ledgers are gc'ed by mistake in MSLedgerManagerFactory.
- [BOOKKEEPER-597] - Add flag to output test logs to stdout
- [BOOKKEEPER-598] - Fails to compile - RESUBSCRIBE_EXCEPTION conflict
- [BOOKKEEPER-599] - NPE in PerChannelBookieClient
- [BOOKKEEPER-600] - shouldClaim flag isn't cleared for hedwig multiplex java client
- [BOOKKEEPER-601] - readahead cache size isn't updated correctly
- [BOOKKEEPER-602] - we should have request timeouts rather than channel timeout in PerChannelBookieClient
- [BOOKKEEPER-603] - Support Boost 1.53 for Hedwig Cpp Client
- [BOOKKEEPER-604] - Ledger storage can log an exception if GC happens concurrently.
- [BOOKKEEPER-607] - Filtered Messages Require ACK from Client Causes User Being Throttled Incorrectly Forever
- [BOOKKEEPER-610] - Make SyncThread use an executor
- [BOOKKEEPER-617] - BOOKKEEPER-544 breaks hedwig-server/bin/hedwig script
- [BOOKKEEPER-619] - Bookie should not create local cookie files if zookeeper is uninitialized
- [BOOKKEEPER-620] - PerChannelBookieClient race during channel disconnect
- [BOOKKEEPER-623] - LedgerChecker should avoid segments of closed ledger with higher start entryId than closed entry.
- [BOOKKEEPER-624] - Reduce logs generated by ReplicationWorker
- [BOOKKEEPER-626] - BOOKIE_EXTRA_OPTS are added twice
- [BOOKKEEPER-632] - AutoRecovery should consider read only bookies
- [BOOKKEEPER-633] - ConcurrentModificationException in RackawareEnsemblePlacementPolicy when a bookie is removed from available list
- [BOOKKEEPER-635] - jenkins build should highlight which lines of the patch cause raw analysis errors
- [BOOKKEEPER-636] - Latest txn logs might be deleted in a race condition which is not recoverable if BK goes down before next txn log created.
- [BOOKKEEPER-638] - Two bookies could start at the same time to access bookie data.
- [BOOKKEEPER-641] - DeathWatcher thread is unnecessarily running even after bookie shutdown
- [BOOKKEEPER-642] - Bookie returns incorrect exitcode, ExitCode.ZK_REG_FAIL is getting overridden
- [BOOKKEEPER-646] - BookieShell readjournal command is throwing BufferUnderflowException
- [BOOKKEEPER-649] - Race condition in sync ZKUtils.createFullPathOptimistic()
- [BOOKKEEPER-652] - Logger class name is wrong in LedgerCacheImpl.java
- [BOOKKEEPER-654] - Bookkeeper client operations are allowed even after its closure, bk#close()
- [BOOKKEEPER-663] - HierarchicalLedgerManager iterator is missing some ranges and the last ledger in the range
- [BOOKKEEPER-664] - Compaction increases latency on journal writes
- [BOOKKEEPER-667] - Client write will fail with BadMetadataVersion in case of multiple Bookie failures with AutoRecovery enabled
- [BOOKKEEPER-668] - Race between PerChannelBookieClient#channelDisconnected() and disconnect() calls can make clients hang while add/reading entries in case of multiple bookie failures
- [BOOKKEEPER-673] - Ledger length can be inaccurate in failure case
- [BOOKKEEPER-676] - Make add asynchrounous in ledger recovery
- [BOOKKEEPER-678] - BookieServer shutdown hangs indefinitely at NioServerSocketChannelFactory.releaseExternalResources
- [BOOKKEEPER-683] - TestSubAfterCloseSub fails on 4.2
- [BOOKKEEPER-686] - Bookie startup will fail if one of the configured ledgerDir is full and the same is used for replaying the journal
- [BOOKKEEPER-688] - NPE exception in PerChannelBookieClient
- [BOOKKEEPER-698] - Bookie client closure is not considering timeoutExecutor
- [BOOKKEEPER-700] - GarbageCollectorThread exsiting with ArrayIndexOutOfBoundsException
- [BOOKKEEPER-708] - Shade protobuf library to avoid incompatible versions
- [BOOKKEEPER-709] - SlowBookieTest#testSlowBookie fails intermittently
- [BOOKKEEPER-710] - OpenLedgerNoRecovery should watch ensemble change.
- [BOOKKEEPER-711] - bookkeeper-daemon.sh will not remove the pid file on successful stop
- [BOOKKEEPER-712] - bookkeeper script should use 'java' from JAVA_HOME
- [BOOKKEEPER-714] - Logging channel exceptions in PerChannelBookieClient
- [BOOKKEEPER-718] - AuditorLedgerCheckerTest is flakey
- [BOOKKEEPER-719] - Inconsistent synchronization of org.apache.bookkeeper.stats.CodahaleMetricsProvider.metrics
- [BOOKKEEPER-720] - CheckpointSource.MIN#compareTo does exactly the opposite of what it should
- [BOOKKEEPER-724] - Shade introduces RAT error
- [BOOKKEEPER-725] - AutoRecoveryMain should exit with error code if deathwatcher finds dead thread
- [BOOKKEEPER-726] - PerChannelBookieClient should print address that it failed to connect to when it fails to correct
- [BOOKKEEPER-727] - Names of bookie write/read threads are backwards
- [BOOKKEEPER-728] - Bookkeeper#Builder is not public, so can't be used outside of client package
- [BOOKKEEPER-729] - Bookie shouldn't exit with 0, if exiting from deathwatcher and thread death was caused by OOM
- [BOOKKEEPER-730] - Shade pom file missing apache license header
- [BOOKKEEPER-731] - Missing BOOKIE_PID_DIR and BOOKIE_STOP_TIMEOUT in env variables list
- [BOOKKEEPER-739] - Test timeouts mostly ignored
- [BOOKKEEPER-740] - AutoRecoveryMainTest#testAutoRecoverySessionLoss is failing
- [BOOKKEEPER-742] - Fix for empty ledgers losing quorum.
- [BOOKKEEPER-743] - Periodic ledger check running too often as doc doesn't match implementation.
- [BOOKKEEPER-744] - Run the auditor bookie check periodically
- [BOOKKEEPER-745] - Fix for false reports of ledger unreplication during rolling restarts.
- [BOOKKEEPER-746] - 5 new shell commands. List ledgers, list metadata, list underreplicated, show auditor and simpletest
- [BOOKKEEPER-750] - Flake in BookieAutoRecoveryTest#testEmptyLedgerLosesQuorumEventually
- [BOOKKEEPER-751] - Ensure all the bookkeeper callbacks not run under ledger handle lock
- [BOOKKEEPER-755] - Incorrect number of seconds specified in a day
- [BOOKKEEPER-763] - findbugs fails to run on jenkins
- [BOOKKEEPER-765] - bookkeeper script should fall back to java in path if JAVA_HOME is not set
- [BOOKKEEPER-766] - Update notice.txt files to include 2014
- [BOOKKEEPER-767] - Allow loopback in tests
- [BOOKKEEPER-768] - fix typo 'seconds' to milliseconds in benchmark output
- [BOOKKEEPER-774] - Flaky test org.apache.bookkeeper.test.ReadOnlyBookieTest.testBookieShouldTurnWritableFromReadOnly
- [BOOKKEEPER-777] - Flake in LedgerCloseTest
- [BOOKKEEPER-778] - Flake in TestTryReadLastConfirmed
- [BOOKKEEPER-779] - jmx reporter for codahale metrics provider
- [BOOKKEEPER-780] - Findbug issue in trunk
- [BOOKKEEPER-783] - Avoid running out of fds in MutlipleThreadReadTest
- [BOOKKEEPER-784] - BookKeeperCloseTest#testLedgerCheck is failing intermittently
- [BOOKKEEPER-785] - Fix javadoc warnings in trunk
- [BOOKKEEPER-786] - Fix Findbugs Error In Codahale Stats Provider
Improvement
- [BOOKKEEPER-429] - Provide separate read and write threads in the bookkeeper server
- [BOOKKEEPER-432] - Improve performance of entry log range read per ledger entries
- [BOOKKEEPER-526] - multiple threads for delivery manager
- [BOOKKEEPER-563] - Avoid Journal polluting page cache
- [BOOKKEEPER-564] - Better checkpoint mechanism
- [BOOKKEEPER-573] - Script to start a bookkeeper cluster
- [BOOKKEEPER-592] - allow application to recommend ledger data locality
- [BOOKKEEPER-605] - Use static Logger objects everywhere for bookkeeper
- [BOOKKEEPER-611] - Speed up bookkeeper tests
- [BOOKKEEPER-614] - Generic stats interface, which multiple providers can be plugged into
- [BOOKKEEPER-618] - Better resolution of bookie address
- [BOOKKEEPER-627] - LedgerDirsMonitor is missing thread name
- [BOOKKEEPER-630] - Add tag to o.a.b.net.* to indict which release of hadoop they came from, move DNS to o.a.b.net.* and indent
- [BOOKKEEPER-637] - NoSuchEntry exception when reading an entry from a bookie should not print ERROR level message
- [BOOKKEEPER-640] - Log improvement - add shutdown/exit log message for the bookie services
- [BOOKKEEPER-645] - Bookkeeper shell command to get a list of readonly bookies
- [BOOKKEEPER-661] - Turn readonly back to writable if spaces are reclaimed.
- [BOOKKEEPER-662] - Major GC should kick in immediately if remaining space reaches a warning threshold
- [BOOKKEEPER-666] - Naming threads of ExecutorService
- [BOOKKEEPER-699] - Codahale metrics implementation of stats API
- [BOOKKEEPER-701] - Improve exception handling of Bookkeeper threads
- [BOOKKEEPER-732] - Add env variable ENTRY_FORMATTER_CLASS to the bkenv.sh
- [BOOKKEEPER-747] - Implement register/unregister LedgerMetadataListener in MSLedgerManagerFactory
- [BOOKKEEPER-756] - Use HashedwheelTimer for request timeouts for PCBC
- [BOOKKEEPER-758] - Add TryReadLastAddConfirmed API
- [BOOKKEEPER-781] - Fix OOM on Hedwig Tests
New Feature
Task
Test
Wish