Apache Hadoop Changelog

Release 0.23.6 - 2013-02-06

IMPROVEMENTS:

JIRA Summary Priority Component Reporter Contributor
YARN-80 Support delay scheduling for node locality in MR2’s capacity scheduler Major capacityscheduler Todd Lipcon Arun C Murthy
HADOOP-8931 Add Java version to startup message Trivial . Eli Collins Eli Collins
MAPREDUCE-4811 JobHistoryServer should show when it was started in WebUI About page Minor jobhistoryserver, mrv2 Ravi Prakash Ravi Prakash
MAPREDUCE-4764 repair test org.apache.hadoop.mapreduce.security.TestBinaryTokenFile Major . Ivan A. Veselovsky
HADOOP-8992 Enhance unit-test coverage of class HarFileSystem Minor . Ivan A. Veselovsky Ivan A. Veselovsky
HADOOP-8561 Introduce HADOOP_PROXY_USER for secure impersonation in child hadoop client processes Major security Luke Lu Yu Gao
MAPREDUCE-4845 ClusterStatus.getMaxMemory() and getUsedMemory() exist in MR1 but not MR2 Major client Sandy Ryza Sandy Ryza
YARN-285 RM should be able to provide a tracking link for apps that have already been purged Major . Derek Dagit Derek Dagit
MAPREDUCE-4899 Provide a plugin to the Yarn Web App Proxy to generate tracking links for M/R appllications given the ID Major . Derek Dagit Derek Dagit
MAPREDUCE-4810 Add admin command options for ApplicationMaster Minor applicationmaster Jason Lowe Jerry Chen

BUG FIXES:

JIRA Summary Priority Component Reporter Contributor
HADOOP-7868 Hadoop native fails to compile when default linker option is -Wl,–as-needed Major native James Page Trevor Robinson
HDFS-2757 Cannot read a local block that’s being written to when using the local read short circuit Major . Jean-Daniel Cryans Jean-Daniel Cryans
HDFS-4186 logSync() is called with the write lock held while releasing lease Critical namenode Kihwal Lee Kihwal Lee
HADOOP-9072 Hadoop-Common-0.23-Build Fails to build in Jenkins Major . Robert Parker Robert Parker
MAPREDUCE-4825 JobImpl.finished doesn’t expect ERROR as a final job state Major mr-am Jason Lowe Jason Lowe
MAPREDUCE-4817 Hardcoded task ping timeout kills tasks localizing large amounts of data Critical applicationmaster, mr-am Jason Lowe Thomas Graves
HDFS-4242 Map.Entry is incorrectly used in LeaseManager Major namenode Tsz Wo Nicholas Sze Tsz Wo Nicholas Sze
HADOOP-9108 Add a method to clear terminateCalled to ExitUtil for test cases Major util Kihwal Lee Kihwal Lee
HDFS-4233 NN keeps serving even after no journals started while rolling edit Blocker namenode Kihwal Lee Kihwal Lee
YARN-251 Proxy URI generation fails for blank tracking URIs Major resourcemanager Tom White Tom White
MAPREDUCE-4836 Elapsed time for running tasks on AM web UI tasks page is 0 Major . Ravi Prakash Ravi Prakash
HDFS-4254 testAllEditsDirsFailOnFlush makes subsequent test cases fail (0.23.6 only) Major test Kihwal Lee Kihwal Lee
YARN-72 NM should handle cleaning up containers when it shuts down Major nodemanager Hitesh Shah Sandy Ryza
YARN-258 RM web page UI shows Invalid Date for start and finish times Major resourcemanager Ravi Prakash Ravi Prakash
YARN-266 RM and JHS Web UIs are blank because AppsBlock is not escaping string properly Critical resourcemanager Ravi Prakash Ravi Prakash
MAPREDUCE-4861 Cleanup: Remove unused mapreduce.security.token.DelegationTokenRenewal Major . Karthik Kambatla Karthik Kambatla
MAPREDUCE-4856 TestJobOutputCommitter uses same directory as TestJobCleanup Major test Sandy Ryza Sandy Ryza
HADOOP-9135 JniBasedUnixGroupsMappingWithFallback should log at debug rather than info during fallback Trivial security Colin P. McCabe Colin P. McCabe
HDFS-4315 DNs with multiple BPs can have BPOfferServices fail to start due to unsynchronized map access Major datanode Aaron T. Myers Aaron T. Myers
HDFS-4232 NN fails to write a fsimage with stale leases Blocker namenode Daryn Sharp Daryn Sharp
HADOOP-9152 HDFS can report negative DFS Used on clusters with very small amounts of data Minor fs Brock Noland Brock Noland
YARN-223 Change processTree interface to work better with native code Critical . Radim Kolar Radim Kolar
MAPREDUCE-4895 Fix compilation failure of org.apache.hadoop.mapred.gridmix.TestResourceUsageEmulators Major . Dennis Y Dennis Y
HADOOP-9105 FsShell -moveFromLocal erroneously fails Major fs Daryn Sharp Daryn Sharp
MAPREDUCE-4842 Shuffle race can hang reducer Blocker mrv2 Jason Lowe Mariappan Asokan
MAPREDUCE-4833 Task can get stuck in FAIL_CONTAINER_CLEANUP Critical applicationmaster, mrv2 Robert Joseph Evans Robert Parker
MAPREDUCE-4890 Invalid TaskImpl state transitions when task fails while speculating Critical mr-am Jason Lowe Jason Lowe
HADOOP-9169 Bring branch-0.23 ExitUtil up to same level as branch-2 Major . Robert Joseph Evans Robert Joseph Evans
MAPREDUCE-4902 Fix typo “receievd” should be “received” in log output Trivial . Albert Chu Albert Chu
YARN-225 Proxy Link in RM UI thows NPE in Secure mode Critical resourcemanager Devaraj K Devaraj K
MAPREDUCE-4813 AM timing out during job commit Critical applicationmaster Jason Lowe Jason Lowe
YARN-293 Node Manager leaks LocalizerRunner object for every Container Critical nodemanager Devaraj K Robert Joseph Evans
MAPREDUCE-4279 getClusterStatus() fails with null pointer exception when running jobs in local mode Major jobtracker Rahul Jain Devaraj K
HDFS-4270 Replications of the highest priority should be allowed to choose a source datanode that has reached its max replication limit Minor namenode Derek Dagit Derek Dagit
MAPREDUCE-4832 MR AM can get in a split brain situation Critical applicationmaster Robert Joseph Evans Jason Lowe
MAPREDUCE-4894 Renewal / cancellation of JobHistory tokens Blocker jobhistoryserver, mrv2 Siddharth Seth Siddharth Seth
MAPREDUCE-4819 AM can rerun job after reporting final job status to the client Blocker mr-am Jason Lowe Bikas Saha
MAPREDUCE-4913 TestMRAppMaster#testMRAppMasterMissingStaging occasionally exits Major mr-am Jason Lowe Jason Lowe
YARN-170 NodeManager stop() gets called twice on shutdown Major nodemanager Sandy Ryza Sandy Ryza
HADOOP-9181 Set daemon flag for HttpServer’s QueuedThreadPool Major . Liang Xie Liang Xie
YARN-320 RM should always be able to renew its own tokens Blocker resourcemanager Daryn Sharp Daryn Sharp
YARN-325 RM CapacityScheduler can deadlock when getQueueInfo() is called and a container is completing Blocker capacityscheduler Jason Lowe Arun C Murthy
MAPREDUCE-4848 TaskAttemptContext cast error during AM recovery Major mr-am Jason Lowe Jerry Chen
HADOOP-9097 Maven RAT plugin is not checking all source files Critical build Tom White Thomas Graves
HDFS-4385 Maven RAT plugin is not checking all source files Critical build Thomas Graves Thomas Graves
MAPREDUCE-4934 Maven RAT plugin is not checking all source files Critical build Thomas Graves Thomas Graves
YARN-334 Maven RAT plugin is not checking all source files Critical . Thomas Graves Thomas Graves
MAPREDUCE-4678 Running the Pentomino example with defaults throws java.lang.NegativeArraySizeException Minor examples Chris McConnell Chris McConnell
MAPREDUCE-4925 The pentomino option parser may be buggy Major examples Karthik Kambatla Karthik Kambatla
MAPREDUCE-4921 JobClient should acquire HS token with RM principal Blocker client Daryn Sharp Daryn Sharp
HDFS-4426 Secondary namenode shuts down immediately after startup Blocker namenode Jason Lowe Arpit Agarwal
YARN-354 WebAppProxyServer exits immediately after startup Blocker . Liang Xie Liang Xie
HADOOP-9242 Duplicate surefire plugin config in hadoop-common Major test Andrey Klochkov Andrey Klochkov
HADOOP-9255 relnotes.py missing last jira Critical scripts Thomas Graves Thomas Graves

TESTS:

JIRA Summary Priority Component Reporter Contributor
HADOOP-9038 provide unit-test coverage of class org.apache.hadoop.fs.LocalDirAllocator.AllocatorPerContext.PathIterator Minor . Ivan A. Veselovsky Ivan A. Veselovsky
HADOOP-9217 Print thread dumps when hadoop-common tests fail Major test Andrey Klochkov Andrey Klochkov

SUB-TASKS:

JIRA Summary Priority Component Reporter Contributor
HADOOP-9020 Add a SASL PLAIN server Major ipc, security Daryn Sharp Daryn Sharp
YARN-204 test coverage for org.apache.hadoop.tools Major applications Aleksey Gorshkov Aleksey Gorshkov
HDFS-4247 saveNamespace should be tolerant of dangling lease Blocker namenode Daryn Sharp Daryn Sharp
HDFS-4248 Renames may remove file leases Blocker namenode Daryn Sharp Daryn Sharp
YARN-280 RM does not reject app submission with invalid tokens Major resourcemanager Daryn Sharp Daryn Sharp
YARN-50 Implement renewal / cancellation of Delegation Tokens Blocker . Siddharth Seth Siddharth Seth

OTHER:

JIRA Summary Priority Component Reporter Contributor
HADOOP-8427 Convert Forrest docs to APT, incremental Major documentation Eli Collins Andy Isaacson