Apache Hadoop Changelog

Release 0.23.10 - 2013-12-09

INCOMPATIBLE CHANGES:

JIRA Summary Priority Component Reporter Contributor
YARN-707 Add user info in the YARN ClientToken Blocker . Bikas Saha Jason Lowe

IMPROVEMENTS:

JIRA Summary Priority Component Reporter Contributor
HDFS-5010 Reduce the frequency of getCurrentUser() calls from namenode Major namenode, performance Kihwal Lee Kihwal Lee
MAPREDUCE-1981 Improve getSplits performance by using listLocatedStatus Major job submission Hairong Kuang Hairong Kuang
YARN-985 Nodemanager should log where a resource was localized Major nodemanager Ravi Prakash Ravi Prakash
HADOOP-9686 Easy access to final parameters in Configuration Major conf Jason Lowe Jason Lowe
HADOOP-8704 add request logging to jetty/httpserver Major . Thomas Graves Jonathan Eagles
MAPREDUCE-5640 Rename TestLineRecordReader in jobclient module Trivial test Jason Lowe Jason Lowe

BUG FIXES:

JIRA Summary Priority Component Reporter Contributor
HDFS-3020 Auto-logSync based on edit log buffer size broken Critical namenode Todd Lipcon Todd Lipcon
MAPREDUCE-4579 TestTaskAttempt fails jdk7 Major . Thomas Graves Thomas Graves
HDFS-3970 BlockPoolSliceStorage#doRollback(..) should use BlockPoolSliceStorage instead of DataStorage to read prev version file. Major datanode Vinayakumar B Vinayakumar B
HDFS-4482 ReplicationMonitor thread can exit with NPE due to the race between delete and replication of same file. Blocker namenode Uma Maheswara Rao G Uma Maheswara Rao G
MAPREDUCE-4571 TestHsWebServicesJobs fails on jdk7 Major webapps Thomas Graves Thomas Graves
MAPREDUCE-4716 TestHsWebServicesJobsQuery.testJobsQueryStateInvalid fails with jdk7 Major jobhistoryserver Thomas Graves Thomas Graves
HDFS-4943 WebHdfsFileSystem does not work when original file path has encoded chars Minor webhdfs Jerry He Jerry He
MAPREDUCE-3193 FileInputFormat doesn’t read files recursively in the input path dir Major mrv1, mrv2 Ramgopal N Devaraj K
HDFS-4984 Incorrect Quota counting in INodeFile Major namenode Kihwal Lee Jing Zhao
HDFS-4972 [branch-0.23] permission check and operation are done in a separate lock for getBlockLocations() Major namenode Kihwal Lee Kihwal Lee
MAPREDUCE-5380 Invalid mapred command should return non-zero exit code Major . Stephen Chu Stephen Chu
HDFS-4998 TestUnderReplicatedBlocks fails intermittently Major test Kihwal Lee Kihwal Lee
MAPREDUCE-5317 Stale files left behind for failed jobs Major mrv2 Ravi Prakash Ravi Prakash
YARN-949 Failed log aggregation can leave a file open. Major nodemanager Kihwal Lee Kihwal Lee
MAPREDUCE-5251 Reducer should not implicate map attempt if it has insufficient space to fetch map output Major mrv2 Jason Lowe Ashwin Shankar
MAPREDUCE-5419 TestSlive is getting FileNotFound Exception Major mrv2 Robert Parker Robert Parker
HDFS-5028 LeaseRenewer throw java.util.ConcurrentModificationException when timeout Major . yunjiong zhao yunjiong zhao
MAPREDUCE-5440 TestCopyCommitter Fails on JDK7 Major mrv2 Robert Parker Robert Parker
HADOOP-9476 Some test cases in TestUserGroupInformation fail if ran after testSetLoginUser. Major security, test Kihwal Lee Robert Parker
MAPREDUCE-5425 Junit in TestJobHistoryServer failing in jdk 7 Major jobhistoryserver Ashwin Shankar Robert Parker
HDFS-5047 Supress logging of full stack trace of quota and lease exceptions Major namenode Kihwal Lee Robert Parker
HDFS-4993 fsck can fail if a file is renamed or deleted Major . Kihwal Lee Robert Parker
HADOOP-9757 Har metadata cache can grow without limit Major fs Jason Lowe Cristina L. Abad
YARN-1036 Distributed Cache gives inconsistent result if cache files get deleted from task tracker Major nodemanager Ravi Prakash Ravi Prakash
YARN-337 RM handles killed application tracking URL poorly Major resourcemanager Jason Lowe Jason Lowe
MAPREDUCE-5001 LocalJobRunner has race condition resulting in job failures Major . Brock Noland Sandy Ryza
HDFS-4994 Audit log getContentSummary() calls Minor namenode Kihwal Lee Robert Parker
MAPREDUCE-5468 AM recovery does not work for map only jobs Blocker . Yesha Vora Vinod Kumar Vavilapalli
HDFS-4329 DFSShell issues with directories with spaces in name Major hdfs-client Andy Isaacson Cristina L. Abad
YARN-1101 Active nodes can be decremented below 0 Major resourcemanager Robert Parker Robert Parker
MAPREDUCE-5475 MRClientService does not verify ACLs properly Blocker mr-am, mrv2 Jason Lowe Jason Lowe
MAPREDUCE-5414 TestTaskAttempt fails jdk7 with NullPointerException Major test Nemon Lou Nemon Lou
YARN-1176 RM web services ClusterMetricsInfo total nodes doesn’t include unhealthy nodes Critical resourcemanager Thomas Graves Jonathan Eagles
MAPREDUCE-5504 mapred queue -info inconsistent with types Major client Thomas Graves Kousuke Saruta
YARN-1243 ResourceManager: Error in handling event type NODE_UPDATE to the scheduler - NPE at SchedulerApp.java:411 Major capacityscheduler Sanjay Upadhyay Jason Lowe
MAPREDUCE-5543 In-memory map outputs can be leaked after shuffle completes in 0.23 Blocker mrv2 Jason Lowe Jason Lowe
MAPREDUCE-5513 ConcurrentModificationException in JobControl Major . Jason Lowe Robert Parker
MAPREDUCE-5569 FloatSplitter is not generating correct splits Major . Nathan Roberts Nathan Roberts
YARN-155 TestAppManager intermittently fails with jdk7 Major . Thomas Graves Thomas Graves
MAPREDUCE-5585 TestCopyCommitter#testNoCommitAction Fails on JDK7 Major . Jonathan Eagles Jonathan Eagles
MAPREDUCE-5586 TestCopyMapper#testCopyFailOnBlockSizeDifference fails when run from hadoop-tools/hadoop-distcp directory Major . Jonathan Eagles Jonathan Eagles
HDFS-5346 Avoid unnecessary call to getNumLiveDataNodes() for each block during IBR processing Major namenode, performance Kihwal Lee Ravi Prakash
MAPREDUCE-5587 TestTextOutputFormat fails on JDK7 Major . Jonathan Eagles Jonathan Eagles
MAPREDUCE-5561 org.apache.hadoop.mapreduce.v2.app.job.impl.TestJobImpl testcase failing on trunk Critical . Cindy Li Karthik Kambatla
YARN-1031 JQuery UI components reference external css in branch-23 Major . Jonathan Eagles Jonathan Eagles
YARN-1386 NodeManager mistakenly loses resources and relocalizes them Critical nodemanager Jason Lowe Jason Lowe
HADOOP-10091 Job with a har archive as input fails on 0.23 Blocker fs Jason Lowe Jason Lowe
HDFS-4995 Make getContentSummary() less expensive Major namenode Kihwal Lee Kihwal Lee
HDFS-5438 Flaws in block report processing can cause data loss Critical namenode Kihwal Lee Kihwal Lee
MAPREDUCE-5373 TestFetchFailure.testFetchFailureMultipleReduces could fail intermittently Major . Chuan Liu Jonathan Eagles
YARN-1419 TestFifoScheduler.testAppAttemptMetrics fails intermittently under jdk7 Minor scheduler Jonathan Eagles Jonathan Eagles
HDFS-5557 Write pipeline recovery for the last packet in the block may cause rejection of valid replicas Critical . Kihwal Lee Kihwal Lee
HDFS-5526 Datanode cannot roll back to previous layout version Blocker datanode Tsz Wo Nicholas Sze Kihwal Lee
HDFS-5558 LeaseManager monitor thread can crash if the last block is complete but another block is not. Major . Kihwal Lee Kihwal Lee

TESTS:

JIRA Summary Priority Component Reporter Contributor
YARN-1119 Add ClusterMetrics checks to tho TestRMNodeTransitions tests Major resourcemanager Robert Parker Mit Desai
MAPREDUCE-5610 TestSleepJob fails in jdk7 Major . Jonathan Eagles Jonathan Eagles

SUB-TASKS:

JIRA Summary Priority Component Reporter Contributor
HADOOP-9748 Reduce blocking on UGI.ensureInitialized Critical security Daryn Sharp Daryn Sharp
YARN-573 Shared data structures in Public Localizer and Private Localizer are not Thread safe. Critical . Omkar Vinit Joshi Omkar Vinit Joshi
HDFS-5239 Allow FSNamesystem lock fairness to be configurable Major namenode Daryn Sharp Daryn Sharp
HADOOP-9956 RPC listener inefficiently assigns connections to readers Major ipc Daryn Sharp Daryn Sharp
HADOOP-9955 RPC idle connection closing is extremely inefficient Major ipc Daryn Sharp Daryn Sharp