| Log Message: |
HBASE-3690 Option to Exclude Bulk Import Files from Minor Compaction
Summary:
We ran an incremental scrape with HFileOutputFormat and
encountered major compaction storms. This is caused by the bug in
HBASE-3404. The permanent fix is a little tricky without HBASE-2856. We
realized that a quicker solution for avoiding these compaction storms is
to simply exclude bulk import files from minor compactions and let them
only be handled by time-based major compactions. Add with functionality
along with a config option to enable it.
Rewrote this feature to be done on a per-bulkload basis.
Test Plan:
- mvn test -Dtest=TestHFileOutputFormat
DiffCamp Revision:
Reviewers: stack, Kannan, JIRA, dhruba
Reviewed By: stack
CC: dhruba, lhofhansl, nspiegelberg, stack
Differential Revision: 357
|
hbase/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/HFileOutputFormat.java