Chukwa Change Log

Release 0.5

• NEW FEATURES    (15)
  1. CHUKWA-590. Migrate from ant to maven.
     (Eric Yang)
  2. CHUKWA-599. Added Metrics 2 support for Hadoop 0.20.203+.
     (Eric Yang)
  3. CHUKWA-575. Added cluster summary pig script.
     (Eric Yang)
  4. CHUKWA-571. Add digested system metrics to SystemMetrics demux parser.
     (Eric Yang)
  5. CHUKWA-563. Added configuration to skip HBase schema validation for HBaseWriter.
     (Eric Yang)
  6. CHUKWA-558. Update default Chukwa configuration files to use SocketAdaptor for Hadoop metrics streaming.
     (Eric Yang)
  7. CHUKWA-527 Configuration spellcheck.
     (Ari Rabkin)
  8. CHUKWA-520. Implemented graphing rest api and graph explorer for plotting HBase data to graph.
     (Eric Yang)
  9. CHUKWA-530. Implemented Syslog Adaptor for mappig Syslog facility name to Chukwa data type.
     (Eric Yang)
  10. CHUKWA-419. Replaced system metrics collection with Sigar.
      (Eric Yang)
  11. CHUKWA-444. Added HBaseWriter for storing time series data in HBase for faster random read/write.
      (Eric Yang)
  12. CHUKWA-473. Make default processor configurable.
      (Bill Graham via asrabkin)
  13. CHUKWA-479. Support HTTP trigger actions
      (Bill Graham via asrabkin)
  14. CHUKWA-469. Add JMSAdaptor.
      (Bill Graham via asrabkin)
  15. CHUKWA-477. Support post-demux trigger.
      (Bill Graham via Eric Yang)
• IMPROVEMENTS    (42)
  1. CHUKWA-629. Updated Hadoop dependency to Hadoop 1.0.0
     (Eric Yang)
  2. CHUKWA-627. Updated DISCLAIMER.txt and LICENSE.txt
     (Eric Yang)
  3. CHUKWA-626. Updated NOTICE.txt to include dependency licenses.
     (Eric Yang)
  4. CHUKWA-624. Added md5 signature signing to maven build for release tarball.
     (Eric Yang)
  5. CHUKWA-623. Added build profile src for building release source tarball.
     (Eric Yang)
  6. CHUKWA-622. Updated HDFS instruction for collector configuration.
     (Eric Yang)
  7. CHUKWA-620. Updated code analysis and license check in build structure.
     (Eric Yang)
  8. CHUKWA-618. Added HICC programming guide.
     (Eric Yang)
  9. CHUKWA-522. Added HICC User guide.
     (Eric Yang)
  10. CHUKWA-615. Changed HICC default view to monitor Hadoop.
      (Eric Yang)
  11. CHUKWA-612. Convert Chukwa document from forrest format to apt format.
      (Eric Yang)
  12. CHUKWA-608. Added Quick start guide.
      (Ahmed Fathalla via Eric Yang)
  13. CHUKWA-605. Update directory structure to be aligned with Hadoop.
      (Eric Yang)
  14. CHUKWA-601. Removed legacy mysql upgrade script.
      (Eric Yang)
  15. CHUKWA-596. Updated schema retrieve REST API to use end time for scanning columns.
      (Eric Yang)
  16. CHUKWA-586. Updated HBase dependency to HBase 0.90.3.
      (Eric Yang)
  17. CHUKWA-569. Added temperory fix for load meta data faster for graph_explorer.jsp.
      (Eric Yang)
  18. CHUKWA-573. Setup ChukwaMetrics to send to port 9097 and use ChukwaMetricsProcessor for parsing.
      (Eric Yang)
  19. CHUKWA-572. Changed disk and network stats to gauge instead of counter.
      (Eric Yang)
  20. CHUKWA-554. Remove jdbc.conf dependency from Chukwa.
      (Eric Yang)
  21. CHUKWA-562. Added RegExDirTailingAdaptor.
      (Deepak Deshpande via Eric Yang)
  22. CHUKWA-474. Replaced JSON library with JSON-Simple.
      (Eric Yang)
  23. CHUKWA-560. Improve HBaseWriter performance.
      (Jerome Boulon via Eric Yang)
  24. CHUKWA-555. Update descriptor files to use HBase REST API.
      (Eric Yang)
  25. CHUKWA-557. Improve metrics differentation between recordName and contextName and clean up chukwa metrics.
      (Eric Yang)
  26. CHUKWA-547. Adjustable Axes on HICC graphs.
      (Michael Chang via asrabkin)
  27. CHUKWA-548. Gray out options with no data.
      (Michael Chang via asrabkin)
  28. CHUKWA-546. SocketAdaptor.
      (Eric Yang via asrabkin)
  29. CHUKWA-493. Change release artifact naming.
      (Eric Yang via asrabkin)
  30. CHUKWA-525. Extensible DirTailingAdaptor. (Jaydeep Ayachit via asrabkin).
  31. CHUKWA-531. Bundle HBase 0.20.6 in library path, and comment out test case for HBaseWriter for now.
      (Eric Yang)
  32. CHUKWA-524. Use TODO-HBASE-HOME and TODO-HBASE-CONF-DIR to control hbase location.
      (Eric Yang)
  33. CHUKWA-521. Changed default HICC port to 4080.
      (Eric Yang)
  34. CHUKWA-506. Added retry logic for MetricDataLoader
      (William Bajzek via Eric Yang)
  35. CHUKWA-507. Imrpoved url validation for JMSAdaptor.
      (Bill Graham via Eric Yang)
  36. CHUKWA-501. Newer version of Ivy, pull in newer deps.
      (Eric Yang via asrabkin)
  37. CHUKWA-504. Ability to delete file in FileAdaptor after successfully uploaded.
      (Kirk True via asrabkin)
  38. CHUKWA-495. Support for pig 0.7.
      (Bill Graham via asrabkin)
  39. CHUKWA-490. Enabling mdl.xml to be split into multiple files for ease of maintenance.
      (Kirk True via Eric Yang)
  40. CHUKWA-480. Allow CreateRecordFile to read job confs.
      (Bill Graham via asrabkin)
  41. CHUKWA-471. Expose JobConf to Demux Processors.
      (Jerome Boulon via asrabkin)
  42. CHUKWA-472. Added ability to configure timestamp format for the default Time Series Processor.
      (Bill Graham via Eric Yang)
• BUG FIXES    (51)
  1. CHUKWA-628. Updated LICENSE.txt and NOTICE.txt to meet IPMC request.
     (Eric Yang)
  2. CHUKWA-625. Updated version number to 0.5.0.
     (Eric Yang)
  3. CHUKWA-621. Implemented findbugs and pmd code analysis check.
     (Eric Yang)
  4. CHUKWA-619. Disabled trace method on Chukwa servlets.
     (Julio Conca via Eric Yang)
  5. CHUKWA-617. Fixed test case casting and race conditions.
     (Eric Yang)
  6. CHUKWA-616. Updated release note and change log generation.
     (Eric Yang)
  7. CHUKWA-614. Updated parent iframe resize code.
     (Eric Yang)
  8. CHUKWA-613. Removed Jersey 1.1 from source code library path.
     (Eric Yang)
  9. CHUKWA-611. Updated Chukwa tarball to include chukwa-0.5.0-client.jar.
     (Eric Yang)
  10. CHUKWA-606. Updated HICC to interface with data in HBase.
      (Eric Yang)
  11. CHUKWA-588. Updated hbase.schema to match Hadoop 0.20.205.0 metrics and job summary.
      (Eric Yang)
  12. CHUKWA-609. Map / to /hicc for HICC web server.
      (Eric Yang)
  13. CHUKWA-593. Fixed infinite loop archiving at midnight.
      (Sourygna Luangsay via Eric Yang)
  14. CHUKWA-602. Set svn ignore properties to ignore eclipse and maven files.
      (Eric Yang)
  15. CHUKWA-597. Close stderr and stdout after parent process detached.
      (Eric Yang)
  16. CHUKWA-38. Download Hadoop jar files from ivy.
      (Eric Yang)
  17. CHUKWA-595. Fixed buffering time calculation for Agent to send data to Collector.
      (Eric Yang)
  18. CHUKWA-594. Fix row key matching in HICC web interface.
      (Preetam Patil via Eric Yang)
  19. CHUKWA-592. Enable test case for HBaseWriter.
      (Eric Yang)
  20. CHUKWA-591. Check for empty result from HBase Scanner.
      (Eric Yang)
  21. CHUKWA-585. Enable HICC server to bind to all IP addresses.
      (Eric Yang)
  22. CHUKWA-582. Change MoveToRepository to ignore anything starting with _.
      (Ari Rabkin)
  23. CHUKWA-587. Improved default cluster selection code.
      (Eric Yang)
  24. CHUKWA-579. Package initial_adaptors config file for tarball.
      (Eric Yang)
  25. CHUKWA-577. Convert HadoopMetricsProcessor to use Simple JSON library.
      (Eric Yang)
  26. CHUKWA-574. Added locking for recording metrics through log4j.
      (Eric Yang)
  27. CHUKWA-568. Moved from json.jar library to JSON Simple library for SystemMetrics demux parser and add tags column family.
      (Eric Yang)
  28. CHUKWA-231. Update default view to show graph explorer and remove deprecated widgets.
      (Eric Yang)
  29. CHUKWA-7. Handle pid in /tmp in case CHUKWA_HOME is not defined.
      (Ari Rabkin via Eric Yang)
  30. CHUKWA-553. Retain file filter argument for DirTailingAdaptor.
      (Deepak Deshpande via Eric Yang)
  31. CHUKWA-559. Switched unit test HICC port to 4080.
      (Eric Yang)
  32. CHUKWA-543. Parse only .view files for processing user view files.
      (Eric Yang)
  33. CHUKWA-552. Handle % signs in series names.
      (Michael Chang via asrabkin)
  34. CHUKWA-544. HICC graph should handle single series or multiple series more gracefully.
      (Eric Yang via asrabkin)
  35. CHUKWA-447. Correct time units in ExecAdaptor.
      (Eric Yang via asrabkin)
  36. CHUKWA-542. Disable hbase jar download from maven, and include hicc rest api in HICC web.xml
      (Eric Yang)
  37. CHUKWA-516. Downgrade to HBase 0.20.6 until 0.90 arrives.
      (Eric Yang)
  38. CHUKWA-496. Fixed config file reference in design.html
      (Bernd Fondermann via Eric Yang)
  39. CHUKWA-510. JMSMessagePropertyTransformer inserts "null" string for null JMS message property values
      (Bill Graham via asrabkin)
  40. CHUKWA-505. Don't include duplicate classes in chukwa-core.jar.
      (Bill Graham via asrabkin)
  41. CHUKWA-487. Collector left in a bad state after temprorary NN outage.
      (Bill Graham via asrabkin)
  42. CHUKWA-499. JMSAdaptor misses last argument in some cases when parsing.
      (Bill Graham via asrabkin)
  43. CHUKWA-498. Agent logs incorrect number of chunks sent.
      (Bill Graham via asrabkin)
  44. CHHUKWA-494. NPE when no time range is specified.
      (Bernd Fondermann via asrabkin)
  45. CHUKWA-492. Correction to database setup instruction.
      (Bernd Fondermann via Eric Yang)
  46. CHUKWA-489. Fixed malformed SQL in MetricDataLoader when values contain single quote character.
      (Kirk True via Eric Yang)
  47. CHUKWA-488. Filter user customized jar file path URL.
      (Kirk True via Eric Yang)
  48. CHUKWA-483. Use SIGTERM to stop chukwa, not signal 1.
      (Bill Graham via asrabkin)
  49. CHUKWA-4. Collectors don't finish writing .dones on close.
      (Ahmed Fathalla via asrabkin)
  50. CHUKWA-485. Fix TSProcessor handling of some dates.
      (Bill Graham via asrabkin)
  51. CHUKWA-478. Fixed TestSocketTee intermittent failure.
      (Chris Douglas via Eric Yang)

Release 0.4

• NEW FEATURES    (32)
  1. CHUKWA-445. Realtime display at collector.
     (asrabkin)
  2. CHUKWA-454. DirTailingAdaptor can filter files.
     (Gerrit Jansen van Vuuren via asrabkin)
  3. CHUKWA-449. Utility to generate sequence file from log file.
     (Bill Graham via asrabkin)
  4. CHUKWA-440. Enable addon jar file for Demux from Distributed Cache.
     (Eric Yang)
  5. CHUKWA-448. Write-ahead buffering for arbitrary adaptors.
     (asrabkin)
  6. CHUKWA-441. Added real time Hadoop activity monitor.
     (Eric Yang)
  7. CHUKWA-433. File-per-post writer for benchmark purposes.
     (asrabkin)
  8. CHUKWA-431. UDP Adaptor.
     (asrabkin)
  9. CHUKWA-46. Ability to allow only local connections to agent control port.
     (asrabkin)
  10. CHUKWA-427. Automate FSM state calculation and load to database.
      (Eric Yang)
  11. CHUKWA-424. FSMBuilder driver script.
      (Jiaqi Tan via asrabkin)
  12. CHUKWA-395. Support for generalized buffering of adaptor data.
      (asrabkin)
  13. CHUKWA-405 Add a "stop all" command.
      (asrabkin)
  14. IMPROVEMENTS
  15. CHUKWA-468. Use SIGHUP instead of SIGTERM for shutdown.
      (Eric Yang)
  16. CHUKWA-470. License headers.
      (asrabkin)
  17. CHUKWA-458. Documentation fixes for 0.4
      (asrabkin)
  18. CHUKWA-446. Refactor start/stop scripts.
      (Eric Yang)
  19. CHUKWA-450. Ability to turn off sort in dumpchunks.
      (asrabkin)
  20. CHUKWA-359. Report detailed error on adaptor start failure.
      (asrabkin)
  21. CHUKWA-420. Clean up adaptor stop methods.
      (asrabkin)
  22. CHUKWA-437. Clean up widget category labels.
      (Eric Yang)
  23. CHUKWA-421. Use modification time to detect rotation.
      (asrabkin)
  24. CHUKWA-432. PipelineableWriter becomes an abstract class.
      (asrabkin)
  25. CHUKWA-429. Update HDFS heatmap color with rainbow colors.
      (Eric Yang)
  26. CHUKWA-418. Standardize logging. Mark adaptor IDs.
      (asrabkin)
  27. CHUKWA-207. Store HICC dashboard data in HDFS.
      (Eric Yang)
  28. CHUKWA-415. Read initial adaptors at start every time.
      (asrabkin)
  29. CHUKWA-413. Improve admin guide.
      (asrabkin)
  30. CHUKWA-345. Remove redundant 'application' field from Chunk API.
      (asrabkin)
  31. CHUKWA-409. Make SocketTeeWriter work in single-stage pipeline.
      (Thushara Wijeratna via asrabkin)
  32. CHUKWA-408. Add optional metadata to real-time feed.
      (asrabkin)
• BUG FIXES    (27)
  1. CHUKWA-523. Fix reload collectors.
     (Eric Yang via asrabkin)
  2. CHUKWA-508. Include contrib directory build files.
     (Bill Graham via Eric Yang)
  3. CHUKWA-465. Setup default value for Post Demux Data Loader.
     (Ahmed Fathalla via Eric Yang)
  4. CHUKWA-467. Disabled war file extraction to /tmp.
     (Eric Yang)
  5. CHUKWA-456. Remove /tmp/chukwa/hicc during test phase to prevent bad data building up and prevent test case to run properly.
     (Eric Yang)
  6. CHUKWA-460. Removed dependency of writer.hdfs.filesystem for demux phase.
     (Eric Yang)
  7. CHUKWA-466. Fix failing tests.
     (asrabkin)
  8. CHUKWA-457. Fix TestUserResource.
     (Eric Yang via asrabkin)
  9. CHUKWA-463. Fix pattern match for tags.
     (Guillermo Perez via asrabkin)
  10. CHUKWA-464. Fix spelling of processClusterDirectory.
      (Guillermo Perez via asrabkin)
  11. CHUKWA-456. Fixed failing unit test.
      (Eric Yang)
  12. CHUKWA-455. Updated slaves.sh to detect collectors list.
      (Guillermo Pérez)
  13. CHUKWA-453. Specify bash instead of sh in script headers.
      (Gerrit Jansen van Vuuren via asrabkin)
  14. CHUKWA-451. Shouldn't use ssh to start processes locally.
      (Bill Graham via asrabkin)
  15. CHUKWA-452. Fix support for using pig on final archives.
      (Gerrit Jansen van Vuuren via asrabkin)
  16. CHUKWA-443. Updated HDFS metrics logging filename.
      (Eric Yang)
  17. CHUKWA-442. Updated Apache license for HICC Rest API.
      (Eric Yang)
  18. CHUKWA-438. Fixed column resizing issue in HICC
      (Eric Yang)
  19. CHUKWA-439. Added blank image for HICC.
      (Eric Yang)
  20. CHUKWA-436. Include time widget related javascript in the dashboard.
      (Eric Yang)
  21. CHUKWA-430. Narrow down the list of demux output for FSM to improve processing time.
      (Eric Yang)
  22. CHUKWA-428. Revised location for Chukwa HDFS repository.
      (Eric Yang)
  23. CHUKWA-241. Revise chukwa-config.sh so that chukwa runs after building from source.
      (Eric Yang via asrabkin)
  24. CHUKWA-2. Removed the default /var/log/messages streaming.
      (Eric Yang)
  25. CHUKWA-414. Better error messaging for pid file errors.
      (Bill Graham via asrabkin)
  26. CHUKWA-411. Hosts for DataNode ClientTrace messages parsed incorrectly.
      (Jiaqi Tan via asrabkin)
  27. CHUKWA-229. Fix file descriptor leak in ExecAdaptor.
      (asrabkin)

Older Releases

Release 0.3

• NEW FEATURES    (26)
  1. CHUKWA-344. State-Machine Generation for input to SALSA visualizations.
     (Jiaqi Tan via asrabkin)
  2. CHUKWA-384. Added REST API charting capability.
     (Eric Yang)
  3. CHUKWA-369. Tolerance of collector failures.
     (asrabkin)
  4. CHUKWA-368. New data integrity validation tool.
     (asrabkin)
  5. CHUKWA-383. Added embed mode for HICC.
     (Eric Yang)
  6. CHUKWA-382. Added export button to export HICC graph as static image.
     (Eric Yang)
  7. CHUKWA-366. Custom tags.
     (asrabkin)
  8. CHUKWA-358. Real-time monitoring at collector.
     (asrabkin)
  9. CHUKWA-352. Xtrace in contrib.
     (asrabkin)
  10. CHUKWA-346. Simplified sink archiver.
      (asrabkin)
  11. CHUKWA-343. Static HDFS Heatmap visualization.
      (Jiaqi Tan via asrabkin)
  12. CHUKWA-342. Static swimlanes visualization widget.
      (Jiaqi Tan via asrabkin)
  13. CHUKWA-299. Added HDFS Spatial heatmaps visualization.
      (Jiaqi Tan via Eric Yang)
  14. CHUKWA-185. Ability to tail a whole directory.
      (asrabkin)
  15. CHUKWA-280. Added end to end test to detect iostat overflow.
      (Eric Yang)
  16. CHUKWA-194. Backfilling tools.
      (Jerome Boulon via asrabkin)
  17. CHUKWA-253. Added aggregations by user.
      (Cheng Zhang via Eric Yang)
  18. CHUKWA-95. Added Web Service API to export data from database.
      (Terence Kwan via Eric Yang)
  19. CHUKWA-281. Created pig scripts to perform down sampling.
      (Cheng Zhang via Eric Yang)
  20. CHUKWA-20. Added pig support for ChukwaRecords.
      (Jerome Boulon via Eric Yang)
  21. CHUKWA-279. Added swim lane visualization for Hadoop job progress.
      (Jiaqi Tan via Eric Yang)
  22. CHUKWA-267. Added TaskLogAppender for collecting Task Tracker Log file.
      (Eric Yang)
  23. CHUKWA-71. Status/version command for chukwa agent.
      (asrabkin)
  24. CHUKWA-271. Added SQL client to HICC for SQL data exploration.
      (Eric Yang)
  25. CHUKWA-210. Add new job conf metrics as database columns.
      (Cheng Zhang via Eric Yang)
  26. CHUKWA-91. Additional test case for acks.
      (asrabkin)
• IMPROVEMENTS    (14)
  1. CHUKWA-390. Improvements to asynchronous acknowledgement mechanism.
     (asrabkin)
  2. CHUKWA-397. Allow "all" as search pattern.
     (asrabkin)
  3. CHUKWA-393. Support using pig on Chunks.
     (asrabkin)
  4. CHUKWA-392. FIFO queueing of threads in collector.
     (asrabkin)
  5. CHUKWA-388. Clean up user interface color.
     (Eric Yang)
  6. CHUKWA-387. Summarize mode for dumpChunks should count bytes.
     (asrabkin)
  7. CHUKWA-379. Refactor sender code.
     (asrabkin)
  8. CHUKWA-374. Adaptor.getStatus() shouldn't throw exceptions.
     (asrabkin)
  9. CHUKWA-373. Test code for backpressure.
     (asrabkin)
  10. CHUKWA-370. Exec adaptor should commit immediately.
      (asrabkin)
  11. CHUKWA-367. Print metadata in DumpChunks.
      (asrabkin)
  12. CHUKWA-74. Agent shouldn't log exceptions for down collectors.
      (Rushin Barot via asrabkin)
  13. CHUKWA-365. Improved DumpArchive tool.
      (asrabkin)
  14. CHUKWA-364. Design and Architecture document in documentation.
      (asrabkin)

CHUKWA-333. Copy release notes from 0.2 forward to Trunk. (asrabkin)

• CHUKWA-362. Archiver group-by-cluster conf name shouldn't be hardcoded.
  (asrabkin)
• CHUKWA-355. Remove obsolete datacollection.protocol package.
  (asrabkin)
• CHUKWA-270. ChukwaAgentController no longer uses stdout.
  (Rushin Barot via asrabkin)
• CHUKWA-356. More powerful file extractor.
  (asrabkin)
• CHUKWA-350. Improve docs, add programmer guide.
  (asrabkin)
• CHUKWA-282. Demux should output detailed per-operation ClientTrace records
  (Jiaqi Tan via asrabkin)
• CHUKWA-341. Heap space in HICC Jetty is configurable.
  (Jiaqi Tan via asrabkin)
• CHUKWA-284. Checkpoint includes last byte Acked, not sent.
  (asrabkin)
• CHUKWA-25. Archiver can group by cluster name.
  (asrabkin)
• CHUKWA-308. Added capability to start hicc without tomcat.
  (Eric Yang)
• CHUKWA-326. If the chukwa records produce errors in post process, move to InErrorDirectory.
  (Jerome Boulon via Eric Yang)
• CHUKWA-318. Added check for disk capacity for collector to write.
  (Jerome Boulon via Eric Yang)
• CHUKWA-276. Fix hourly and daily rolling to use a single reducer.
  (Jerome Boulon via Eric Yang)
• CHUKWA-313. Removed the 10 second ack from SeqFileWriter.
  (Eric Yang)
• CHUKWA-5. Adaptors have durable names.
  (asrabkin)
• CHUKWA-280. Added end to end test to detect iostat overflow.
  (Eric Yang)
• CHUKWA-278. Improve post process manager and metric data loader to support data loading from pig aggregation.
  (Eric Yang)
• CHUKWA-263. Added ability to configure demux parameters during build time.
  (Jerome Boulon via Eric Yang)
• CHUKWA-195. Update demux compilation to use Hadoop 0.20 jar file.
  (Jerome Boulon via Eric Yang)
• CHUKWA-268. Expose adaptor manager interface, refactor Agent class hierarchy.
  (asrabkin)
• CHUKWA-259. More flexible CollectorStub.
  (asrabkin)
• CHUKWA-274. Improve rate calculation for accumulated values for visualization.
  (Eric Yang)
• CHUKWA-275. Add standard deviation to statics calculation for HICC charts.
  (Eric Yang)
• CHUKWA-137. Add collector IP to the dataSink fileName.
  (Jerome Boulon via asrabkin)
• CHUKWA-258. More detailed logging of SQL errors.
  (asrabkin)
• CHUKWA-30. Remove HDFS flush and connection holding in Collector.
  (Jerome Boulon via asrabkin)
• CHUKWA-142. Remove add(Chunk) from SeqFileWriter.
  (Jerome Boulon via asrabkin)
• CHUKWA-82. Added min,max,average to HICC Charting widget.
  (Terence Kwan via Eric Yang)
• CHUKWA-219. Improved usability of host selector, and updated caching for host list.
  (Eric Yang)
• CHUKWA-407. PatternSyntaxExceptions should be sent to client.
  (thushara wijeratna via asrabkin)
• CHUKWA-402. Added readme and license notice file.
  (asrabkin)
• CHUKWA-401. Fix race condition in Dir Tailing adaptor test case.
  (asrabkin)
• CHUKWA-398. Make ConstRateAdaptor more deterministic.
  (asrabkin)
• CHUKWA-386. Fix a bug in echo id to the browser for the iframe servlet.
  (Eric Yang)
• CHUKWA-380. FTA shouldn't emit empty chunks.
  (asrabkin)
• CHUKWA-378. Disable TestArchive unit test.
  (asrabkin)
• CHUKWA-377. Revised xtrace adaptor code.
  (asrabkin)
• CHUKWA-372. select_multiple_callback does not work in hicc
  (Eric Yang via asrabkin)
• CHUKWA-354. Fix corner case in SinkArchiver.
  (asrabkin)
• CHUKWA-334. Fix LocalHdfsMover exception handling.
  (Jerome Boulon via asrabkin)
• CHUKWA-335. Discard meta line in job history file.
  (Cheng Zhang via asrabkin)
• CHUKWA-330. Make separation between fixed adaptor parameters and optional adaptor parameters for generating adaptor id.
  (Ari Rabkin via Eric Yang)
• CHUKWA-37. Remove stale and unused bin scripts.
  (asrabkin)
• CHUKWA-331. Fixed regular expression for down sampling base on record type.
  (Cheng Zhang via Eric Yang)
• CHUKWA-329. Correction to user job summary script.
  (Cheng Zhang via Eric Yang)
• CHUKWA-328. Handle empty sequence file more gracefully for MetricDataLoader.
  (Cheng Zhang via Eric Yang)
• CHUKWA-327. Remove START_TIME="0", FINISH_TIME="0" from JobHistory.
  (Cheng Zhang via Eric Yang)
• CHUKWA-325. Removed hourly rolling flag from daily rolling.
  (Jerome Boulon via Eric Yang)
• CHUKWA-324. For 0 matched field, changed MDL to skip empty SQL statement.
  (Eric Yang)
• CHUKWA-321. Added 2 seconds sleep to avoid chukwa agent race condition.
  (Eric Yang)
• CHUKWA-319. Added cluster name to pig user aggregation script.
  (Cheng Zhang via Eric Yang)
• CHUKWA-316. Added explicit rule for MetricDataLoader to prevent loading of data.
  (Eric Yang)
• CHUKWA-42. Clean up logging of adaptor creation.
  (Ari Rabkin)
• CHUKWA-312. Fixed chukwa-pig.jar class path for pig aggregation.
  (Cheng Zhang via Eric Yang)
• CHUKWA-304. Propagate exception upward for database test cases.
  (Jerome Boulon via Eric Yang)
• CHUKWA-314. Added filtering for start_time to remove invalid value.
  (Eric Yang)
• CHUKWA-304. Propagate exception upward for database test cases.
  (Eric Yang)
• CHUKWA-309. Fix MDL configuration for mr_job_conf table.
  (Cheng Zhang via Eric Yang)
• CHUKWA-301. Set TestFileExpirationPolicy controller port to 0 to avoid race condition.
  (Jerome Boulon via asrabkin)
• CHUKWA-302. Removed hard coded dependency of Hadoop jar file.
  (Eric Yang)
• CHUKWA-298. Update branch 0.2 and trunk version number.
  (Eric Yang)
• CHUKWA-296. Fix jsp syntax error.
  (Terence Kwan via Eric Yang)
• CHUKWA-294. Added post process data loader configuration.
  (Eric Yang)
• CHUKWA-10. Fix TestAgentConfig
  (asrabkin)
• CHUKWA-291. Append slash to collector paths.
  (asrabkin)
• CHUKWA-287. Fix collector crash on startup when no args.
  (Eric Yang via asrabkin)
• CHUKWA-273. Agents correctly use configured collector port numbers.
  (asrabkin)
• CHUKWA-216. postProcess.sh should use HADOOP_CONF_DIR to find hadoop-site.xml.
  (Jerome Boulon via asrabkin)
• CHUKWA-261. Added X axis label for non Time Series graphs.
  (Eric Yang)

Release 0.2 - Unreleased

Release 0.1.2 - 2009-05-14

• INCOMPATIBLE CHANGES    (none)
• NEW FEATURES    (13)
  1. CHUKWA-97. Refactored FileTailers, added LWFTAdaptor.
     (asrabkin)
  2. CHUKWA-236. Added migration script for moving database schema for Chukwa 0.1.1 to Chukwa 0.1.2.
     (Eric Yang)
  3. CHUKWA-78. Added down sample SQL aggregation for job data, task data and utilization data.
     (Eric Yang)
  4. CHUKWA-17. Collect PS output every 10 minutes and parse into key/value pairs sequence file.
     (Cheng Zhang via Eric Yang)
  5. CHUKWA-55. Aggregate data from HDFS bytes usage, and Mapreduce Job slot time to compute user usage of the cluster.
     (Eric Yang)
  6. CHUKWA-50. Added parser for extract Job History and Job Conf into key value pairs, and database loader dictionary file.
     (Contribute by Cheng Zhang via Eric Yang)
  7. CHUKWA-69. Calculate trapezoid area for a given series of data.
     (Contribute by Cheng Zhang via Eric Yang)
  8. CHUKWA-62. Add script to start tailing files.
     (asrabkin)
  9. CHUKWA-12. Add instrumention Api for Chukwa components.
     (Jerome Boulon via asrabkin)
  10. CHUKWA-14. Added permalink support to HICC GUI.
      (Eric Yang)
  11. CHUKWA-45. Added docs to folder structure.
      (Corinne Chandel via asrabkin)
  12. CHUKWA-13. Create a new Macro class for macro substitution. Changed Aggregator to use Macro class. Added macro substitution support to Chukwa Charting. Added Macro test case.
      (Eric Yang)
  13. HADOOP-4989. Added capability to add scatter chart.
      (Eric Yang)
• IMPROVEMENTS    (31)
  1. CHUKWA-226. Changed HDFS usage collection frequency from 10 minutes to 60 minutes.
     (Cheng Zhang via Eric Yang)
  2. CHUKWA-104. Remove permission setting code for Log4JMetricsContext and ChukwaDailyRollingFileAppender.
     (Jerome Boulon via Eric Yang)
  3. CHUKWA-178. Extend aggregation by 5 minutes for cluster metrics.
     (Eric Yang)
  4. CHUKWA-176. Rearrange the parameter order for aggregator.sh.
     (Eric Yang)
  5. CHUKWA-173. Parameterize configuration, and enable substitution at build time.
     (Jerome Boulon via Eric Yang)
  6. CHUKWA-131. Added additional Mapred job/task metrics.
     (Eric Yang)
  7. CHUKWA-145. Tuned hadoop parameters for demux.
     (Jerome Boulon via Eric Yang)
  8. CHUKWA-163. Updated reference to mdl.xml file.
     (Terence Kwan via Eric Yang)
  9. CHUKWA-157. Added javadoc target, api-xml, api-report, and change log 2 html.
     (Eric Yang)
  10. CHUKWA-141. Updated Chukwa Docs overview page for 0.1.2.
      (Corinne Chandel via Eric Yang)
  11. CHUKWA-138. Updated Chukwa Admin Guide.
      (Corinne Chandel via Eric Yang)
  12. CHUKWA-128. Added tools to compute aggregation in database.
      (Eric Yang)
  13. CHUKWA-112. Updated README file.
      (Corinne Chandel via Eric Yang)
  14. CHUKWA-134. Add release audit target.
      (Giridharan Kesavan via Eric Yang)
  15. CHUKWA-80. Extracted rpm spec file from build.xml file. Fix start up script, and config script.
      (Eric Yang)
  16. CHUKWA-26. * DemuxManager, ArchiveManager and PostProcessorManager are now a single daemon process each. * Each one working independently from others, as soon as something is available. * Start-data-processor is now using those new daemons instead of pocessSink.sh * Daily will process a daily compaction only when all hourly would have been done. * Demux is now able to send NSCA commands to Nagios.
      (Jerome Boulon via Eric Yang)
  17. CHUKWA-81. Fix file descriptor leak.
      (asrabkin)
  18. CHUKWA-29. Added TaskTracker and DataNode client trace log parser and database loader.
      (Chris Douglas via Eric Yang)
  19. CHUKWA-86. Improved JDBC compatibility layer
      (Cheng Zhang via Eric Yang)
  20. CHUKWA-64. Changed version number to 0.1.2, and release number to alpha.
      (Eric Yang)
  21. CHUKWA-59. Collect HDFS Usage information for users.
      (Contribute by Cheng Zhang via Eric Yang)
  22. CHUKWA-54. Filters web based input from HICC to prevent cross site scripting attack.
      (Eric Yang)
  23. CHUKWA-31. Added duration logging for SQL statements.
      (Eric Yang)
  24. HADOOP-5373. Collectors track lifetime-received chunks.
      (asrabkin)
  25. HADOOP-5370. Collectors don't write empty sink files
      (asrabkin)
  26. HADOOP-5228. Chukwa tests shouldn't write to /tmp.
      (asrabkin)
  27. HADOOP-5035. Improved Y axis ticker labelling. Used TreeMap to build non-time series data for charting. Improved handling of "not a number "values.
      (Eric Yang)
  28. HADOOP-5030. Changed RPM install location to the value specified by build.properties file.
      (Eric Yang)
  29. HADOOP-5029. Added mdl script to manually load chukwa sequence file to database.
      (Eric Yang)
  30. HADOOP-4843. Enable job history log file streaming into Chukwa by using JobTrackerInstrumentation API.
      (Eric Yang)
  31. HADOOP-4827. * Replace Consolidator with Aggregator macros * Updated database table schema * Fix interval for aggregation execute * Improved SQL macros support, add group_avg() and past_*_minutes key word
      (Eric Yang)
• OPTIMIZATIONS    (none)
• BUG FIXES    (89)
  1. CHUKWA-249. Use ChukwaConfiguration in Demux.
     (Jerome Boulon via asrabkin)
  2. CHUKWA-257. fixed right arrow image for date picker.
     (Eric Yang)
  3. CHUKWA-251. Added per widget Time range input.
     (Terence Kwan via Eric Yang)
  4. CHUKWA-239. Demux settings now work out of the box.
     (Ari Rabkin)
  5. CHUKWA-250. Updated hadoop jar reference in chukwa config.
     (Cheng Zhang via Eric Yang)
  6. CHUKWA-211. Manage symlink correctly for RPM upgrade and uninstall.
     (Eric Yang)
  7. CHUKWA-245. Includes ChukwaJobTrackerInstrumentation class as part of the input tools compilation.
     (Eric Yang)
  8. CHUKWA-234. Improved handling of null session for XssFilter.
     (Eric Yang)
  9. CHUKWA-232. Fixed classpath for chukwa core jar file.
     (Eric Yang)
  10. CHUKWA-215. Corrected postProcess.sh environment setup.
      (Jerome Boulon via Eric Yang)
  11. CHUKWA-223. Renamed read only session caching object to cache.*, and skip those values for caching.
      (Eric Yang)
  12. CHUKWA-238. Resolve race condition in archiving.
      (Ari Rabkin)
  13. CHUKWA-228. Added rpm.hdfsusage.uid option to run HDFS usage as a separate user.
      (Cheng Zhang via Eric Yang)
  14. CHUKWA-227. Improved the user privilege for reading HDFS usage.
      (Cheng Zhang via Eric Yang)
  15. CHUKWA-229. Fix descriptor leak for ExecPlugin.
      (Ari Rabkin via Eric Yang)
  16. CHUKWA-212. Fix file descriptor leak in MDL.
      (Jerome Boulon via Eric Yang)
  17. CHUKWA-220. Correct min, max settings for yaxis charting.
      (Terence Kwan via Eric Yang)
  18. CHUKWA-222. Reduce Sar version dependency.
      (Ari Rabkin)
  19. CHUKWA-103. Fix documentation broken link.
      (Eric Yang)
  20. CHUKWA-208. Skip copying of test/sample directory if it doesn't exist.
      (Eric Yang)
  21. CHUKWA-206. Removed hard coded path from configuration files, and sample data.
      (Eric Yang)
  22. CHUKWA-201. Removed sourcing of Timeline widget from simile.mit.edu.
      (Eric Yang)
  23. CHUKWA-193. Remove unrecognized tag from Job History Log file parser.
      (Cheng Zhang via Eric Yang)
  24. CHUKWA-189. Added JAVA_LIBRARY_PATH to chukwa-env.sh for enabling compression.
      (Jerome Boulon via Eric Yang)
  25. CHUKWA-186. Remove duplicated timestamp from aggregation script.
      (Eric Yang)
  26. CHUKWA-187. Correction to status for job and task status.
      (Cheng Zhang via Eric Yang)
  27. CHUKWA-183. Added HICC startup script.
      (Eric Yang)
  28. CHUKWA-177. Added test case for verify the value between JSON values and database values.
      (Terence Kwan via Eric Yang)
  29. CHUKWA-181. Changed chukwa check point file to $CHUKWA_LOG_DIR.
      (Jerome Boulon via Eric Yang)
  30. CHUKWA-182. Added nagios appender configuration.
      (Eric Yang)
  31. CHUKWA-158. Consolidate namenode address to a single configuration object.
      (Eric Yang)
  32. CHUKWA-168. Added watchdog for database.
      (Eric Yang)
  33. CHUKWA-175. Removed error message for shutting down data processors.
      (Eric Yang)
  34. CHUKWA-155. Store final job status only
      (Cheng Zhang via Eric Yang)
  35. CHUKWA-164. Use year corresponding to the sender time stamp.
      (Cheng Zhang via Eric Yang)
  36. CHUKWA-166. Handle null parameter case for XSSFilter.
      (Terence Kwan via Eric Yang)
  37. CHUKWA-156. Test Macro testcase changed to use timestamp check for the generated macros.
      (Eric Yang)
  38. CHUKWA-154. Handle adaptor exception, close file pointers on failure condition.
      (Jerome Boulon via Eric Yang)
  39. CHUKWA-139. Rewrite collector bail out code.
      (Cheng Zhang via Eric Yang)
  40. CHUKWA-119. Removed dependency of ChukwaAgent from ChunkImpl for preventing multiple MetricsContext to be initialized in the same VM.
      (Jerome Boulon via Eric Yang)
  41. CHUKWA-121. Added logic to detect partition number less than 0.
      (Eric Yang)
  42. CHUKWA-132. Handle multi-line output in Job History file more gracefully.
      (Cheng Zhang via Eric Yang)
  43. CHUKWA-124. Fixed JDBC connection initialization and closing. Added escapeQuotes utility method.
      (Eric Yang)
  44. CHUKWA-122. Updated logic for finding time partition for Macro.
      (Eric Yang)
  45. CHUKWA-125. Updated chukwa demux template to include JobHistory parser.
      (Eric Yang)
  46. CHUKWA-98. Added Daemon watcher to capture signal for pid file management.
      (Cheng Zhang via Eric Yang)
  47. CHUKWA-120. Added the missing commons-cli library.
      (Jerome Boulon via Eric Yang)
  48. CHUKWA-61. Added report widget for accounting information.
      (Eric Yang)
  49. CHUKWA-77. Added Database Schema for Hadoop accounting information.
      (Eric Yang)
  50. CHUKWA-92. AbstractMetricsContext was using the wrong value
      (Jerome Boulon via asrabkin)
  51. CHUKWA-70. Rewrite FileAdaptor.
      (Jerome Boulon via asrabkin)
  52. CHUKWA-93. Fix NPE in SeqFileWriter.
      (Jiaqi Tan via asrabkin)
  53. CHUKWA-1. Remove lzo job configuration from Chukwa data processors.
      (Contribute by Jerome Boulon via Eric Yang)
  54. CHUKWA-40. Check for null pointer exception before unregister adaptor.
      (Contribute By Jerome Boulon via Eric Yang)
  55. CHUKWA-75. Filter out iostat values that is greater than 1+e10.
  56. CHUKWA-47. ChukwaJobTrackerInstrumentation class extends JobTrackerMetricsInst and added finalizedJob method for stop streaming job history log file.
      (Jerome Boulon via Eric Yang)
  57. CHUKWA-66. Corrected TerminatorThread logger class name.
      (Eric Yang)
  58. CHUKWA-65. Redirect metrics log file to CHUKWA_LOG_DIR, and setup read/writable permission on the metrics log files.
      (Eric Yang)
  59. CHUKWA-73. Added Socket Timeout 60 seconds.
      (Jerome Boulon via Eric Yang)
  60. CHUKWA-58. Changed watchdog to look for CHUKWA_PID_DIR.
      (Eric Yang)
  61. CHUKWA-43. ChukwaLog4jAppender should send the current file offset instead of sending 0.
      (Jerome Boulon via asrabkin)
  62. CHUKWA-41. ChukwaLog4jAppender does not escape \n for exception.
      (Jerome Boulon via asrabkin)
  63. CHUKWA-28. Late initalization of log4j adaptor.
      (Jerome Boulon via asrabkin)
  64. CHUKWA-48. Cleanup code to resolve compiler warnings.
      (asrabkin)
  65. CHUKWA-9. MetricDataLoader should close JDBC connection.
      (Jerome Boulon via asrabkin)
  66. CHUKWA-37. Remove ChuckwaArchiveBuilder.
      (asrabkin)
  67. CHUKWA-33. Reformat code to fit hadoop style.
      (asrabkin)
  68. CHUKWA-8. Remove deprecated conf files.
      (eyang via asrabkin)
  69. CHUKWA-11. Remove non .template files from conf dir.
      (asrabkin)
  70. HADOOP-5409. Changed opt directory copy to a optional operation, if opt exist.
      (Eric Yang)
  71. HADOOP-5138. Fixing failing test cases by separating TestCharFileTailing and TestFileTailing adaptors.
      (Jerome Boulon via acmurthy)
  72. HADOOP-5401. Standardize control port conf option name as chukwaAgent.control.port
      (asrabkin)
  73. HADOOP-5087. Fix regex for command parsing.
      (asrabkin)
  74. HADOOP-5057. Better test coverage for missing checkpoint cases.
      (asrabkin)
  75. HADOOP-5055. Changed alert.conf location from $CHUKWA_HOME/conf/alert.conf to $CHUKWA_CONF_DIR/alert.conf.
      (Eric Yang)
  76. HADOOP-5054. Improved database partitioning by date.
      (Eric Yang)
  77. HADOOP-5051. * Added macro token subsitution for sum(table_name) * Added correct hdfs throughput aggregation SQL macros.
      (Eric Yang)
  78. HADOOP-5032. Export CHUKWA_CONF_DIR in chukwa-config.sh.
  79. HADOOP-4960. Changed metrics time from current system time to data source time.
  80. HADOOP-4959. Support parsing of top output for Redhat EL 5.1.
      (Eric Yang)
  81. HADOOP-4916. * Added external property file to reference location and ownership of chukwa * Added ability to control the user name to run chukwa
      (Eric Yang)
  82. HADOOP-4914. Added description fields to chukwa init.d scripts.
  83. HADOOP-4889. Move chown from post install phase to build phase of the RPM file.
      (Eric Yang)
  84. HADOOP-4884. Change date format from dd/mm/yyyy to yyyy/mm/dd for display in the chart tool tips.
      (Eric Yang)
  85. HADOOP-4860. Changed test cases into 3 different test classes to prevent Agent from loading chukwa_check_point file which interfered with test cases.
      (Eric Yang)
  86. HADOOP-4825. Replaced jps with ps ax for shutdown scripts. Clean up reference of jar file from top level instead of build directory.
      (Eric Yang)
  87. HADOOP-4805. Removed black list collector feature from Chukwa Agent HTTP Sender.
      (Eric Yang)
  88. HADOOP-4796. Change component unit test targets from "ant test-agent" to "ant -Dtestcase=TestAgent test".
      (Eric Yang)
  89. HADOOP-4791. Add build configuration parameter to specify where Chukwa will be installed for RPM packaging.
      (Eric Yang)

Release 0.1.1 - Unreleased

• IMPROVEMENTS    (4)
  1. HADOOP-4431. Add versionning/tags to Chukwa Chunk.
     (Jerome Boulon via Johan)
  2. HADOOP-4433. Improve data loader for collecting metrics and log files.
     (Eric Yang via omalley)
  3. HADOOP-5205. Change the value of CHUKWA_IDENT_STRING from demo to TODO-AGENTS-INSTANCE-NAME in chukwa-env.sh.template
     (Jerome Boulon via asrabkin)
  4. HADOOP-5033. Simplified ChukwaWriter API.
     (asrabkin)
• NEW FEATURES    (1)
  1. HADOOP-3719. Initial checkin of Chukwa, which is a data collection and analysis framework.
     (Jerome Boulon, Andy Konwinski, Ari Rabkin, and Eric Yang)