org.apache.hadoop.hive.ql.stats.fs

Class FSStatsPublisher

java.lang.Object

org.apache.hadoop.hive.ql.stats.fs.FSStatsPublisher

All Implemented Interfaces:

StatsCollectionTaskIndependent, StatsPublisher

public class FSStatsPublisher
extends Object
implements StatsPublisher, StatsCollectionTaskIndependent

Constructor Summary

Constructors

Constructor and Description
FSStatsPublisher()

Method Summary

Methods

Modifier and Type	Method and Description
boolean	closeConnection() This method closes the connection to the temporary storage.
boolean	connect(org.apache.hadoop.conf.Configuration hconf) This method connects to the intermediate statistics database.
boolean	init(org.apache.hadoop.conf.Configuration hconf) This method does the necessary one-time initializations, possibly creating the tables and database (if not exist).
boolean	publishStat(String partKV, Map<String,String> stats) This method publishes a given statistic into a disk storage, possibly HBase or MySQL.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

FSStatsPublisher

public FSStatsPublisher()

Method Detail

init

public boolean init(org.apache.hadoop.conf.Configuration hconf)

Description copied from interface: StatsPublisher

This method does the necessary one-time initializations, possibly creating the tables and database (if not exist). This method is usually called in the Hive client side rather than by the mappers/reducers so that it is initialized only once.

Specified by:

init in interface StatsPublisher

Parameters:

hconf - HiveConf that contains the configurations parameters used to connect to intermediate stats database.

Returns:

true if initialization is successful, false otherwise.

connect

public boolean connect(org.apache.hadoop.conf.Configuration hconf)

Description copied from interface: StatsPublisher

This method connects to the intermediate statistics database.

Specified by:

connect in interface StatsPublisher

Parameters:

hconf - HiveConf that contains the connection parameters.

Returns:

true if connection is successful, false otherwise.

publishStat

public boolean publishStat(String partKV,
                  Map<String,String> stats)

Description copied from interface: StatsPublisher

This method publishes a given statistic into a disk storage, possibly HBase or MySQL.

Specified by:

publishStat in interface StatsPublisher

Parameters:

partKV - : a string identification the statistics to be published by all mappers/reducers and then gathered. The statID is unique per output partition per task, e.g.,: the output directory name (uniq per FileSinkOperator) + the partition specs (only for dynamic partitions) + taskID (last component of task file)

stats - : a map containing key-value pairs, where key is a string representing the statistic to be published, and value is a string representing the value for the given statistic

Returns:

true if successful, false otherwise

closeConnection

public boolean closeConnection()

Description copied from interface: StatsPublisher

This method closes the connection to the temporary storage.

Specified by:

closeConnection in interface StatsPublisher