org.apache.hadoop.hive.ql.udf.generic

Class GenericUDF

java.lang.Object

org.apache.hadoop.hive.ql.udf.generic.GenericUDF

All Implemented Interfaces:

Closeable, AutoCloseable

Direct Known Subclasses:

AbstractGenericUDFEWAHBitmapBop, AbstractGenericUDFReflect, GenericUDFAbs, GenericUDFAddMonths, GenericUDFArray, GenericUDFArrayContains, GenericUDFAssertTrue, GenericUDFBaseBinary, GenericUDFBasePad, GenericUDFBaseTrim, GenericUDFBaseUnary, GenericUDFBetween, GenericUDFBridge, GenericUDFCase, GenericUDFCbrt, GenericUDFCoalesce, GenericUDFConcat, GenericUDFConcatWS, GenericUDFCurrentDate, GenericUDFCurrentTimestamp, GenericUDFCurrentUser, GenericUDFDate, GenericUDFDateAdd, GenericUDFDateDiff, GenericUDFDateFormat, GenericUDFDateSub, GenericUDFDBOutput, GenericUDFDecode, GenericUDFElt, GenericUDFEncode, GenericUDFEWAHBitmapEmpty, GenericUDFFactorial, GenericUDFField, GenericUDFFloorCeilBase, GenericUDFFormatNumber, GenericUDFFromUtcTimestamp, GenericUDFGreatest, GenericUDFHash, GenericUDFIf, GenericUDFIn, GenericUDFIndex, GenericUDFInFile, GenericUDFInitCap, GenericUDFInstr, GenericUDFLastDay, GenericUDFLeadLag, GenericUDFLevenshtein, GenericUDFLocate, GenericUDFLower, GenericUDFMacro, GenericUDFMap, GenericUDFMapKeys, GenericUDFMapValues, GenericUDFMonthsBetween, GenericUDFNamedStruct, GenericUDFNextDay, GenericUDFNvl, GenericUDFOPAnd, GenericUDFOPNot, GenericUDFOPNotNull, GenericUDFOPNull, GenericUDFOPOr, GenericUDFPower, GenericUDFPrintf, GenericUDFRound, GenericUDFSentences, GenericUDFSize, GenericUDFSortArray, GenericUDFSoundex, GenericUDFSplit, GenericUDFStringToMap, GenericUDFStruct, GenericUDFTimestamp, GenericUDFToBinary, GenericUDFToChar, GenericUDFToDate, GenericUDFToDecimal, GenericUDFToIntervalDayTime, GenericUDFToIntervalYearMonth, GenericUDFToUnixTimeStamp, GenericUDFToVarchar, GenericUDFTranslate, GenericUDFTrunc, GenericUDFUnion, GenericUDFUpper, GenericUDFWhen, GenericUDFXPath, UDFCurrentDB

public abstract class GenericUDF
extends Object
implements Closeable

A Generic User-defined function (GenericUDF) for the use with Hive. New GenericUDF classes need to inherit from this GenericUDF class. The GenericUDF are superior to normal UDFs in the following ways: 1. It can accept arguments of complex types, and return complex types. 2. It can accept variable length of arguments. 3. It can accept an infinite number of function signature - for example, it's easy to write a GenericUDF that accepts array, array> and so on (arbitrary levels of nesting). 4. It can do short-circuit evaluations using DeferedObject.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	GenericUDF.DeferredJavaObject A basic dummy implementation of DeferredObject which just stores a Java Object reference.
static interface	GenericUDF.DeferredObject A Defered Object allows us to do lazy-evaluation and short-circuiting.

Constructor Summary

Constructors

Constructor and Description
GenericUDF() The constructor.

Method Summary

Methods

Modifier and Type	Method and Description
protected void	checkArgGroups(ObjectInspector[] arguments, int i, PrimitiveObjectInspector.PrimitiveCategory[] inputTypes, PrimitiveObjectInspectorUtils.PrimitiveGrouping... grps)
protected void	checkArgPrimitive(ObjectInspector[] arguments, int i)
protected void	checkArgsSize(ObjectInspector[] arguments, int min, int max)
void	close() Close GenericUDF.
void	configure(MapredContext context) Additionally setup GenericUDF with MapredContext before initializing.
void	copyToNewInstance(Object newInstance) Some information may be set during initialize() which needs to be saved when the UDF is copied.
abstract Object	evaluate(GenericUDF.DeferredObject[] arguments) Evaluate the GenericUDF with the arguments.
GenericUDF	flip() Some functions like comparisons may be affected by appearing order of arguments.
protected String	getArgOrder(int i)
protected Integer	getConstantIntValue(ObjectInspector[] arguments, int i)
protected Long	getConstantLongValue(ObjectInspector[] arguments, int i)
protected String	getConstantStringValue(ObjectInspector[] arguments, int i)
protected Date	getDateValue(GenericUDF.DeferredObject[] arguments, int i, PrimitiveObjectInspector.PrimitiveCategory[] inputTypes, ObjectInspectorConverters.Converter[] converters)
abstract String	getDisplayString(String[] children) Get the String to be displayed in explain.
protected Double	getDoubleValue(GenericUDF.DeferredObject[] arguments, int i, ObjectInspectorConverters.Converter[] converters)
protected String	getFuncName()
protected Integer	getIntValue(GenericUDF.DeferredObject[] arguments, int i, ObjectInspectorConverters.Converter[] converters)
protected Long	getLongValue(GenericUDF.DeferredObject[] arguments, int i, ObjectInspectorConverters.Converter[] converters)
String[]	getRequiredFiles()
String[]	getRequiredJars() The following two functions can be overridden to automatically include additional resources required by this UDF.
protected String	getStandardDisplayString(String name, String[] children)
protected String	getStandardDisplayString(String name, String[] children, String delim)
protected String	getStringValue(GenericUDF.DeferredObject[] arguments, int i, ObjectInspectorConverters.Converter[] converters)
protected Timestamp	getTimestampValue(GenericUDF.DeferredObject[] arguments, int i, ObjectInspectorConverters.Converter[] converters)
String	getUdfName()
abstract ObjectInspector	initialize(ObjectInspector[] arguments) Initialize this GenericUDF.
ObjectInspector	initializeAndFoldConstants(ObjectInspector[] arguments) Initialize this GenericUDF.
GenericUDF	negative() Gets the negative function of the current one.
protected void	obtainDateConverter(ObjectInspector[] arguments, int i, PrimitiveObjectInspector.PrimitiveCategory[] inputTypes, ObjectInspectorConverters.Converter[] converters)
protected void	obtainDoubleConverter(ObjectInspector[] arguments, int i, PrimitiveObjectInspector.PrimitiveCategory[] inputTypes, ObjectInspectorConverters.Converter[] converters)
protected void	obtainIntConverter(ObjectInspector[] arguments, int i, PrimitiveObjectInspector.PrimitiveCategory[] inputTypes, ObjectInspectorConverters.Converter[] converters)
protected void	obtainLongConverter(ObjectInspector[] arguments, int i, PrimitiveObjectInspector.PrimitiveCategory[] inputTypes, ObjectInspectorConverters.Converter[] converters)
protected void	obtainStringConverter(ObjectInspector[] arguments, int i, PrimitiveObjectInspector.PrimitiveCategory[] inputTypes, ObjectInspectorConverters.Converter[] converters)
protected void	obtainTimestampConverter(ObjectInspector[] arguments, int i, PrimitiveObjectInspector.PrimitiveCategory[] inputTypes, ObjectInspectorConverters.Converter[] converters)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GenericUDF

public GenericUDF()

The constructor.

Method Detail

initialize

public abstract ObjectInspector initialize(ObjectInspector[] arguments)
                                    throws UDFArgumentException

Initialize this GenericUDF. This will be called once and only once per GenericUDF instance.

Parameters:

arguments - The ObjectInspector for the arguments

Returns:

The ObjectInspector for the return value

Throws:

UDFArgumentException - Thrown when arguments have wrong types, wrong length, etc.

configure

public void configure(MapredContext context)

Additionally setup GenericUDF with MapredContext before initializing. This is only called in runtime of MapRedTask.

Parameters:

context - context

initializeAndFoldConstants

public ObjectInspector initializeAndFoldConstants(ObjectInspector[] arguments)
                                           throws UDFArgumentException

Initialize this GenericUDF. Additionally, if the arguments are constant and the function is eligible to be folded, then the constant value returned by this UDF will be computed and stored in the ConstantObjectInspector returned. Otherwise, the function behaves exactly like initialize().

Throws:

UDFArgumentException

getRequiredJars

public String[] getRequiredJars()

The following two functions can be overridden to automatically include additional resources required by this UDF. The return types should be arrays of paths.

getRequiredFiles

public String[] getRequiredFiles()

evaluate

public abstract Object evaluate(GenericUDF.DeferredObject[] arguments)
                         throws HiveException

Evaluate the GenericUDF with the arguments.

Parameters:

arguments - The arguments as DeferedObject, use DeferedObject.get() to get the actual argument Object. The Objects can be inspected by the ObjectInspectors passed in the initialize call.

Returns:

The

Throws:

HiveException

getDisplayString

public abstract String getDisplayString(String[] children)

Get the String to be displayed in explain.

close

public void close()
           throws IOException

Close GenericUDF. This is only called in runtime of MapRedTask.

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Throws:

IOException

flip

public GenericUDF flip()

Some functions like comparisons may be affected by appearing order of arguments. This is to convert a function, such as 3 > x to x < 3. The flip function of GenericUDFOPGreaterThan is GenericUDFOPLessThan.

negative

public GenericUDF negative()

Gets the negative function of the current one. E.g., GenericUDFOPNotEqual for GenericUDFOPEqual, or GenericUDFOPNull for GenericUDFOPNotNull.

Returns:

Negative function

Throws:

UDFArgumentException

getUdfName

public String getUdfName()

copyToNewInstance

public void copyToNewInstance(Object newInstance)
                       throws UDFArgumentException

Some information may be set during initialize() which needs to be saved when the UDF is copied. This will be called by FunctionRegistry.cloneGenericUDF()

Throws:

UDFArgumentException

getStandardDisplayString

protected String getStandardDisplayString(String name,
                              String[] children)

getStandardDisplayString

protected String getStandardDisplayString(String name,
                              String[] children,
                              String delim)

getFuncName

protected String getFuncName()

checkArgsSize

protected void checkArgsSize(ObjectInspector[] arguments,
                 int min,
                 int max)
                      throws UDFArgumentLengthException

Throws:

UDFArgumentLengthException

checkArgPrimitive

protected void checkArgPrimitive(ObjectInspector[] arguments,
                     int i)
                          throws UDFArgumentTypeException

Throws:

UDFArgumentTypeException

checkArgGroups

protected void checkArgGroups(ObjectInspector[] arguments,
                  int i,
                  PrimitiveObjectInspector.PrimitiveCategory[] inputTypes,
                  PrimitiveObjectInspectorUtils.PrimitiveGrouping... grps)
                       throws UDFArgumentTypeException

Throws:

UDFArgumentTypeException

obtainStringConverter

protected void obtainStringConverter(ObjectInspector[] arguments,
                         int i,
                         PrimitiveObjectInspector.PrimitiveCategory[] inputTypes,
                         ObjectInspectorConverters.Converter[] converters)
                              throws UDFArgumentTypeException

Throws:

UDFArgumentTypeException

obtainIntConverter

protected void obtainIntConverter(ObjectInspector[] arguments,
                      int i,
                      PrimitiveObjectInspector.PrimitiveCategory[] inputTypes,
                      ObjectInspectorConverters.Converter[] converters)
                           throws UDFArgumentTypeException

Throws:

UDFArgumentTypeException

obtainLongConverter

protected void obtainLongConverter(ObjectInspector[] arguments,
                       int i,
                       PrimitiveObjectInspector.PrimitiveCategory[] inputTypes,
                       ObjectInspectorConverters.Converter[] converters)
                            throws UDFArgumentTypeException

Throws:

UDFArgumentTypeException

obtainDoubleConverter

protected void obtainDoubleConverter(ObjectInspector[] arguments,
                         int i,
                         PrimitiveObjectInspector.PrimitiveCategory[] inputTypes,
                         ObjectInspectorConverters.Converter[] converters)
                              throws UDFArgumentTypeException

Throws:

UDFArgumentTypeException

obtainDateConverter

protected void obtainDateConverter(ObjectInspector[] arguments,
                       int i,
                       PrimitiveObjectInspector.PrimitiveCategory[] inputTypes,
                       ObjectInspectorConverters.Converter[] converters)
                            throws UDFArgumentTypeException

Throws:

UDFArgumentTypeException

obtainTimestampConverter

protected void obtainTimestampConverter(ObjectInspector[] arguments,
                            int i,
                            PrimitiveObjectInspector.PrimitiveCategory[] inputTypes,
                            ObjectInspectorConverters.Converter[] converters)
                                 throws UDFArgumentTypeException

Throws:

UDFArgumentTypeException

getStringValue

protected String getStringValue(GenericUDF.DeferredObject[] arguments,
                    int i,
                    ObjectInspectorConverters.Converter[] converters)
                         throws HiveException

Throws:

HiveException

getIntValue

protected Integer getIntValue(GenericUDF.DeferredObject[] arguments,
                  int i,
                  ObjectInspectorConverters.Converter[] converters)
                       throws HiveException

Throws:

HiveException

getLongValue

protected Long getLongValue(GenericUDF.DeferredObject[] arguments,
                int i,
                ObjectInspectorConverters.Converter[] converters)
                     throws HiveException

Throws:

HiveException

getDoubleValue

protected Double getDoubleValue(GenericUDF.DeferredObject[] arguments,
                    int i,
                    ObjectInspectorConverters.Converter[] converters)
                         throws HiveException

Throws:

HiveException

getDateValue

protected Date getDateValue(GenericUDF.DeferredObject[] arguments,
                int i,
                PrimitiveObjectInspector.PrimitiveCategory[] inputTypes,
                ObjectInspectorConverters.Converter[] converters)
                     throws HiveException

Throws:

HiveException

getTimestampValue

protected Timestamp getTimestampValue(GenericUDF.DeferredObject[] arguments,
                          int i,
                          ObjectInspectorConverters.Converter[] converters)
                               throws HiveException

Throws:

HiveException

getConstantStringValue

protected String getConstantStringValue(ObjectInspector[] arguments,
                            int i)

getConstantIntValue

protected Integer getConstantIntValue(ObjectInspector[] arguments,
                          int i)
                               throws UDFArgumentTypeException

Throws:

UDFArgumentTypeException

getConstantLongValue

protected Long getConstantLongValue(ObjectInspector[] arguments,
                        int i)
                             throws UDFArgumentTypeException

Throws:

UDFArgumentTypeException

getArgOrder

protected String getArgOrder(int i)