org.apache.hadoop.hive.serde2.lazybinary

Class LazyBinaryUtils

java.lang.Object

org.apache.hadoop.hive.serde2.lazybinary.LazyBinaryUtils

public final class LazyBinaryUtils
extends Object

LazyBinaryUtils.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	LazyBinaryUtils.RecordInfo Record is the unit that data is serialized in.
static class	LazyBinaryUtils.VInt A zero-compressed encoded integer.
static class	LazyBinaryUtils.VLong A zero-compressed encoded long.

Field Summary

Fields

Modifier and Type	Field and Description
static ThreadLocal<LazyBinaryUtils.VInt>	threadLocalVInt

Method Summary

Methods

Modifier and Type	Method and Description
static int	byteArrayToInt(byte[] b, int offset) Convert the byte array to an int starting from the given offset.
static long	byteArrayToLong(byte[] b, int offset) Convert the byte array to a long starting from the given offset.
static short	byteArrayToShort(byte[] b, int offset) Convert the byte array to a short starting from the given offset.
static void	checkObjectByteInfo(ObjectInspector objectInspector, byte[] bytes, int offset, LazyBinaryUtils.RecordInfo recordInfo, LazyBinaryUtils.VInt vInt) Check a particular field and set its size and offset in bytes based on the field type and the bytes arrays.
static ObjectInspector	getLazyBinaryObjectInspectorFromTypeInfo(TypeInfo typeInfo) Returns the lazy binary object inspector that can be used to inspect an lazy binary object of that typeInfo For primitive types, we use the standard writable object inspector.
static void	readVInt(byte[] bytes, int offset, LazyBinaryUtils.VInt vInt) Reads a zero-compressed encoded int from a byte array and returns it.
static void	readVLong(byte[] bytes, int offset, LazyBinaryUtils.VLong vlong) Reads a zero-compressed encoded long from a byte array and returns it.
static long	readVLongFromByteArray(byte[] bytes, int offset) Read a zero-compressed encoded long from a byte array.
static void	writeDouble(ByteStream.RandomAccessOutput byteStream, double d)
static void	writeVInt(ByteStream.RandomAccessOutput byteStream, int i) Writes a zero-compressed encoded int to a byte array.
static void	writeVLong(ByteStream.RandomAccessOutput byteStream, long l)
static int	writeVLongToByteArray(byte[] bytes, int offset, long l)
static int	writeVLongToByteArray(byte[] bytes, long l) Write a zero-compressed encoded long to a byte array.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

threadLocalVInt

public static final ThreadLocal<LazyBinaryUtils.VInt> threadLocalVInt

Method Detail

byteArrayToInt

public static int byteArrayToInt(byte[] b,
                 int offset)

Convert the byte array to an int starting from the given offset. Refer to code by aeden on DZone Snippets:

Parameters:

b - the byte array

offset - the array offset

Returns:

the integer

byteArrayToLong

public static long byteArrayToLong(byte[] b,
                   int offset)

Convert the byte array to a long starting from the given offset.

Parameters:

b - the byte array

offset - the array offset

Returns:

the long

byteArrayToShort

public static short byteArrayToShort(byte[] b,
                     int offset)

Convert the byte array to a short starting from the given offset.

Parameters:

b - the byte array

offset - the array offset

Returns:

the short

checkObjectByteInfo

public static void checkObjectByteInfo(ObjectInspector objectInspector,
                       byte[] bytes,
                       int offset,
                       LazyBinaryUtils.RecordInfo recordInfo,
                       LazyBinaryUtils.VInt vInt)

Check a particular field and set its size and offset in bytes based on the field type and the bytes arrays. For void, boolean, byte, short, int, long, float and double, there is no offset and the size is fixed. For string, map, list, struct, the first four bytes are used to store the size. So the offset is 4 and the size is computed by concating the first four bytes together. The first four bytes are defined with respect to the offset in the bytes arrays. For timestamp, if the first bit is 0, the record length is 4, otherwise a VInt begins at the 5th byte and its length is added to 4.

Parameters:

objectInspector - object inspector of the field

bytes - bytes arrays store the table row

offset - offset of this field

recordInfo - modify this byteinfo object and return it

readVLong

public static void readVLong(byte[] bytes,
             int offset,
             LazyBinaryUtils.VLong vlong)

Reads a zero-compressed encoded long from a byte array and returns it.

Parameters:

bytes - the byte array

offset - offset of the array to read from

vlong - storing the deserialized long and its size in byte

readVInt

public static void readVInt(byte[] bytes,
            int offset,
            LazyBinaryUtils.VInt vInt)

Reads a zero-compressed encoded int from a byte array and returns it.

Parameters:

bytes - the byte array

offset - offset of the array to read from

vInt - storing the deserialized int and its size in byte

writeVInt

public static void writeVInt(ByteStream.RandomAccessOutput byteStream,
             int i)

Writes a zero-compressed encoded int to a byte array.

Parameters:

byteStream - the byte array/stream

i - the int

readVLongFromByteArray

public static long readVLongFromByteArray(byte[] bytes,
                          int offset)

Read a zero-compressed encoded long from a byte array.

Parameters:

bytes - the byte array

offset - the offset in the byte array where the VLong is stored

Returns:

the long

writeVLongToByteArray

public static int writeVLongToByteArray(byte[] bytes,
                        long l)

Write a zero-compressed encoded long to a byte array.

Parameters:

bytes - the byte array/stream

l - the long

writeVLongToByteArray

public static int writeVLongToByteArray(byte[] bytes,
                        int offset,
                        long l)

writeVLong

public static void writeVLong(ByteStream.RandomAccessOutput byteStream,
              long l)

writeDouble

public static void writeDouble(ByteStream.RandomAccessOutput byteStream,
               double d)

getLazyBinaryObjectInspectorFromTypeInfo

public static ObjectInspector getLazyBinaryObjectInspectorFromTypeInfo(TypeInfo typeInfo)

Returns the lazy binary object inspector that can be used to inspect an lazy binary object of that typeInfo For primitive types, we use the standard writable object inspector.