Modifier and Type | Field and Description |
---|---|
static String |
COLUMNS_BUFFER_SIZE_CONF_STR |
Constructor and Description |
---|
Writer(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.Path name)
Constructs a RCFile Writer.
|
Writer(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.Path name,
int bufferSize,
short replication,
long blockSize,
org.apache.hadoop.util.Progressable progress,
org.apache.hadoop.io.SequenceFile.Metadata metadata,
org.apache.hadoop.io.compress.CompressionCodec codec)
Constructs a RCFile Writer.
|
Writer(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.Path name,
org.apache.hadoop.util.Progressable progress,
org.apache.hadoop.io.compress.CompressionCodec codec)
Constructs a RCFile Writer.
|
Writer(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.conf.Configuration conf,
org.apache.hadoop.fs.Path name,
org.apache.hadoop.util.Progressable progress,
org.apache.hadoop.io.SequenceFile.Metadata metadata,
org.apache.hadoop.io.compress.CompressionCodec codec)
Constructs a RCFile Writer.
|
Modifier and Type | Method and Description |
---|---|
void |
append(org.apache.hadoop.io.Writable val)
Append a row of values.
|
void |
close() |
void |
flushBlock(RCFile.KeyBuffer keyBuffer,
RCFile.ValueBuffer valueBuffer,
int recordLen,
int keyLength,
int compressedKeyLen)
flush a block out without doing anything except compressing the key part.
|
org.apache.hadoop.io.compress.CompressionCodec |
getCompressionCodec()
Deprecated.
|
long |
getLength() |
void |
sync()
create a sync point.
|
public static String COLUMNS_BUFFER_SIZE_CONF_STR
public Writer(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path name) throws IOException
IOException
public Writer(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path name, org.apache.hadoop.util.Progressable progress, org.apache.hadoop.io.compress.CompressionCodec codec) throws IOException
fs
- the file system usedconf
- the configuration filename
- the file nameIOException
public Writer(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path name, org.apache.hadoop.util.Progressable progress, org.apache.hadoop.io.SequenceFile.Metadata metadata, org.apache.hadoop.io.compress.CompressionCodec codec) throws IOException
fs
- the file system usedconf
- the configuration filename
- the file nameprogress
- a progress meter to update as the file is writtenmetadata
- a string to string map in the file headerIOException
public Writer(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.Path name, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress, org.apache.hadoop.io.SequenceFile.Metadata metadata, org.apache.hadoop.io.compress.CompressionCodec codec) throws IOException
fs
- the file system usedconf
- the configuration filename
- the file namebufferSize
- the size of the file bufferreplication
- the number of replicas for the fileblockSize
- the block size of the fileprogress
- the progress meter for writing the filemetadata
- a string to string map in the file headerIOException
public long getLength() throws IOException
IOException
@Deprecated public org.apache.hadoop.io.compress.CompressionCodec getCompressionCodec()
public void sync() throws IOException
IOException
public void append(org.apache.hadoop.io.Writable val) throws IOException
BytesRefArrayWritable
. If its size()
is less than the
column number in the file, zero bytes are appended for the empty columns.
If its size() is greater then the column number in the file, the exceeded
columns' bytes are ignored.val
- a BytesRefArrayWritable with the list of serialized columnsIOException
public void flushBlock(RCFile.KeyBuffer keyBuffer, RCFile.ValueBuffer valueBuffer, int recordLen, int keyLength, int compressedKeyLen) throws IOException
IOException
public void close() throws IOException
IOException
Copyright © 2017 The Apache Software Foundation. All rights reserved.