org.apache.crunch.contrib.io.jdbc

Class DataBaseSource<T extends org.apache.hadoop.mapreduce.lib.db.DBWritable & org.apache.hadoop.io.Writable>

java.lang.Object

org.apache.crunch.io.impl.FileSourceImpl<T>

org.apache.crunch.contrib.io.jdbc.DataBaseSource<T>

Type Parameters:

T - The input type of this source

All Implemented Interfaces:

ReadableSource<T>, Source<T>

public class DataBaseSource<T extends org.apache.hadoop.mapreduce.lib.db.DBWritable & org.apache.hadoop.io.Writable>
extends org.apache.crunch.io.impl.FileSourceImpl<T>

Source from reading from a database via a JDBC connection. Underlying database reading is provided by DBInputFormat.

A type that is input via this class must be a Writable that also implements DBWritable. On the DBWritable.readFields(java.sql.ResultSet) method needs to be fully implemented form DBWritable.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	DataBaseSource.Builder<T extends org.apache.hadoop.mapreduce.lib.db.DBWritable & org.apache.hadoop.io.Writable>

Method Summary

Modifier and Type	Method and Description
long	getLastModifiedAt(org.apache.hadoop.conf.Configuration configuration) Returns the time (in milliseconds) that this Source was most recently modified (e.g., because an input file was edited or new files were added to a directory.)
long	getSize(org.apache.hadoop.conf.Configuration configuration) Returns the number of bytes in this Source.

Methods inherited from class org.apache.crunch.io.impl.FileSourceImpl

asReadable, configureSource, equals, getBundle, getConverter, getPath, getPaths, getType, hashCode, inputConf, read, toString

Methods inherited from class java.lang.Object

getClass, notify, notifyAll, wait, wait, wait

Method Detail

getSize

public long getSize(org.apache.hadoop.conf.Configuration configuration)

Description copied from interface: Source

Returns the number of bytes in this Source.

Specified by:

getSize in interface Source<T extends org.apache.hadoop.mapreduce.lib.db.DBWritable & org.apache.hadoop.io.Writable>

Overrides:

getSize in class org.apache.crunch.io.impl.FileSourceImpl<T extends org.apache.hadoop.mapreduce.lib.db.DBWritable & org.apache.hadoop.io.Writable>

getLastModifiedAt

public long getLastModifiedAt(org.apache.hadoop.conf.Configuration configuration)

Description copied from interface: Source

Returns the time (in milliseconds) that this Source was most recently modified (e.g., because an input file was edited or new files were added to a directory.)

Specified by:

getLastModifiedAt in interface Source<T extends org.apache.hadoop.mapreduce.lib.db.DBWritable & org.apache.hadoop.io.Writable>

Overrides:

getLastModifiedAt in class org.apache.crunch.io.impl.FileSourceImpl<T extends org.apache.hadoop.mapreduce.lib.db.DBWritable & org.apache.hadoop.io.Writable>