|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object org.apache.hadoop.mapreduce.InputFormat<K,T> org.apache.gora.mapreduce.GoraInputFormat<K,T>
public class GoraInputFormat<K,T extends Persistent>
InputFormat
to fetch the input from Gora data stores. The
query to fetch the items from the datastore should be prepared and
set via setQuery(Job, Query)
, before submitting the job.
The InputSplit
s are prepared from the PartitionQuery
s
obtained by calling DataStore.getPartitions(Query)
.
Hadoop jobs can be either configured through static
setInput()
methods, or from GoraMapper
.
GoraMapper
Field Summary | |
---|---|
static String |
QUERY_KEY
|
Constructor Summary | |
---|---|
GoraInputFormat()
|
Method Summary | ||
---|---|---|
org.apache.hadoop.mapreduce.RecordReader<K,T> |
createRecordReader(org.apache.hadoop.mapreduce.InputSplit split,
org.apache.hadoop.mapreduce.TaskAttemptContext context)
|
|
org.apache.hadoop.conf.Configuration |
getConf()
|
|
Query<K,T> |
getQuery(org.apache.hadoop.conf.Configuration conf)
|
|
List<org.apache.hadoop.mapreduce.InputSplit> |
getSplits(org.apache.hadoop.mapreduce.JobContext context)
|
|
void |
setConf(org.apache.hadoop.conf.Configuration conf)
|
|
static
|
setInput(org.apache.hadoop.mapreduce.Job job,
Class<? extends DataStore<K1,V1>> dataStoreClass,
Class<K1> inKeyClass,
Class<V1> inValueClass,
boolean reuseObjects)
Sets the input parameters for the job |
|
static
|
setInput(org.apache.hadoop.mapreduce.Job job,
Query<K1,V1> query,
boolean reuseObjects)
Sets the input parameters for the job |
|
static
|
setInput(org.apache.hadoop.mapreduce.Job job,
Query<K1,V1> query,
DataStore<K1,V1> dataStore,
boolean reuseObjects)
Sets the input parameters for the job |
|
static
|
setQuery(org.apache.hadoop.mapreduce.Job job,
Query<K,T> query)
|
Methods inherited from class java.lang.Object |
---|
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
public static final String QUERY_KEY
Constructor Detail |
---|
public GoraInputFormat()
Method Detail |
---|
public org.apache.hadoop.mapreduce.RecordReader<K,T> createRecordReader(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context) throws IOException, InterruptedException
createRecordReader
in class org.apache.hadoop.mapreduce.InputFormat<K,T extends Persistent>
IOException
InterruptedException
public List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext context) throws IOException, InterruptedException
getSplits
in class org.apache.hadoop.mapreduce.InputFormat<K,T extends Persistent>
IOException
InterruptedException
public org.apache.hadoop.conf.Configuration getConf()
getConf
in interface org.apache.hadoop.conf.Configurable
public void setConf(org.apache.hadoop.conf.Configuration conf)
setConf
in interface org.apache.hadoop.conf.Configurable
public static <K,T extends Persistent> void setQuery(org.apache.hadoop.mapreduce.Job job, Query<K,T> query) throws IOException
IOException
public Query<K,T> getQuery(org.apache.hadoop.conf.Configuration conf) throws IOException
IOException
public static <K1,V1 extends Persistent> void setInput(org.apache.hadoop.mapreduce.Job job, Query<K1,V1> query, boolean reuseObjects) throws IOException
job
- the job to set the properties forquery
- the query to get the inputs fromreuseObjects
- whether to reuse objects in serialization
IOException
public static <K1,V1 extends Persistent> void setInput(org.apache.hadoop.mapreduce.Job job, Query<K1,V1> query, DataStore<K1,V1> dataStore, boolean reuseObjects) throws IOException
job
- the job to set the properties forquery
- the query to get the inputs fromdataStore
- the datastore as the inputreuseObjects
- whether to reuse objects in serialization
IOException
public static <K1,V1 extends Persistent> void setInput(org.apache.hadoop.mapreduce.Job job, Class<? extends DataStore<K1,V1>> dataStoreClass, Class<K1> inKeyClass, Class<V1> inValueClass, boolean reuseObjects) throws IOException
job
- the job to set the properties fordataStoreClass
- the datastore classinKeyClass
- Map input key classinValueClass
- Map input value classreuseObjects
- whether to reuse objects in serialization
IOException
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |