public class RemoteHiveSparkClient extends Object implements HiveSparkClient
SparkClient
, which
wrap a spark job request and send to an remote SparkContext.Modifier and Type | Method and Description |
---|---|
void |
close() |
SparkJobRef |
execute(DriverContext driverContext,
SparkWork sparkWork)
HiveSparkClient should generate Spark RDD graph by given sparkWork and driverContext,
and submit RDD graph to Spark cluster.
|
int |
getDefaultParallelism()
For standalone mode, this can be used to get total number of cores.
|
int |
getExecutorCount() |
org.apache.spark.SparkConf |
getSparkConf() |
public org.apache.spark.SparkConf getSparkConf()
getSparkConf
in interface HiveSparkClient
public int getExecutorCount() throws Exception
getExecutorCount
in interface HiveSparkClient
Exception
public int getDefaultParallelism() throws Exception
HiveSparkClient
getDefaultParallelism
in interface HiveSparkClient
Exception
public SparkJobRef execute(DriverContext driverContext, SparkWork sparkWork) throws Exception
HiveSparkClient
execute
in interface HiveSparkClient
Exception
public void close()
close
in interface Closeable
close
in interface AutoCloseable
Copyright © 2016 The Apache Software Foundation. All rights reserved.