public interface SparkSession
Modifier and Type | Method and Description |
---|---|
void |
close()
Close session and release resources.
|
HiveConf |
getConf() |
org.apache.hadoop.fs.Path |
getHDFSSessionDir()
Get an HDFS dir specific to the SparkSession
|
ObjectPair<Long,Integer> |
getMemoryAndCores()
Get Spark shuffle memory per task, and total number of cores.
|
String |
getSessionId() |
boolean |
isOpen() |
void |
open(HiveConf conf)
Initializes a Spark session for DAG execution.
|
SparkJobRef |
submit(DriverContext driverContext,
SparkWork sparkWork)
Submit given sparkWork to SparkClient.
|
void open(HiveConf conf) throws HiveException
conf
- Hive configuration.HiveException
SparkJobRef submit(DriverContext driverContext, SparkWork sparkWork) throws Exception
driverContext
- sparkWork
- Exception
ObjectPair<Long,Integer> getMemoryAndCores() throws Exception
Exception
boolean isOpen()
HiveConf getConf()
String getSessionId()
void close()
org.apache.hadoop.fs.Path getHDFSSessionDir() throws IOException
IOException
Copyright © 2021 The Apache Software Foundation. All rights reserved.