org.apache.hadoop.hive.ql.exec.spark.session

Interface SparkSessionManager

All Known Implementing Classes:

SparkSessionManagerImpl

public interface SparkSessionManager

Defines interface for managing multiple SparkSessions in Hive when multiple users are executing queries simultaneously on Spark execution engine.

Method Summary

Methods

Modifier and Type	Method and Description
void	closeSession(SparkSession sparkSession) Close the given session and return it to pool.
SparkSession	getSession(SparkSession existingSession, HiveConf conf, boolean doOpen) Get a valid SparkSession.
void	returnSession(SparkSession sparkSession) Return the given sparkSession to pool.
void	setup(HiveConf hiveConf) Initialize based on given configuration.
void	shutdown() Shutdown the session manager.

Method Detail

setup

void setup(HiveConf hiveConf)
           throws HiveException

Initialize based on given configuration.

Parameters:

hiveConf -

Throws:

HiveException

getSession

SparkSession getSession(SparkSession existingSession,
                      HiveConf conf,
                      boolean doOpen)
                        throws HiveException

Get a valid SparkSession. First try to check if existing session is reusable based on the given conf. If not release existingSession and return a new session based on session manager criteria and conf.

Parameters:

existingSession - Existing session (can be null)

conf -

doOpen - Should the session be opened before returning?

Returns:

SparkSession

Throws:

HiveException

returnSession

void returnSession(SparkSession sparkSession)
                   throws HiveException

Return the given sparkSession to pool. This is used when the client still holds references to session and may want to reuse it in future. When client wants to reuse the session, it should pass the it getSession method.

Throws:

HiveException

closeSession

void closeSession(SparkSession sparkSession)
                  throws HiveException

Close the given session and return it to pool. This is used when the client no longer needs a SparkSession.

Throws:

HiveException

shutdown

void shutdown()

Shutdown the session manager. Also closing up SparkSessions in pool.