org.apache.hadoop.hive.ql.exec.spark.session

Class SparkSessionManagerImpl

java.lang.Object

org.apache.hadoop.hive.ql.exec.spark.session.SparkSessionManagerImpl

All Implemented Interfaces:

SparkSessionManager

public class SparkSessionManagerImpl
extends Object
implements SparkSessionManager

Simple implementation of SparkSessionManager - returns SparkSession when requested through getSession and keeps track of created sessions. Currently no limit on the number sessions. - SparkSession is reused if the userName in new conf and user name in session conf match.

Method Summary

Methods

Modifier and Type	Method and Description
void	closeSession(SparkSession sparkSession) Close the given session and return it to pool.
static SparkSessionManagerImpl	getInstance()
SparkSession	getSession(SparkSession existingSession, HiveConf conf, boolean doOpen) If the existingSession can be reused return it.
void	returnSession(SparkSession sparkSession) Return the given sparkSession to pool.
void	setup(HiveConf hiveConf) Initialize based on given configuration.
void	shutdown() Shutdown the session manager.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getInstance

public static SparkSessionManagerImpl getInstance()
                                           throws HiveException

Throws:

HiveException

setup

public void setup(HiveConf hiveConf)
           throws HiveException

Description copied from interface: SparkSessionManager

Initialize based on given configuration.

Specified by:

setup in interface SparkSessionManager

Throws:

HiveException

getSession

public SparkSession getSession(SparkSession existingSession,
                      HiveConf conf,
                      boolean doOpen)
                        throws HiveException

If the existingSession can be reused return it. Otherwise - close it and remove it from the list. - create a new session and add it to the list.

Specified by:

getSession in interface SparkSessionManager

Parameters:

existingSession - Existing session (can be null)

doOpen - Should the session be opened before returning?

Returns:

SparkSession

Throws:

HiveException

returnSession

public void returnSession(SparkSession sparkSession)
                   throws HiveException

Description copied from interface: SparkSessionManager

Return the given sparkSession to pool. This is used when the client still holds references to session and may want to reuse it in future. When client wants to reuse the session, it should pass the it getSession method.

Specified by:

returnSession in interface SparkSessionManager

Throws:

HiveException

closeSession

public void closeSession(SparkSession sparkSession)
                  throws HiveException

Description copied from interface: SparkSessionManager

Close the given session and return it to pool. This is used when the client no longer needs a SparkSession.

Specified by:

closeSession in interface SparkSessionManager

Throws:

HiveException

shutdown

public void shutdown()

Description copied from interface: SparkSessionManager

Shutdown the session manager. Also closing up SparkSessions in pool.

Specified by:

shutdown in interface SparkSessionManager