org.apache.reef.runtime.yarn.driver

Class YarnDriverRuntimeRestartManager

java.lang.Object

org.apache.reef.runtime.yarn.driver.YarnDriverRuntimeRestartManager

All Implemented Interfaces:

DriverRuntimeRestartManager

public final class YarnDriverRuntimeRestartManager
extends Object
implements DriverRuntimeRestartManager

The implementation of restart manager for YARN. Handles evaluator preservation as well as evaluator recovery on YARN.

Method Summary

Methods

Modifier and Type	Method and Description
RestartEvaluators	getPreviousEvaluators() Used by DriverRestartManager.
int	getResubmissionAttempts() Determines the number of times the Driver has been submitted based on the container ID environment variable provided by YARN.
void	informAboutEvaluatorFailures(Set<String> evaluatorIds) Calls the appropriate handler via REEFEventHandlers, which is a runtime specific implementation of the YARN runtime.
void	recordAllocatedEvaluator(String id) Records the evaluators when it is allocated.
void	recordRemovedEvaluator(String id) Records a removed evaluator into the evaluator log.

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getResubmissionAttempts

public int getResubmissionAttempts()

Determines the number of times the Driver has been submitted based on the container ID environment variable provided by YARN. If that fails, determine whether the application master is a restart based on the number of previous containers reported by YARN. In the failure scenario, returns 1 if restart, 0 otherwise.

Specified by:

getResubmissionAttempts in interface DriverRuntimeRestartManager

Returns:

positive value if the application master is a restarted instance, 0 otherwise.

recordAllocatedEvaluator

public void recordAllocatedEvaluator(String id)

Description copied from interface: DriverRuntimeRestartManager

Records the evaluators when it is allocated.

Specified by:

recordAllocatedEvaluator in interface DriverRuntimeRestartManager

Parameters:

id - The evaluator ID of the allocated evaluator.

recordRemovedEvaluator

public void recordRemovedEvaluator(String id)

Description copied from interface: DriverRuntimeRestartManager

Records a removed evaluator into the evaluator log.

Specified by:

recordRemovedEvaluator in interface DriverRuntimeRestartManager

Parameters:

id - The evaluator ID of the removed evaluator.

getPreviousEvaluators

public RestartEvaluators getPreviousEvaluators()

Used by DriverRestartManager. Gets the list of previous containers from the resource manager, compares that list to the YarnDriverRuntimeRestartManager's own list based on the evalutor preserver, and determine which evaluators are alive and which have failed during restart.

Specified by:

getPreviousEvaluators in interface DriverRuntimeRestartManager

Returns:

a map of Evaluator ID to EvaluatorRestartInfo for evaluators that have either failed or survived driver restart.

informAboutEvaluatorFailures

public void informAboutEvaluatorFailures(Set<String> evaluatorIds)

Calls the appropriate handler via REEFEventHandlers, which is a runtime specific implementation of the YARN runtime.

Specified by:

informAboutEvaluatorFailures in interface DriverRuntimeRestartManager

Parameters:

evaluatorIds - the set of evaluator IDs of failed evaluators during restart.