org.apache.hadoop.yarn.client.api.async.impl

Class AMRMClientAsyncImpl<T extends AMRMClient.ContainerRequest>

java.lang.Object

org.apache.hadoop.service.AbstractService

org.apache.hadoop.yarn.client.api.async.AMRMClientAsync<T>

org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl<T>

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.service.Service

@InterfaceAudience.Private
@InterfaceStability.Unstable
public class AMRMClientAsyncImpl<T extends AMRMClient.ContainerRequest>
extends AMRMClientAsync<T>

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.yarn.client.api.async.AMRMClientAsync

AMRMClientAsync.AbstractCallbackHandler, AMRMClientAsync.CallbackHandler

Nested classes/interfaces inherited from interface org.apache.hadoop.service.Service

org.apache.hadoop.service.Service.STATE

Field Summary

Fields inherited from class org.apache.hadoop.yarn.client.api.async.AMRMClientAsync

client, handler, heartbeatIntervalMs

Constructor Summary

Constructors

Constructor and Description
AMRMClientAsyncImpl(AMRMClient<T> client, int intervalMs, AMRMClientAsync.AbstractCallbackHandler callbackHandler)
AMRMClientAsyncImpl(AMRMClient<T> client, int intervalMs, AMRMClientAsync.CallbackHandler callbackHandler) Deprecated.
AMRMClientAsyncImpl(int intervalMs, AMRMClientAsync.AbstractCallbackHandler callbackHandler)
AMRMClientAsyncImpl(int intervalMs, AMRMClientAsync.CallbackHandler callbackHandler) Deprecated. Use AMRMClientAsyncImpl(int, AMRMClientAsync.AbstractCallbackHandler) instead.

Method Summary

Methods

Modifier and Type	Method and Description
void	addContainerRequest(T req) Request containers for resources before calling allocate
org.apache.hadoop.yarn.api.records.Resource	getAvailableResources() Get the currently available resources in the cluster.
int	getClusterNodeCount() Get the current number of nodes in the cluster.
List<? extends Collection<T>>	getMatchingRequests(org.apache.hadoop.yarn.api.records.Priority priority, String resourceName, org.apache.hadoop.yarn.api.records.Resource capability)
org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse	registerApplicationMaster(String appHostName, int appHostPort, String appTrackingUrl) Registers this application master with the resource manager.
void	releaseAssignedContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId) Release containers assigned by the Resource Manager.
void	removeContainerRequest(T req) Remove previous container request.
void	requestContainerUpdate(org.apache.hadoop.yarn.api.records.Container container, org.apache.hadoop.yarn.api.records.UpdateContainerRequest updateContainerRequest) Request a container update before calling allocate.
protected void	serviceInit(org.apache.hadoop.conf.Configuration conf)
protected void	serviceStart()
protected void	serviceStop() Tells the heartbeat and handler threads to stop and waits for them to terminate.
void	unregisterApplicationMaster(org.apache.hadoop.yarn.api.records.FinalApplicationStatus appStatus, String appMessage, String appTrackingUrl) Unregister the application master.
void	updateBlacklist(List<String> blacklistAdditions, List<String> blacklistRemovals) Update application's blacklist with addition or removal resources.

Methods inherited from class org.apache.hadoop.yarn.client.api.async.AMRMClientAsync

createAMRMClientAsync, createAMRMClientAsync, createAMRMClientAsync, createAMRMClientAsync, getMatchingRequests, getMatchingRequests, getRegisteredTimelineV2Client, registerTimelineV2Client, requestContainerResourceChange, setHeartbeatInterval, waitFor, waitFor, waitFor

Methods inherited from class org.apache.hadoop.service.AbstractService

close, getBlockers, getConfig, getFailureCause, getFailureState, getLifecycleHistory, getName, getServiceState, getStartTime, init, isInState, noteFailure, putBlocker, registerGlobalListener, registerServiceListener, removeBlocker, setConfig, start, stop, toString, unregisterGlobalListener, unregisterServiceListener, waitForServiceToStop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

AMRMClientAsyncImpl

public AMRMClientAsyncImpl(int intervalMs,
                   AMRMClientAsync.AbstractCallbackHandler callbackHandler)

Parameters:

intervalMs - heartbeat interval in milliseconds between AM and RM

callbackHandler - callback handler that processes responses from the ResourceManager

AMRMClientAsyncImpl

public AMRMClientAsyncImpl(AMRMClient<T> client,
                   int intervalMs,
                   AMRMClientAsync.AbstractCallbackHandler callbackHandler)

AMRMClientAsyncImpl

@Deprecated
public AMRMClientAsyncImpl(int intervalMs,
                              AMRMClientAsync.CallbackHandler callbackHandler)

Deprecated. Use AMRMClientAsyncImpl(int, AMRMClientAsync.AbstractCallbackHandler) instead.

AMRMClientAsyncImpl

@InterfaceAudience.Private
@Deprecated
public AMRMClientAsyncImpl(AMRMClient<T> client,
                                                        int intervalMs,
                                                        AMRMClientAsync.CallbackHandler callbackHandler)

Deprecated.

Method Detail

serviceInit

protected void serviceInit(org.apache.hadoop.conf.Configuration conf)
                    throws Exception

Overrides:

serviceInit in class org.apache.hadoop.service.AbstractService

Throws:

Exception

serviceStart

protected void serviceStart()
                     throws Exception

Overrides:

serviceStart in class org.apache.hadoop.service.AbstractService

Throws:

Exception

serviceStop

protected void serviceStop()
                    throws Exception

Tells the heartbeat and handler threads to stop and waits for them to terminate.

Overrides:

serviceStop in class org.apache.hadoop.service.AbstractService

Throws:

Exception

getMatchingRequests

public List<? extends Collection<T>> getMatchingRequests(org.apache.hadoop.yarn.api.records.Priority priority,
                                                String resourceName,
                                                org.apache.hadoop.yarn.api.records.Resource capability)

Specified by:

getMatchingRequests in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

registerApplicationMaster

public org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse registerApplicationMaster(String appHostName,
                                                                                                     int appHostPort,
                                                                                                     String appTrackingUrl)
                                                                                                       throws org.apache.hadoop.yarn.exceptions.YarnException,
                                                                                                              IOException

Registers this application master with the resource manager. On successful registration, starts the heartbeating thread.

Specified by:

registerApplicationMaster in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

unregisterApplicationMaster

public void unregisterApplicationMaster(org.apache.hadoop.yarn.api.records.FinalApplicationStatus appStatus,
                               String appMessage,
                               String appTrackingUrl)
                                 throws org.apache.hadoop.yarn.exceptions.YarnException,
                                        IOException

Unregister the application master. This must be called in the end.

Specified by:

unregisterApplicationMaster in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Parameters:

appStatus - Success/Failure status of the master

appMessage - Diagnostics message on failure

appTrackingUrl - New URL to get master info

Throws:

org.apache.hadoop.yarn.exceptions.YarnException

IOException

addContainerRequest

public void addContainerRequest(T req)

Request containers for resources before calling allocate

Specified by:

addContainerRequest in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Parameters:

req - Resource request

removeContainerRequest

public void removeContainerRequest(T req)

Remove previous container request. The previous container request may have already been sent to the ResourceManager. So even after the remove request the app must be prepared to receive an allocation for the previous request even after the remove request

Specified by:

removeContainerRequest in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Parameters:

req - Resource request

requestContainerUpdate

public void requestContainerUpdate(org.apache.hadoop.yarn.api.records.Container container,
                          org.apache.hadoop.yarn.api.records.UpdateContainerRequest updateContainerRequest)

Description copied from class: AMRMClientAsync

Request a container update before calling allocate. Any previous pending update request of the same container will be removed.

Specified by:

requestContainerUpdate in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Parameters:

container - The container returned from the last successful resource allocation or update

updateContainerRequest - The UpdateContainerRequest.

releaseAssignedContainer

public void releaseAssignedContainer(org.apache.hadoop.yarn.api.records.ContainerId containerId)

Release containers assigned by the Resource Manager. If the app cannot use the container or wants to give up the container then it can release them. The app needs to make new requests for the released resource capability if it still needs it. eg. it released non-local resources

Specified by:

releaseAssignedContainer in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Parameters:

containerId -

getAvailableResources

public org.apache.hadoop.yarn.api.records.Resource getAvailableResources()

Get the currently available resources in the cluster. A valid value is available after a call to allocate has been made

Specified by:

getAvailableResources in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Returns:

Currently available resources

getClusterNodeCount

public int getClusterNodeCount()

Get the current number of nodes in the cluster. A valid values is available after a call to allocate has been made

Specified by:

getClusterNodeCount in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Returns:

Current number of nodes in the cluster

updateBlacklist

public void updateBlacklist(List<String> blacklistAdditions,
                   List<String> blacklistRemovals)

Update application's blacklist with addition or removal resources.

Specified by:

updateBlacklist in class AMRMClientAsync<T extends AMRMClient.ContainerRequest>

Parameters:

blacklistAdditions - list of resources which should be added to the application blacklist

blacklistRemovals - list of resources which should be removed from the application blacklist