@InterfaceAudience.Private @InterfaceStability.Unstable public class DefaultResourceCalculator extends ResourceCalculator
Constructor and Description |
---|
DefaultResourceCalculator() |
Modifier and Type | Method and Description |
---|---|
int |
compare(org.apache.hadoop.yarn.api.records.Resource unused,
org.apache.hadoop.yarn.api.records.Resource lhs,
org.apache.hadoop.yarn.api.records.Resource rhs,
boolean singleType)
On a cluster with capacity
clusterResource , compare lhs
and rhs . |
long |
computeAvailableContainers(org.apache.hadoop.yarn.api.records.Resource available,
org.apache.hadoop.yarn.api.records.Resource required)
Compute the number of containers which can be allocated given
available and required resources. |
float |
divide(org.apache.hadoop.yarn.api.records.Resource unused,
org.apache.hadoop.yarn.api.records.Resource numerator,
org.apache.hadoop.yarn.api.records.Resource denominator)
Divide resource
numerator by resource denominator
using specified policy (domination, average, fairness etc.); hence overall
clusterResource is provided for context. |
org.apache.hadoop.yarn.api.records.Resource |
divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator,
float denominator)
Divide-and-ceil
numerator by denominator . |
org.apache.hadoop.yarn.api.records.Resource |
divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator,
int denominator)
Divide-and-ceil
numerator by denominator . |
boolean |
fitsIn(org.apache.hadoop.yarn.api.records.Resource cluster,
org.apache.hadoop.yarn.api.records.Resource smaller,
org.apache.hadoop.yarn.api.records.Resource bigger)
Check if a smaller resource can be contained by bigger resource.
|
boolean |
isAnyMajorResourceZero(org.apache.hadoop.yarn.api.records.Resource resource)
Check if resource has any major resource types (which are all NodeManagers
included) a zero value.
|
boolean |
isInvalidDivisor(org.apache.hadoop.yarn.api.records.Resource r)
Determine if a resource is not suitable for use as a divisor
(will result in divide by 0, etc)
|
org.apache.hadoop.yarn.api.records.Resource |
multiplyAndNormalizeDown(org.apache.hadoop.yarn.api.records.Resource r,
double by,
org.apache.hadoop.yarn.api.records.Resource stepFactor)
Multiply resource
r by factor by
and normalize down using step-factor stepFactor . |
org.apache.hadoop.yarn.api.records.Resource |
multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r,
double by,
org.apache.hadoop.yarn.api.records.Resource stepFactor)
Multiply resource
r by factor by
and normalize up using step-factor stepFactor . |
org.apache.hadoop.yarn.api.records.Resource |
normalize(org.apache.hadoop.yarn.api.records.Resource r,
org.apache.hadoop.yarn.api.records.Resource minimumResource,
org.apache.hadoop.yarn.api.records.Resource maximumResource,
org.apache.hadoop.yarn.api.records.Resource stepFactor)
Normalize resource
r given the base
minimumResource and verify against max allowed
maximumResource using a step factor for the normalization. |
float |
ratio(org.apache.hadoop.yarn.api.records.Resource a,
org.apache.hadoop.yarn.api.records.Resource b)
Ratio of resource
a to resource b . |
org.apache.hadoop.yarn.api.records.Resource |
roundDown(org.apache.hadoop.yarn.api.records.Resource r,
org.apache.hadoop.yarn.api.records.Resource stepFactor)
Round-down resource
r given factor stepFactor . |
org.apache.hadoop.yarn.api.records.Resource |
roundUp(org.apache.hadoop.yarn.api.records.Resource r,
org.apache.hadoop.yarn.api.records.Resource stepFactor)
Round-up resource
r given factor stepFactor . |
compare, divideAndCeil, divideAndCeil, divideAndCeil, divideAndCeil, roundDown, roundDown, roundUp, roundUp
public int compare(org.apache.hadoop.yarn.api.records.Resource unused, org.apache.hadoop.yarn.api.records.Resource lhs, org.apache.hadoop.yarn.api.records.Resource rhs, boolean singleType)
ResourceCalculator
clusterResource
, compare lhs
and rhs
. Consider all resources unless singleType
is set
to true. When singleType
is set to true, consider only one
resource as per the ResourceCalculator
implementation; the
DefaultResourceCalculator
considers memory and
DominantResourceCalculator
considers the dominant resource.compare
in class ResourceCalculator
unused
- cluster capacitylhs
- First Resource
to comparerhs
- Second Resource
to comparesingleType
- Whether to consider a single resource type or all
resource typeslhs
is smaller, 0 if equal and 1 if it is largerpublic long computeAvailableContainers(org.apache.hadoop.yarn.api.records.Resource available, org.apache.hadoop.yarn.api.records.Resource required)
ResourceCalculator
available
and required
resources.computeAvailableContainers
in class ResourceCalculator
available
- available resourcesrequired
- required resourcespublic float divide(org.apache.hadoop.yarn.api.records.Resource unused, org.apache.hadoop.yarn.api.records.Resource numerator, org.apache.hadoop.yarn.api.records.Resource denominator)
ResourceCalculator
numerator
by resource denominator
using specified policy (domination, average, fairness etc.); hence overall
clusterResource
is provided for context.divide
in class ResourceCalculator
unused
- cluster resourcesnumerator
- numeratordenominator
- denominatornumerator
/denominator
using specific policypublic boolean isInvalidDivisor(org.apache.hadoop.yarn.api.records.Resource r)
ResourceCalculator
isInvalidDivisor
in class ResourceCalculator
r
- resourcepublic float ratio(org.apache.hadoop.yarn.api.records.Resource a, org.apache.hadoop.yarn.api.records.Resource b)
ResourceCalculator
a
to resource b
.ratio
in class ResourceCalculator
a
- resourceb
- resourcea
to resource b
public org.apache.hadoop.yarn.api.records.Resource divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator, int denominator)
ResourceCalculator
numerator
by denominator
.divideAndCeil
in class ResourceCalculator
numerator
- numerator resourcedenominator
- denominatorpublic org.apache.hadoop.yarn.api.records.Resource divideAndCeil(org.apache.hadoop.yarn.api.records.Resource numerator, float denominator)
ResourceCalculator
numerator
by denominator
.divideAndCeil
in class ResourceCalculator
numerator
- numerator resourcedenominator
- denominatorpublic org.apache.hadoop.yarn.api.records.Resource normalize(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource minimumResource, org.apache.hadoop.yarn.api.records.Resource maximumResource, org.apache.hadoop.yarn.api.records.Resource stepFactor)
ResourceCalculator
r
given the base
minimumResource
and verify against max allowed
maximumResource
using a step factor for the normalization.normalize
in class ResourceCalculator
r
- resourceminimumResource
- minimum valuemaximumResource
- the upper bound of the resource to be allocatedstepFactor
- the increment for resources to be allocatedpublic org.apache.hadoop.yarn.api.records.Resource roundUp(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource stepFactor)
ResourceCalculator
r
given factor stepFactor
.roundUp
in class ResourceCalculator
r
- resourcestepFactor
- step-factorpublic org.apache.hadoop.yarn.api.records.Resource roundDown(org.apache.hadoop.yarn.api.records.Resource r, org.apache.hadoop.yarn.api.records.Resource stepFactor)
ResourceCalculator
r
given factor stepFactor
.roundDown
in class ResourceCalculator
r
- resourcestepFactor
- step-factorpublic org.apache.hadoop.yarn.api.records.Resource multiplyAndNormalizeUp(org.apache.hadoop.yarn.api.records.Resource r, double by, org.apache.hadoop.yarn.api.records.Resource stepFactor)
ResourceCalculator
r
by factor by
and normalize up using step-factor stepFactor
.multiplyAndNormalizeUp
in class ResourceCalculator
r
- resource to be multipliedby
- multiplierstepFactor
- factor by which to normalize uppublic org.apache.hadoop.yarn.api.records.Resource multiplyAndNormalizeDown(org.apache.hadoop.yarn.api.records.Resource r, double by, org.apache.hadoop.yarn.api.records.Resource stepFactor)
ResourceCalculator
r
by factor by
and normalize down using step-factor stepFactor
.multiplyAndNormalizeDown
in class ResourceCalculator
r
- resource to be multipliedby
- multiplierstepFactor
- factor by which to normalize downpublic boolean fitsIn(org.apache.hadoop.yarn.api.records.Resource cluster, org.apache.hadoop.yarn.api.records.Resource smaller, org.apache.hadoop.yarn.api.records.Resource bigger)
ResourceCalculator
fitsIn
in class ResourceCalculator
public boolean isAnyMajorResourceZero(org.apache.hadoop.yarn.api.records.Resource resource)
ResourceCalculator
isAnyMajorResourceZero
in class ResourceCalculator
resource
- resourceCopyright © 2017 Apache Software Foundation. All Rights Reserved.