org.apache.storm.scheduler.resource.strategies.scheduling

Class BaseResourceAwareStrategy

java.lang.Object

org.apache.storm.scheduler.resource.strategies.scheduling.BaseResourceAwareStrategy

All Implemented Interfaces:

IStrategy

Direct Known Subclasses:

ConstraintSolverStrategy, DefaultResourceAwareStrategy, DefaultResourceAwareStrategyOld, GenericResourceAwareStrategy, GenericResourceAwareStrategyOld, RoundRobinResourceAwareStrategy

public abstract class BaseResourceAwareStrategy
extends Object
implements IStrategy

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BaseResourceAwareStrategy.NodeSortType Different node sorting types available.

Field Summary

Fields

Modifier and Type	Field and Description
protected Cluster	cluster
protected Map<String,Set<ExecutorDetails>>	compToExecs
protected Map<String,Object>	config
protected IExecSorter	execSorter
protected Map<ExecutorDetails,String>	execToComp
protected RasNodes	nodes
protected INodeSorter	nodeSorter
protected BaseResourceAwareStrategy.NodeSortType	nodeSortType
protected boolean	orderExecutorsByProximity
protected SchedulingSearcherState	searcherState
protected boolean	sortNodesForEachExecutor
protected TopologyDetails	topologyDetails
protected String	topoName

Constructor Summary

Constructors

Constructor and Description
BaseResourceAwareStrategy()
BaseResourceAwareStrategy(boolean sortNodesForEachExecutor, BaseResourceAwareStrategy.NodeSortType nodeSortType) Initialize for the default implementation of schedule().

Method Summary

Modifier and Type	Method and Description
protected int	assignBoundAckersForNewWorkerSlot(ExecutorDetails exec, RasNode node, WorkerSlot workerSlot) Determine how many bound ackers to put into the given workerSlot.
protected SchedulingResult	checkSchedulingFeasibility() Check scheduling feasibility for a quick failure as the second step in schedule(Cluster, TopologyDetails).
static int	getMaxStateSearchFromTopoConf(Map<String,Object> topoConf)
List<RasNode>	hostnameToNodes(String hostname) hostname to Ids.
RasNode	idToNode(String id) Find RASNode for specified node id.
protected boolean	isExecAssignmentToWorkerValid(ExecutorDetails exec, WorkerSlot worker) Check if the assignment of the executor to the worker is valid.
static boolean	isOrderByProximity(Map<String,Object> topoConf)
void	prepare(Map<String,Object> config) Prepare the Strategy for scheduling.
protected void	prepareForScheduling(Cluster cluster, TopologyDetails topologyDetails) Initialize instance variables as the first step in schedule(Cluster, TopologyDetails).
SchedulingResult	schedule(Cluster cluster, TopologyDetails td) Note that this method is not thread-safe.
protected SchedulingResult	scheduleExecutorsOnNodes(List<ExecutorDetails> orderedExecutors, Iterable<String> sortedNodesIter) Try to schedule till successful or till limits (backtrack count or time) have been exceeded.
protected void	setExecSorter(IExecSorter execSorter) Set the pluggable sorter for ExecutorDetails.
protected void	setNodeSorter(INodeSorter nodeSorter) Set the pluggable sorter for Nodes.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

sortNodesForEachExecutor

protected final boolean sortNodesForEachExecutor

nodeSortType

protected final BaseResourceAwareStrategy.NodeSortType nodeSortType

config

protected Map<String,Object> config

cluster

protected Cluster cluster

topologyDetails

protected TopologyDetails topologyDetails

nodes

protected RasNodes nodes

topoName

protected String topoName

compToExecs

protected Map<String,Set<ExecutorDetails>> compToExecs

execToComp

protected Map<ExecutorDetails,String> execToComp

orderExecutorsByProximity

protected boolean orderExecutorsByProximity

searcherState

protected SchedulingSearcherState searcherState

execSorter

protected IExecSorter execSorter

nodeSorter

protected INodeSorter nodeSorter

Constructor Detail

BaseResourceAwareStrategy

public BaseResourceAwareStrategy()

BaseResourceAwareStrategy

public BaseResourceAwareStrategy(boolean sortNodesForEachExecutor,
                                 BaseResourceAwareStrategy.NodeSortType nodeSortType)

Initialize for the default implementation of schedule().

Parameters:

sortNodesForEachExecutor - Sort nodes before scheduling each executor.

nodeSortType - type of sorting to be applied to object resource collection BaseResourceAwareStrategy.NodeSortType.

Method Detail

prepare

public void prepare(Map<String,Object> config)

Description copied from interface: IStrategy

Prepare the Strategy for scheduling.

Specified by:

prepare in interface IStrategy

Parameters:

config - the cluster configuration

schedule

public SchedulingResult schedule(Cluster cluster,
                                 TopologyDetails td)

Note that this method is not thread-safe. Several instance variables are generated from supplied parameters. In addition, the following instance variables are set to complete scheduling:

searcherState execSorter to sort executors nodeSorter to sort nodes

Scheduling consists of three main steps:

prepareForScheduling(Cluster, TopologyDetails) checkSchedulingFeasibility(), and scheduleExecutorsOnNodes(List, Iterable)

The executors and nodes are sorted in the order most conducive to scheduling for the strategy. Those interfaces may be overridden by subclasses using mutators:

setExecSorter(IExecSorter) and setNodeSorter(INodeSorter)

Specified by:

schedule in interface IStrategy

Parameters:

cluster - on which executors will be scheduled.

td - the topology to schedule for.

Returns:

result of scheduling (success, failure, or null when interrupted).

prepareForScheduling

protected void prepareForScheduling(Cluster cluster,
                                    TopologyDetails topologyDetails)

Initialize instance variables as the first step in schedule(Cluster, TopologyDetails). This method may be extended by subclasses to initialize additional variables as in ConstraintSolverStrategy.prepareForScheduling(Cluster, TopologyDetails).

Parameters:

cluster - on which executors will be scheduled.

topologyDetails - to be scheduled.

setExecSorter

protected void setExecSorter(IExecSorter execSorter)

Set the pluggable sorter for ExecutorDetails.

Parameters:

execSorter - to use for sorting executorDetails when scheduling.

setNodeSorter

protected void setNodeSorter(INodeSorter nodeSorter)

Set the pluggable sorter for Nodes.

Parameters:

nodeSorter - to use for sorting nodes when scheduling.

getMaxStateSearchFromTopoConf

public static int getMaxStateSearchFromTopoConf(Map<String,Object> topoConf)

isOrderByProximity

public static boolean isOrderByProximity(Map<String,Object> topoConf)

checkSchedulingFeasibility

protected SchedulingResult checkSchedulingFeasibility()

Check scheduling feasibility for a quick failure as the second step in schedule(Cluster, TopologyDetails). If scheduling is not possible, then return a SchedulingStatus object with a failure status. If fully scheduled then return a successful SchedulingStatus. This method can be extended by subclasses ConstraintSolverStrategy.checkSchedulingFeasibility() to check for additional failure conditions.

Returns:

A non-null SchedulingResult to terminate scheduling, otherwise return null to continue scheduling.

isExecAssignmentToWorkerValid

protected boolean isExecAssignmentToWorkerValid(ExecutorDetails exec,
                                                WorkerSlot worker)

Check if the assignment of the executor to the worker is valid. In simple cases, this is simply a check of RasNode.wouldFit(WorkerSlot, ExecutorDetails, TopologyDetails). This method may be extended by subclasses to add additional checks, see ConstraintSolverStrategy.isExecAssignmentToWorkerValid(ExecutorDetails, WorkerSlot).

Parameters:

exec - being scheduled.

worker - on which to schedule.

Returns:

true if executor can be assigned to the worker, false otherwise.

hostnameToNodes

public List<RasNode> hostnameToNodes(String hostname)

hostname to Ids.

Parameters:

hostname - the hostname.

Returns:

the ids n that node.

idToNode

public RasNode idToNode(String id)

Find RASNode for specified node id.

Parameters:

id - the node/supervisor id to lookup

Returns:

a RASNode object

scheduleExecutorsOnNodes

protected SchedulingResult scheduleExecutorsOnNodes(List<ExecutorDetails> orderedExecutors,
                                                    Iterable<String> sortedNodesIter)

Try to schedule till successful or till limits (backtrack count or time) have been exceeded.

Parameters:

orderedExecutors - Executors sorted in the preferred order cannot be null - note that ackers are isolated at the end.

sortedNodesIter - Node iterable which may be null.

Returns:

SchedulingResult with success attribute set to true or false indicting whether ALL executors were assigned.

assignBoundAckersForNewWorkerSlot

protected int assignBoundAckersForNewWorkerSlot(ExecutorDetails exec,
                                                RasNode node,
                                                WorkerSlot workerSlot)

Determine how many bound ackers to put into the given workerSlot. Then try to assign the ackers one by one into this workerSlot upto the calculated maximum required. Return the number of ackers assigned. Return 0 if one of the conditions hold true: 1. No bound ackers are used. 2. This is not first exec assigned to this worker. 3. No ackers could be assigned because of space or exception.

Parameters:

exec - being scheduled.

node - RasNode on which to schedule.

workerSlot - WorkerSlot on which to schedule.

Returns:

Number of ackers assigned.