org.apache.storm.scheduler.resource.strategies.scheduling

Class GenericResourceAwareStrategy

java.lang.Object

org.apache.storm.scheduler.resource.strategies.scheduling.BaseResourceAwareStrategy

org.apache.storm.scheduler.resource.strategies.scheduling.GenericResourceAwareStrategy

All Implemented Interfaces:

IStrategy

public class GenericResourceAwareStrategy
extends BaseResourceAwareStrategy
implements IStrategy

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.storm.scheduler.resource.strategies.scheduling.BaseResourceAwareStrategy

BaseResourceAwareStrategy.AllResources, BaseResourceAwareStrategy.ExistingScheduleFunc, BaseResourceAwareStrategy.ObjectResources

Field Summary

Fields inherited from class org.apache.storm.scheduler.resource.strategies.scheduling.BaseResourceAwareStrategy

cluster, nodes

Constructor Summary

Constructors

Constructor and Description
GenericResourceAwareStrategy()

Method Summary

Modifier and Type	Method and Description
SchedulingResult	schedule(Cluster cluster, TopologyDetails td) This method is invoked to calculate a scheduling for topology td.
protected TreeSet<BaseResourceAwareStrategy.ObjectResources>	sortObjectResources(BaseResourceAwareStrategy.AllResources allResources, ExecutorDetails exec, TopologyDetails topologyDetails, BaseResourceAwareStrategy.ExistingScheduleFunc existingScheduleFunc) Sort objects by the following two criteria.
static TreeSet<BaseResourceAwareStrategy.ObjectResources>	sortObjectResourcesImpl(BaseResourceAwareStrategy.AllResources allResources, ExecutorDetails exec, TopologyDetails topologyDetails, BaseResourceAwareStrategy.ExistingScheduleFunc existingScheduleFunc) Implementation of the sortObjectResources method so other strategies can reuse it.

Methods inherited from class org.apache.storm.scheduler.resource.strategies.scheduling.BaseResourceAwareStrategy

findWorkerForExec, getSpouts, hostnameToNodes, idToNode, makeHostToNodeIds, mkNotEnoughResources, nodeToRack, orderExecutors, prepare, scheduleExecutor, sortAllNodes, sortNodes

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.storm.scheduler.resource.strategies.scheduling.IStrategy

prepare

Constructor Detail

GenericResourceAwareStrategy

public GenericResourceAwareStrategy()

Method Detail

sortObjectResourcesImpl

public static TreeSet<BaseResourceAwareStrategy.ObjectResources> sortObjectResourcesImpl(BaseResourceAwareStrategy.AllResources allResources,
                                                                                         ExecutorDetails exec,
                                                                                         TopologyDetails topologyDetails,
                                                                                         BaseResourceAwareStrategy.ExistingScheduleFunc existingScheduleFunc)

Implementation of the sortObjectResources method so other strategies can reuse it.

schedule

public SchedulingResult schedule(Cluster cluster,
                                 TopologyDetails td)

Description copied from interface: IStrategy

This method is invoked to calculate a scheduling for topology td. Cluster will reject any changes that are not for the given topology. Any changes made to the cluster will be committed if the scheduling is successful.

NOTE: scheduling occurs as a runnable in an interruptible thread. Scheduling should consider being interrupted if long running.

Specified by:

schedule in interface IStrategy

Parameters:

cluster - the current state of the cluster

td - the topology to schedule for

Returns:

returns a SchedulingResult object containing SchedulingStatus object to indicate whether scheduling is successful.

sortObjectResources

protected TreeSet<BaseResourceAwareStrategy.ObjectResources> sortObjectResources(BaseResourceAwareStrategy.AllResources allResources,
                                                                                 ExecutorDetails exec,
                                                                                 TopologyDetails topologyDetails,
                                                                                 BaseResourceAwareStrategy.ExistingScheduleFunc existingScheduleFunc)

Sort objects by the following two criteria. 1) the number executors of the topology that needs to be scheduled is already on the object (node or rack) in descending order. The reasoning to sort based on criterion 1 is so we schedule the rest of a topology on the same object (node or rack) as the existing executors of the topology. 2) the subordinate/subservient resource availability percentage of a rack in descending order We calculate the resource availability percentage by dividing the resource availability of the object (node or rack) by the resource availability of the entire rack or cluster depending on if object references a node or a rack. How this differs from the DefaultResourceAwareStrategy is that the percentage boosts the node or rack if it is requested by the executor that the sorting is being done for and pulls it down if it is not. By doing this calculation, objects (node or rack) that have exhausted or little of one of the resources mentioned above will be ranked after racks that have more balanced resource availability and nodes or racks that have resources that are not requested will be ranked below . So we will be less likely to pick a rack that have a lot of one resource but a low amount of another and have a lot of resources that are not requested by the executor.

Specified by:

sortObjectResources in class BaseResourceAwareStrategy

Parameters:

allResources - contains all individual ObjectResources as well as cumulative stats

exec - executor for which the sorting is done

topologyDetails - topologyDetails for the above executor

existingScheduleFunc - a function to get existing executors already scheduled on this object

Returns:

a sorted list of ObjectResources