Package org.apache.storm.scheduler

Interface ISchedulingState

All Known Implementing Classes:
Cluster, SingleTopologyCluster
public interface ISchedulingState
An interface that provides access to the current scheduling state. The scheduling state is not guaranteed to be thread safe.

  • Method Details

    • getTopologies

      Topologies getTopologies()
      Get all of the topologies.
      Returns:
      all of the topologies that are a part of the cluster.

    • needsSchedulingTopologies

      List<TopologyDetails> needsSchedulingTopologies()
      Get all of the topologies that need scheduling.
      Returns:
      all of the topologies that are not fully scheduled.

    • needsScheduling

      boolean needsScheduling(TopologyDetails topology)
      Does the topology need scheduling.

      A topology needs scheduling if one of the following conditions holds:

      • Although the topology is assigned slots, but is squeezed. i.e. the topology is assigned less slots than desired.
      • There are unassigned executors in this topology

    • needsSchedulingRas

      boolean needsSchedulingRas(TopologyDetails topology)
      Like needsScheduling(TopologyDetails) but does not take into account the number of workers requested. This is because the number of workers is ignored in RAS
      Parameters:
      topology - the topology to check
      Returns:
      true if the topology needs scheduling else false.

    • getBlacklistedHosts

      Set<String> getBlacklistedHosts()
      Get all of the hosts that are blacklisted.
      Returns:
      all of the hosts that are blacklisted

    • isBlackListed

      boolean isBlackListed(String supervisorId)
      Check is a given supervisor is on a blacklisted host.
      Parameters:
      supervisorId - the id of the supervisor
      Returns:
      true if it is else false

    • isBlacklistedHost

      boolean isBlacklistedHost(String host)
      Check if a given host is blacklisted.
      Parameters:
      host - the name of the host
      Returns:
      true if it is else false.

    • getHost

      String getHost(String supervisorId)
      Map a supervisor to a given host.
      Parameters:
      supervisorId - the id of the supervisor
      Returns:
      the actual host name the supervisor is on

    • getUnassignedExecutors

      Collection<ExecutorDetails> getUnassignedExecutors(TopologyDetails topology)
      get the unassigned executors of the topology.
      Parameters:
      topology - the topology to check
      Returns:
      the unassigned executors of the topology.

    • getNeedsSchedulingExecutorToComponents

      Map<ExecutorDetails,String> getNeedsSchedulingExecutorToComponents(TopologyDetails topology)
      Get the executor to component name map for executors that need to be scheduled.
      Parameters:
      topology - the topology this is for
      Returns:
      a executor -> component-id map which needs scheduling in this topology.

    • getNeedsSchedulingComponentToExecutors

      Map<String,List<ExecutorDetails>> getNeedsSchedulingComponentToExecutors(TopologyDetails topology)
      Get the component name to executor list for executors that need to be scheduled.
      Parameters:
      topology - the topology this is for
      Returns:
      a component-id -> executors map which needs scheduling in this topology.

    • getUsedPorts

      Set<Integer> getUsedPorts(SupervisorDetails supervisor)
      Get all the used ports of this supervisor.

    • getAvailablePorts

      Set<Integer> getAvailablePorts(SupervisorDetails supervisor)
      Return the available ports of this supervisor.

    • getAssignablePorts

      Set<Integer> getAssignablePorts(SupervisorDetails supervisor)
      Get the ports that are not blacklisted.
      Parameters:
      supervisor - the supervisor
      Returns:
      the ports that are not blacklisted

    • getAvailableSlots

      List<WorkerSlot> getAvailableSlots(SupervisorDetails supervisor)
      Return all the available slots on this supervisor.

    • getAvailableSlots

      List<WorkerSlot> getAvailableSlots()
      Get all the available worker slots in the cluster.

    • getNonBlacklistedAvailableSlots

      List<WorkerSlot> getNonBlacklistedAvailableSlots(List<String> blacklistedSupervisorIds)
      Get all the available worker slots in the cluster, that are not blacklisted.
      Parameters:
      blacklistedSupervisorIds - list of supervisor ids that should also be considered blacklisted.

    • getAssignableSlots

      List<WorkerSlot> getAssignableSlots(SupervisorDetails supervisor)
      Return all non-blacklisted slots on this supervisor.
      Parameters:
      supervisor - the supervisor
      Returns:
      the non-blacklisted slots

    • getAssignableSlots

      List<WorkerSlot> getAssignableSlots()
      Get all non-blacklisted slots in the cluster.

    • getUsedSlots

      Collection<WorkerSlot> getUsedSlots()
      Get all currently occupied slots.

    • isSlotOccupied

      boolean isSlotOccupied(WorkerSlot slot)
      Check if a slot is occupied or not.
      Parameters:
      slot - the slot be to checked.
      Returns:
      true if the specified slot is occupied.

    • getAssignedNumWorkers

      int getAssignedNumWorkers(TopologyDetails topology)
      Get the number of workers assigned to a topology.
      Parameters:
      topology - the topology this is for
      Returns:
      the number of workers assigned to this topology.

    • getAvailableResources

      NormalizedResourceOffer getAvailableResources(SupervisorDetails sd)
      Get the resources on the supervisor that are available to be scheduled.
      Parameters:
      sd - the supervisor.
      Returns:
      the resources available to be scheduled.

    • wouldFit

      boolean wouldFit(WorkerSlot ws, ExecutorDetails exec, TopologyDetails td, NormalizedResourceOffer resourcesAvailable, double maxHeap)
      Would scheduling exec on ws fit? With a heap <= maxHeap total memory added <= memoryAvailable and cpu added <= cpuAvailable.
      Parameters:
      ws - the slot to put it in
      exec - the executor to investigate
      td - the topology detains for this executor
      resourcesAvailable - all the available resources
      maxHeap - the maximum heap size for ws
      Returns:
      true it fits else false

    • getAssignmentById

      SchedulerAssignment getAssignmentById(String topologyId)
      get the current assignment for the topology.

    • getUsedSlotsByTopologyId

      Collection<WorkerSlot> getUsedSlotsByTopologyId(String topologyId)
      get slots used by a topology.

    • getSupervisorById

      SupervisorDetails getSupervisorById(String nodeId)
      Get a specific supervisor with the nodeId.

    • getSupervisorsByHost

      List<SupervisorDetails> getSupervisorsByHost(String host)
      Get all the supervisors on the specified host.
      Parameters:
      host - hostname of the supervisor
      Returns:
      the SupervisorDetails object.

    • getAssignments

      Map<String,SchedulerAssignment> getAssignments()
      Get all the assignments.

    • getSupervisors

      Map<String,SupervisorDetails> getSupervisors()
      Get all the supervisors.

    • getAllScheduledResourcesForNode

      NormalizedResourceRequest getAllScheduledResourcesForNode(String nodeId)
      Get all scheduled resources for node.

    • getNonBlacklistedClusterAvailableResources

      NormalizedResourceOffer getNonBlacklistedClusterAvailableResources(Collection<String> blacklistedSupervisorIds)
      Get the resources in the cluster that are available for scheduling.
      Parameters:
      blacklistedSupervisorIds - other ids that are tentatively blacklisted.

    • getClusterTotalCpuResource

      double getClusterTotalCpuResource()
      Get the total amount of CPU resources in cluster.

    • getClusterTotalMemoryResource

      double getClusterTotalMemoryResource()
      Get the total amount of memory resources in cluster.

    • getClusterTotalGenericResources

      Map<String,Double> getClusterTotalGenericResources()
      Get the total amount of generic resources (excluding CPU and memory) in cluster.

    • getNetworkTopography

      Map<String,List<String>> getNetworkTopography()
      Get the network topography (rackId -> nodes in the rack).

    • getHostToRack

      default Map<String,String> getHostToRack()
      Get host -> rack map - the inverse of networkTopography.

    • getStatusMap

      Map<String,String> getStatusMap()
      Get all topology scheduler statuses.

    • getTopologyResourcesMap

      Map<String,TopologyResources> getTopologyResourcesMap()
      Get the amount of resources used by topologies. Used for displaying resource information on the UI.
      Returns:
      a map that contains multiple topologies and the resources the topology requested and assigned. Key: topology id Value: an array that describes the resources the topology requested and assigned in the following format: {requestedMemOnHeap, requestedMemOffHeap, requestedCpu, assignedMemOnHeap, assignedMemOffHeap, assignedCpu}

    • getSupervisorsResourcesMap

      Map<String,SupervisorResources> getSupervisorsResourcesMap()
      Get the amount of used and free resources on a supervisor. Used for displaying resource information on the UI
      Returns:
      a map where the key is the supervisor id and the value is a map that represents resource usage for a supervisor in the following format: {totalMem, totalCpu, usedMem, usedCpu}

    • getWorkerResourcesMap

      Map<String,Map<WorkerSlot,WorkerResources>> getWorkerResourcesMap()
      Gets the reference to the full topology->worker resource map.
      Returns:
      map of topology -> map of worker slot ->resources for that worker

    • getWorkerResources

      WorkerResources getWorkerResources(WorkerSlot ws)
      Get the resources for a given slot.
      Parameters:
      ws - the slot
      Returns:
      the resources currently assigned

    • getScheduledMemoryForNode

      double getScheduledMemoryForNode(String nodeId)
      Get the total memory currently scheduled on a node.
      Parameters:
      nodeId - the id of the node
      Returns:
      the total memory currently scheduled on the node

    • getScheduledCpuForNode

      double getScheduledCpuForNode(String nodeId)
      Get the total cpu currently scheduled on a node.
      Parameters:
      nodeId - the id of the node
      Returns:
      the total cpu currently scheduled on the node

    • getConf

      Map<String,Object> getConf()
      Get the nimbus configuration.

    • getAssignedRacks

      default Set<String> getAssignedRacks(String... topologyIds)
      Determine the list of racks on which topologyIds have been assigned. Note that the returned set may contain DNSToSwitchMapping.DEFAULT_RACK if getHostToRack() is null or does not contain the assigned host.
      Parameters:
      topologyIds - for which assignments are examined.
      Returns:
      set of racks on which assignments have been made.