org.apache.storm.cluster

Interface IStormClusterState

All Known Implementing Classes:

StormClusterStateImpl

public interface IStormClusterState

Method Summary

Modifier and Type	Method and Description
void	activateStorm(String stormId, StormBase stormBase, Map<String,Object> topoConf)
List<String>	activeKeys()
List<String>	activeStorms()
void	addNimbusHost(String nimbusId, NimbusSummary nimbusSummary)
void	addPrivateWorkerKey(WorkerTokenServiceType type, String topologyId, long keyVersion, PrivateWorkerKey key) Store a new version of a private key.
default Map<String,SupervisorInfo>	allSupervisorInfo() Get all of the supervisors with the ID as the key.
default Map<String,SupervisorInfo>	allSupervisorInfo(Runnable callback) Get all supervisor info.
Assignment	assignmentInfo(String stormId, Runnable callback) Get the assignment based on storm id from local backend.
VersionedData<Assignment>	assignmentInfoWithVersion(String stormId, Runnable callback)
List<String>	assignments(Runnable callback)
Map<String,Assignment>	assignmentsInfo() Get all the topologies assignments mapping stormId -> Assignment from local backend.
Integer	assignmentVersion(String stormId, Runnable callback)
List<String>	backpressureTopologies() Deprecated.
List<String>	blobstore(Runnable callback)
List<String>	blobstoreInfo(String blobKey)
Credentials	credentials(String stormId, Runnable callback)
void	deleteTopologyProfileRequests(String stormId, ProfileRequest profileRequest)
void	disconnect()
List<ErrorInfo>	errors(String stormId, String componentId)
List<String>	errorTopologies()
Map<ExecutorInfo,ExecutorBeat>	executorBeats(String stormId, Map<List<Long>,NodeInfo> executorNodePort)
NimbusInfo	getLeader(Runnable callback) Get leader info from state store, which was written when a master gains leadership.
long	getNextPrivateWorkerKeyVersion(WorkerTokenServiceType type, String topologyId) Get the next key version number that should be used for this topology id.
PrivateWorkerKey	getPrivateWorkerKey(WorkerTokenServiceType type, String topologyId, long keyVersion) Get a private key used to validate a token is correct.
default Optional<String>	getTopoId(String topologyName) Get a topology ID from the name of a topology.
List<ProfileRequest>	getTopologyProfileRequests(String stormId)
ClusterWorkerHeartbeat	getWorkerHeartbeat(String stormId, String node, Long port)
List<ProfileRequest>	getWorkerProfileRequests(String stormId, NodeInfo nodeInfo)
List<String>	heartbeatStorms()
Set<String>	idsOfTopologiesWithPrivateWorkerKeys() Get a list of all topologyIds that currently have private worker keys stored, of any kind.
boolean	isAssignmentsBackendSynchronized() Flag to indicate if the assignments synced successfully, see syncRemoteAssignments(Map).
ErrorInfo	lastError(String stormId, String componentId)
List<NimbusSummary>	nimbuses()
Assignment	remoteAssignmentInfo(String stormId, Runnable callback) Get the assignment based on storm id from remote state store, eg: ZK.
void	removeAllPrivateWorkerKeys(String topologyId) Remove all of the worker keys for a given topology.
void	removeBackpressure(String stormId) Deprecated.
void	removeBlobstoreKey(String blobKey)
void	removeExpiredPrivateWorkerKeys(String topologyId) Remove all keys for the given topology that have expired.
void	removeKeyVersion(String blobKey)
void	removeStorm(String stormId)
void	removeStormBase(String stormId)
void	removeWorkerBackpressure(String stormId, String node, Long port) Deprecated.
void	removeWorkerHeartbeat(String stormId, String node, Long port)
void	reportError(String stormId, String componentId, String node, Long port, Throwable error)
void	setAssignment(String stormId, Assignment info, Map<String,Object> topoConf)
void	setAssignmentsBackendSynchronized() Mark the assignments as synced successfully, see isAssignmentsBackendSynchronized().
void	setCredentials(String stormId, Credentials creds, Map<String,Object> topoConf)
void	setTopologyLogConfig(String stormId, LogConfig logConfig, Map<String,Object> topoConf)
void	setupBackpressure(String stormId, Map<String,Object> topoConf) Deprecated.
void	setupBlob(String key, NimbusInfo nimbusInfo, Integer versionInfo)
void	setupErrors(String stormId, Map<String,Object> topoConf)
void	setupHeatbeats(String stormId, Map<String,Object> topoConf)
void	setWorkerProfileRequest(String stormId, ProfileRequest profileRequest)
StormBase	stormBase(String stormId, Runnable callback) Get a storm base for a topology.
String	stormId(String stormName) Get storm id from passed name, null if the name doesn’t exist on cluster.
void	supervisorHeartbeat(String supervisorId, SupervisorInfo info)
SupervisorInfo	supervisorInfo(String supervisorId)
List<String>	supervisors(Runnable callback)
void	syncRemoteAssignments(Map<String,byte[]> remote) Sync the remote state store assignments to local backend, used when master gains leadership, see LeaderListenerCallback.
void	syncRemoteIds(Map<String,String> ids) Sync all the active storm ids of the cluster, used now when master gains leadership.
void	teardownHeartbeats(String stormId)
void	teardownTopologyErrors(String stormId)
boolean	topologyBackpressure(String stormId, long timeoutMs, Runnable callback) Deprecated.
default Map<String,StormBase>	topologyBases()
LogConfig	topologyLogConfig(String stormId, Runnable cb)
void	updateStorm(String stormId, StormBase newElems)
void	workerHeartbeat(String stormId, String node, Long port, ClusterWorkerHeartbeat info)

Method Detail

assignments

List<String> assignments(Runnable callback)

assignmentInfo

Assignment assignmentInfo(String stormId,
                          Runnable callback)

Get the assignment based on storm id from local backend.

Parameters:

stormId - topology id

callback - callback function

Returns:

Assignment

remoteAssignmentInfo

Assignment remoteAssignmentInfo(String stormId,
                                Runnable callback)

Get the assignment based on storm id from remote state store, eg: ZK.

Parameters:

stormId - topology id

callback - callback function

Returns:

Assignment

assignmentsInfo

Map<String,Assignment> assignmentsInfo()

Get all the topologies assignments mapping stormId -> Assignment from local backend.

Returns:

stormId -> Assignment mapping

syncRemoteAssignments

void syncRemoteAssignments(Map<String,byte[]> remote)

Sync the remote state store assignments to local backend, used when master gains leadership, see LeaderListenerCallback.

Parameters:

remote - assigned assignments for a specific IStormClusterState instance, usually a supervisor/node.

isAssignmentsBackendSynchronized

boolean isAssignmentsBackendSynchronized()

Flag to indicate if the assignments synced successfully, see syncRemoteAssignments(Map).

Returns:

true if is synced successfully

setAssignmentsBackendSynchronized

void setAssignmentsBackendSynchronized()

Mark the assignments as synced successfully, see isAssignmentsBackendSynchronized().

assignmentInfoWithVersion

VersionedData<Assignment> assignmentInfoWithVersion(String stormId,
                                                    Runnable callback)

assignmentVersion

Integer assignmentVersion(String stormId,
                          Runnable callback)
                   throws Exception

Throws:

Exception

blobstoreInfo

List<String> blobstoreInfo(String blobKey)

nimbuses

List<NimbusSummary> nimbuses()

addNimbusHost

void addNimbusHost(String nimbusId,
                   NimbusSummary nimbusSummary)

activeStorms

List<String> activeStorms()

stormBase

StormBase stormBase(String stormId,
                    Runnable callback)

Get a storm base for a topology.

Parameters:

stormId - the id of the topology

callback - something to call if the data changes (best effort)

Returns:

the StormBase or null if it is not alive.

stormId

String stormId(String stormName)

Get storm id from passed name, null if the name doesn’t exist on cluster.

Parameters:

stormName - storm name

Returns:

storm id

syncRemoteIds

void syncRemoteIds(Map<String,String> ids)

Sync all the active storm ids of the cluster, used now when master gains leadership.

Parameters:

ids - stormName -> stormId mapping

getWorkerHeartbeat

ClusterWorkerHeartbeat getWorkerHeartbeat(String stormId,
                                          String node,
                                          Long port)

getWorkerProfileRequests

List<ProfileRequest> getWorkerProfileRequests(String stormId,
                                              NodeInfo nodeInfo)

getTopologyProfileRequests

List<ProfileRequest> getTopologyProfileRequests(String stormId)

setWorkerProfileRequest

void setWorkerProfileRequest(String stormId,
                             ProfileRequest profileRequest)

deleteTopologyProfileRequests

void deleteTopologyProfileRequests(String stormId,
                                   ProfileRequest profileRequest)

executorBeats

Map<ExecutorInfo,ExecutorBeat> executorBeats(String stormId,
                                             Map<List<Long>,NodeInfo> executorNodePort)

supervisors

List<String> supervisors(Runnable callback)

supervisorInfo

SupervisorInfo supervisorInfo(String supervisorId)

setupHeatbeats

void setupHeatbeats(String stormId,
                    Map<String,Object> topoConf)

teardownHeartbeats

void teardownHeartbeats(String stormId)

teardownTopologyErrors

void teardownTopologyErrors(String stormId)

heartbeatStorms

List<String> heartbeatStorms()

errorTopologies

List<String> errorTopologies()

backpressureTopologies

@Deprecated
List<String> backpressureTopologies()

Deprecated.

Get backpressure topologies.

getLeader

NimbusInfo getLeader(Runnable callback)

Get leader info from state store, which was written when a master gains leadership.

Caution: it can not be used for fencing and is only for informational purposes because we use ZK as our backend now, which could have a overdue info of nodes.

Parameters:

callback - callback func

Returns:

NimbusInfo

setTopologyLogConfig

void setTopologyLogConfig(String stormId,
                          LogConfig logConfig,
                          Map<String,Object> topoConf)

topologyLogConfig

LogConfig topologyLogConfig(String stormId,
                            Runnable cb)

workerHeartbeat

void workerHeartbeat(String stormId,
                     String node,
                     Long port,
                     ClusterWorkerHeartbeat info)

removeWorkerHeartbeat

void removeWorkerHeartbeat(String stormId,
                           String node,
                           Long port)

supervisorHeartbeat

void supervisorHeartbeat(String supervisorId,
                         SupervisorInfo info)

topologyBackpressure

@Deprecated
boolean topologyBackpressure(String stormId,
                                         long timeoutMs,
                                         Runnable callback)

Deprecated.

Get topoloy backpressure.

setupBackpressure

@Deprecated
void setupBackpressure(String stormId,
                                   Map<String,Object> topoConf)

Deprecated.

Setup backpressure.

removeBackpressure

@Deprecated
void removeBackpressure(String stormId)

Deprecated.

Remove backpressure.

removeWorkerBackpressure

@Deprecated
void removeWorkerBackpressure(String stormId,
                                          String node,
                                          Long port)

Deprecated.

Remove worker backpressure.

activateStorm

void activateStorm(String stormId,
                   StormBase stormBase,
                   Map<String,Object> topoConf)

updateStorm

void updateStorm(String stormId,
                 StormBase newElems)

removeStormBase

void removeStormBase(String stormId)

setAssignment

void setAssignment(String stormId,
                   Assignment info,
                   Map<String,Object> topoConf)

setupBlob

void setupBlob(String key,
               NimbusInfo nimbusInfo,
               Integer versionInfo)

activeKeys

List<String> activeKeys()

blobstore

List<String> blobstore(Runnable callback)

removeStorm

void removeStorm(String stormId)

removeBlobstoreKey

void removeBlobstoreKey(String blobKey)

removeKeyVersion

void removeKeyVersion(String blobKey)

reportError

void reportError(String stormId,
                 String componentId,
                 String node,
                 Long port,
                 Throwable error)

setupErrors

void setupErrors(String stormId,
                 Map<String,Object> topoConf)

errors

List<ErrorInfo> errors(String stormId,
                       String componentId)

lastError

ErrorInfo lastError(String stormId,
                    String componentId)

setCredentials

void setCredentials(String stormId,
                    Credentials creds,
                    Map<String,Object> topoConf)

credentials

Credentials credentials(String stormId,
                        Runnable callback)

disconnect

void disconnect()

getPrivateWorkerKey

PrivateWorkerKey getPrivateWorkerKey(WorkerTokenServiceType type,
                                     String topologyId,
                                     long keyVersion)

Get a private key used to validate a token is correct. This is expected to be called from a privileged daemon, and the ACLs should be set up to only allow nimbus and these privileged daemons access to these private keys.

Parameters:

type - the type of service the key is for.

topologyId - the topology id the key is for.

keyVersion - the version of the key this is for.

Returns:

the private key or null if it could not be found.

addPrivateWorkerKey

void addPrivateWorkerKey(WorkerTokenServiceType type,
                         String topologyId,
                         long keyVersion,
                         PrivateWorkerKey key)

Store a new version of a private key. This is expected to only ever be called from nimbus. All ACLs however need to be setup to allow the given services access to the stored information.

Parameters:

type - the type of service this key is for.

topologyId - the topology this key is for

keyVersion - the version of the key this is for.

key - the key to store.

getNextPrivateWorkerKeyVersion

long getNextPrivateWorkerKeyVersion(WorkerTokenServiceType type,
                                    String topologyId)

Get the next key version number that should be used for this topology id. This is expected to only ever be called from nimbus, but it is acceptable if the ACLs are setup so that it can work from a privileged daemon for the given service.

Parameters:

type - the type of service this is for.

topologyId - the topology id this is for.

Returns:

the next version number. It should be 0 for a new topology id/service combination.

removeExpiredPrivateWorkerKeys

void removeExpiredPrivateWorkerKeys(String topologyId)

Remove all keys for the given topology that have expired. The number of keys should be small enough that doing an exhaustive scan of them all is acceptable as there is no guarantee that expiration time and version number are related. This should be for all service types. This is expected to only ever be called from nimbus and some ACLs may be setup so being called from other daemons will cause it to fail.

Parameters:

topologyId - the id of the topology to scan.

removeAllPrivateWorkerKeys

void removeAllPrivateWorkerKeys(String topologyId)

Remove all of the worker keys for a given topology. Used to clean up after a topology finishes. This is expected to only ever be called from nimbus and ideally should only ever work from nimbus.

Parameters:

topologyId - the topology to clean up after.

idsOfTopologiesWithPrivateWorkerKeys

Set<String> idsOfTopologiesWithPrivateWorkerKeys()

Get a list of all topologyIds that currently have private worker keys stored, of any kind. This is expected to only ever be called from nimbus.

Returns:

the list of topology ids with any kind of private worker key stored.

allSupervisorInfo

default Map<String,SupervisorInfo> allSupervisorInfo()

Get all of the supervisors with the ID as the key.

allSupervisorInfo

default Map<String,SupervisorInfo> allSupervisorInfo(Runnable callback)

Get all supervisor info.

Parameters:

callback - be alerted if the list of supervisors change

Returns:

All of the supervisors with the ID as the key

getTopoId

default Optional<String> getTopoId(String topologyName)

Get a topology ID from the name of a topology.

Parameters:

topologyName - the name of the topology to look for

Returns:

the id of the topology or null if it is not alive.

topologyBases

default Map<String,StormBase> topologyBases()