Package org.apache.storm.daemon.nimbus
Class Nimbus
java.lang.Object
org.apache.storm.daemon.nimbus.Nimbus
- All Implemented Interfaces:
DaemonCommon
,Shutdownable
,Nimbus.Iface
-
Nested Class Summary
-
Field Summary
Modifier and TypeFieldDescriptionstatic final SimpleVersion
static final Subject
static final List<org.apache.storm.shade.org.apache.zookeeper.data.ACL>
-
Constructor Summary
ConstructorDescriptionNimbus
(Map<String, Object> conf, INimbus inimbus, IStormClusterState stormClusterState, NimbusInfo hostPortInfo, BlobStore blobStore, TopoCache topoCache, ILeaderElector leaderElector, IGroupMappingServiceProvider groupMapper, StormMetricsRegistry metricsRegistry) Nimbus
(Map<String, Object> conf, INimbus inimbus, IStormClusterState stormClusterState, NimbusInfo hostPortInfo, BlobStore blobStore, ILeaderElector leaderElector, IGroupMappingServiceProvider groupMapper, StormMetricsRegistry metricsRegistry) -
Method Summary
Modifier and TypeMethodDescriptionvoid
void
addSupervisor
(Supervisor supervisor) Used for local cluster.boolean
awaitLeadership
(long timeout, TimeUnit timeUnit) beginBlobDownload
(String key) beginCreateBlob
(String key, SettableBlobMeta meta) beginUpdateBlob
(String key) void
cancelBlobUpload
(String session) void
void
checkAuthorization
(String topoName, Map<String, Object> topoConf, String operation, ReqContext context) static void
cleanInbox
(String dirLoc, int seconds) Deletes jar files in dirLoc older than seconds.void
void
deactivate
(String topoName) void
Enable/disable logging the tuples generated in topology via an internal EventLogger bolt.void
deleteBlob
(String key) void
Cleanup topologies and Jars.downloadBlobChunk
(String session) downloadChunk
(String id) filterAuthorized
(String operation, Collection<String> topoIds) void
finishBlobUpload
(String session) void
finishFileUpload
(String location) void
forceDeleteTopoDistDir
(String topoId) getBlobMeta
(String key) int
getBlobReplication
(String key) getComponentPageInfo
(String topoId, String componentId, String window, boolean includeSys) getComponentPendingProfileActions
(String id, String componentId, ProfileAction action) getLogConfig
(String topoId) static List<org.apache.storm.shade.org.apache.zookeeper.data.ACL>
getNimbusAcls
(Map<String, Object> conf) getOwnerResourceSummaries
(String owner) getSupervisorAssignments
(String nodeId) Get assigned assignments for a specific supervisorgetSupervisorPageInfo
(String superId, String host, boolean includeSys) getTopology
(String id) Returns the compiled topology that contains ackers and metrics consumsers.getTopologyHistory
(String user) getTopologyInfoByName
(String name) getTopologyInfoByNameWithOpts
(String name, GetInfoOptions options) getTopologyInfoWithOpts
(String topoId, GetInfoOptions options) getTopologyPageInfo
(String topoId, String window, boolean includeSys) Returns the user specified topology as submitted originally.static int
getVersionForKey
(String key, NimbusInfo nimbusInfo, org.apache.storm.shade.org.apache.curator.framework.CuratorFramework zkClient) boolean
isRemoteBlobExists
(String blobKey) Decide if the blob is removed from cluster.boolean
isTopologyNameAllowed
(String name) boolean
void
killTopology
(String name) void
killTopologyWithOpts
(String topoName, KillOptions options) static Nimbus
void
static void
void
processWorkerMetrics
(WorkerMetrics metrics) void
rebalance
(String topoName, RebalanceOptions options) void
rmDependencyJarsInTopology
(String topoId) void
rmTopologyKeys
(String topoId) void
Send supervisor local worker heartbeat when a supervisor is unreachablevoid
Send supervisor worker heartbeats for a specific supervisorvoid
setAuthorizationHandler
(IAuthorizer authorizationHandler) void
setBlobMeta
(String key, SettableBlobMeta meta) void
setLogConfig
(String topoId, LogConfig config) static void
setUpAckerExecutorConfigs
(String topoName, Map<String, Object> totalConfToSave, Map<String, Object> totalConf, int estimatedNumWorker) void
setWorkerProfiler
(String topoId, ProfileRequest profileRequest) void
shutdown()
void
submitTopology
(String name, String uploadedJarLocation, String jsonConf, StormTopology topology) void
submitTopologyWithOpts
(String topoName, String uploadedJarLocation, String jsonConf, StormTopology topology, SubmitOptions options) topoIdsToClean
(IStormClusterState state, BlobStore store, Map<String, Object> conf) topologiesOnSupervisor
(Map<String, Assignment> assignments, String supervisorId) int
updateBlobReplication
(String key, int replication) void
uploadBlobChunk
(String session, ByteBuffer chunk) void
uploadChunk
(String location, ByteBuffer chunk) void
uploadNewCredentials
(String topoName, Credentials credentials)
-
Field Details
-
ZK_ACLS
-
MIN_VERSION_SUPPORT_RPC_HEARTBEAT
-
NIMBUS_SUBJECT
-
-
Constructor Details
-
Nimbus
public Nimbus(Map<String, Object> conf, INimbus inimbus, StormMetricsRegistry metricsRegistry) throws Exception- Throws:
Exception
-
Nimbus
public Nimbus(Map<String, Object> conf, INimbus inimbus, IStormClusterState stormClusterState, NimbusInfo hostPortInfo, BlobStore blobStore, ILeaderElector leaderElector, IGroupMappingServiceProvider groupMapper, StormMetricsRegistry metricsRegistry) throws Exception- Throws:
Exception
-
Nimbus
public Nimbus(Map<String, Object> conf, INimbus inimbus, IStormClusterState stormClusterState, NimbusInfo hostPortInfo, BlobStore blobStore, TopoCache topoCache, ILeaderElector leaderElector, IGroupMappingServiceProvider groupMapper, StormMetricsRegistry metricsRegistry) throws Exception- Throws:
Exception
-
-
Method Details
-
getNimbusAcls
-
getVersionForKey
public static int getVersionForKey(String key, NimbusInfo nimbusInfo, org.apache.storm.shade.org.apache.curator.framework.CuratorFramework zkClient) throws KeyNotFoundException - Throws:
KeyNotFoundException
-
topoIdsToClean
-
cleanInbox
Deletes jar files in dirLoc older than seconds.- Parameters:
dirLoc
- the location to look in for fileseconds
- how old is too old and should be deleted
-
topologiesOnSupervisor
-
launchServer
- Throws:
Exception
-
launch
- Throws:
Exception
-
main
- Throws:
Exception
-
setAuthorizationHandler
-
getHeartbeatsCache
-
getIdToExecutors
-
addSupervisor
Used for local cluster.- Parameters:
supervisor
-Supervisor
-
checkAuthorization
public void checkAuthorization(String topoName, Map<String, Object> topoConf, String operation) throws AuthorizationException- Throws:
AuthorizationException
-
checkAuthorization
public void checkAuthorization(String topoName, Map<String, Object> topoConf, String operation, ReqContext context) throws AuthorizationException- Throws:
AuthorizationException
-
filterAuthorized
public Set<String> filterAuthorized(String operation, Collection<String> topoIds) throws NotAliveException, AuthorizationException, IOException -
rmDependencyJarsInTopology
-
rmTopologyKeys
-
forceDeleteTopoDistDir
- Throws:
IOException
-
doCleanup
public void doCleanup()Cleanup topologies and Jars. -
awaitLeadership
- Throws:
InterruptedException
-
submitTopology
public void submitTopology(String name, String uploadedJarLocation, String jsonConf, StormTopology topology) throws AlreadyAliveException, InvalidTopologyException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
submitTopology
in interfaceNimbus.Iface
- Throws:
AlreadyAliveException
InvalidTopologyException
AuthorizationException
org.apache.storm.thrift.TException
-
submitTopologyWithOpts
public void submitTopologyWithOpts(String topoName, String uploadedJarLocation, String jsonConf, StormTopology topology, SubmitOptions options) throws AlreadyAliveException, InvalidTopologyException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
submitTopologyWithOpts
in interfaceNimbus.Iface
- Throws:
AlreadyAliveException
InvalidTopologyException
AuthorizationException
org.apache.storm.thrift.TException
-
setUpAckerExecutorConfigs
-
killTopology
public void killTopology(String name) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
killTopology
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
killTopologyWithOpts
public void killTopologyWithOpts(String topoName, KillOptions options) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
killTopologyWithOpts
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
activate
public void activate(String topoName) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
activate
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
deactivate
public void deactivate(String topoName) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
deactivate
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
rebalance
public void rebalance(String topoName, RebalanceOptions options) throws NotAliveException, InvalidTopologyException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
rebalance
in interfaceNimbus.Iface
- Throws:
NotAliveException
InvalidTopologyException
AuthorizationException
org.apache.storm.thrift.TException
-
setLogConfig
- Specified by:
setLogConfig
in interfaceNimbus.Iface
- Throws:
org.apache.storm.thrift.TException
-
getLogConfig
- Specified by:
getLogConfig
in interfaceNimbus.Iface
- Throws:
org.apache.storm.thrift.TException
-
debug
public void debug(String topoName, String componentId, boolean enable, double samplingPercentage) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException Description copied from interface:Nimbus.Iface
Enable/disable logging the tuples generated in topology via an internal EventLogger bolt. The component name is optional and if null or empty, the debug flag will apply to the entire topology. The 'samplingPercentage' will limit loggging to a percentage of generated tuples.- Specified by:
debug
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
setWorkerProfiler
public void setWorkerProfiler(String topoId, ProfileRequest profileRequest) throws org.apache.storm.thrift.TException - Specified by:
setWorkerProfiler
in interfaceNimbus.Iface
- Throws:
org.apache.storm.thrift.TException
-
getComponentPendingProfileActions
public List<ProfileRequest> getComponentPendingProfileActions(String id, String componentId, ProfileAction action) throws org.apache.storm.thrift.TException - Specified by:
getComponentPendingProfileActions
in interfaceNimbus.Iface
- Throws:
org.apache.storm.thrift.TException
-
uploadNewCredentials
public void uploadNewCredentials(String topoName, Credentials credentials) throws NotAliveException, InvalidTopologyException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
uploadNewCredentials
in interfaceNimbus.Iface
- Throws:
NotAliveException
InvalidTopologyException
AuthorizationException
org.apache.storm.thrift.TException
-
beginCreateBlob
public String beginCreateBlob(String key, SettableBlobMeta meta) throws AuthorizationException, KeyAlreadyExistsException, org.apache.storm.thrift.TException - Specified by:
beginCreateBlob
in interfaceNimbus.Iface
- Throws:
AuthorizationException
KeyAlreadyExistsException
org.apache.storm.thrift.TException
-
beginUpdateBlob
public String beginUpdateBlob(String key) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException - Specified by:
beginUpdateBlob
in interfaceNimbus.Iface
- Throws:
AuthorizationException
KeyNotFoundException
org.apache.storm.thrift.TException
-
uploadBlobChunk
public void uploadBlobChunk(String session, ByteBuffer chunk) throws AuthorizationException, org.apache.storm.thrift.TException - Specified by:
uploadBlobChunk
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
finishBlobUpload
public void finishBlobUpload(String session) throws AuthorizationException, org.apache.storm.thrift.TException - Specified by:
finishBlobUpload
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
cancelBlobUpload
public void cancelBlobUpload(String session) throws AuthorizationException, org.apache.storm.thrift.TException - Specified by:
cancelBlobUpload
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
getBlobMeta
public ReadableBlobMeta getBlobMeta(String key) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException - Specified by:
getBlobMeta
in interfaceNimbus.Iface
- Throws:
AuthorizationException
KeyNotFoundException
org.apache.storm.thrift.TException
-
setBlobMeta
public void setBlobMeta(String key, SettableBlobMeta meta) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException - Specified by:
setBlobMeta
in interfaceNimbus.Iface
- Throws:
AuthorizationException
KeyNotFoundException
org.apache.storm.thrift.TException
-
beginBlobDownload
public BeginDownloadResult beginBlobDownload(String key) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException - Specified by:
beginBlobDownload
in interfaceNimbus.Iface
- Throws:
AuthorizationException
KeyNotFoundException
org.apache.storm.thrift.TException
-
downloadBlobChunk
public ByteBuffer downloadBlobChunk(String session) throws AuthorizationException, org.apache.storm.thrift.TException - Specified by:
downloadBlobChunk
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
deleteBlob
public void deleteBlob(String key) throws AuthorizationException, KeyNotFoundException, IllegalStateException, org.apache.storm.thrift.TException - Specified by:
deleteBlob
in interfaceNimbus.Iface
- Throws:
AuthorizationException
KeyNotFoundException
IllegalStateException
org.apache.storm.thrift.TException
-
listBlobs
- Specified by:
listBlobs
in interfaceNimbus.Iface
- Throws:
org.apache.storm.thrift.TException
-
getBlobReplication
public int getBlobReplication(String key) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException - Specified by:
getBlobReplication
in interfaceNimbus.Iface
- Throws:
AuthorizationException
KeyNotFoundException
org.apache.storm.thrift.TException
-
updateBlobReplication
public int updateBlobReplication(String key, int replication) throws AuthorizationException, KeyNotFoundException, org.apache.storm.thrift.TException - Specified by:
updateBlobReplication
in interfaceNimbus.Iface
- Throws:
AuthorizationException
KeyNotFoundException
org.apache.storm.thrift.TException
-
createStateInZookeeper
- Specified by:
createStateInZookeeper
in interfaceNimbus.Iface
- Throws:
org.apache.storm.thrift.TException
-
beginFileUpload
- Specified by:
beginFileUpload
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
uploadChunk
public void uploadChunk(String location, ByteBuffer chunk) throws AuthorizationException, org.apache.storm.thrift.TException - Specified by:
uploadChunk
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
finishFileUpload
public void finishFileUpload(String location) throws AuthorizationException, org.apache.storm.thrift.TException - Specified by:
finishFileUpload
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
downloadChunk
public ByteBuffer downloadChunk(String id) throws AuthorizationException, org.apache.storm.thrift.TException - Specified by:
downloadChunk
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
getNimbusConf
- Specified by:
getNimbusConf
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
getTopologyInfo
public TopologyInfo getTopologyInfo(String id) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getTopologyInfo
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getTopologyInfoByName
- Specified by:
getTopologyInfoByName
in interfaceNimbus.Iface
- Throws:
org.apache.storm.thrift.TException
-
getTopologyInfoByNameWithOpts
public TopologyInfo getTopologyInfoByNameWithOpts(String name, GetInfoOptions options) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getTopologyInfoByNameWithOpts
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getTopologyInfoWithOpts
public TopologyInfo getTopologyInfoWithOpts(String topoId, GetInfoOptions options) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getTopologyInfoWithOpts
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getTopologyPageInfo
public TopologyPageInfo getTopologyPageInfo(String topoId, String window, boolean includeSys) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getTopologyPageInfo
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getSupervisorPageInfo
public SupervisorPageInfo getSupervisorPageInfo(String superId, String host, boolean includeSys) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getSupervisorPageInfo
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getComponentPageInfo
public ComponentPageInfo getComponentPageInfo(String topoId, String componentId, String window, boolean includeSys) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getComponentPageInfo
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getTopologyConf
public String getTopologyConf(String id) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getTopologyConf
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getTopology
public StormTopology getTopology(String id) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException Description copied from interface:Nimbus.Iface
Returns the compiled topology that contains ackers and metrics consumsers. CompareNimbus.Iface.getUserTopology(String id)
.- Specified by:
getTopology
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getUserTopology
public StormTopology getUserTopology(String id) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException Description copied from interface:Nimbus.Iface
Returns the user specified topology as submitted originally. CompareNimbus.Iface.getTopology(String id)
.- Specified by:
getUserTopology
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getTopologyHistory
public TopologyHistoryInfo getTopologyHistory(String user) throws AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getTopologyHistory
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
getClusterInfo
public ClusterSummary getClusterInfo() throws AuthorizationException, org.apache.storm.thrift.TException- Specified by:
getClusterInfo
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
getTopologySummaries
public List<TopologySummary> getTopologySummaries() throws AuthorizationException, org.apache.storm.thrift.TException- Specified by:
getTopologySummaries
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
getTopologySummaryByName
public TopologySummary getTopologySummaryByName(String name) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getTopologySummaryByName
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getTopologySummary
public TopologySummary getTopologySummary(String id) throws NotAliveException, AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getTopologySummary
in interfaceNimbus.Iface
- Throws:
NotAliveException
AuthorizationException
org.apache.storm.thrift.TException
-
getLeader
- Specified by:
getLeader
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
isTopologyNameAllowed
public boolean isTopologyNameAllowed(String name) throws AuthorizationException, org.apache.storm.thrift.TException - Specified by:
isTopologyNameAllowed
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
getOwnerResourceSummaries
public List<OwnerResourceSummary> getOwnerResourceSummaries(String owner) throws AuthorizationException, org.apache.storm.thrift.TException - Specified by:
getOwnerResourceSummaries
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
getSupervisorAssignments
public SupervisorAssignments getSupervisorAssignments(String nodeId) throws AuthorizationException, org.apache.storm.thrift.TException Description copied from interface:Nimbus.Iface
Get assigned assignments for a specific supervisor- Specified by:
getSupervisorAssignments
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
sendSupervisorWorkerHeartbeats
public void sendSupervisorWorkerHeartbeats(SupervisorWorkerHeartbeats heartbeats) throws AuthorizationException, org.apache.storm.thrift.TException Description copied from interface:Nimbus.Iface
Send supervisor worker heartbeats for a specific supervisor- Specified by:
sendSupervisorWorkerHeartbeats
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
sendSupervisorWorkerHeartbeat
public void sendSupervisorWorkerHeartbeat(SupervisorWorkerHeartbeat hb) throws AuthorizationException, org.apache.storm.thrift.TException Description copied from interface:Nimbus.Iface
Send supervisor local worker heartbeat when a supervisor is unreachable- Specified by:
sendSupervisorWorkerHeartbeat
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-
shutdown
public void shutdown()- Specified by:
shutdown
in interfaceShutdownable
-
isWaiting
public boolean isWaiting()- Specified by:
isWaiting
in interfaceDaemonCommon
-
processWorkerMetrics
- Specified by:
processWorkerMetrics
in interfaceNimbus.Iface
- Throws:
org.apache.storm.thrift.TException
-
isRemoteBlobExists
public boolean isRemoteBlobExists(String blobKey) throws AuthorizationException, org.apache.storm.thrift.TException Description copied from interface:Nimbus.Iface
Decide if the blob is removed from cluster.- Specified by:
isRemoteBlobExists
in interfaceNimbus.Iface
- Throws:
AuthorizationException
org.apache.storm.thrift.TException
-