public class Config extends HashMap<String,Object>
This class also provides constants for all the configurations possible on a Storm cluster and Storm topology. Each constant is paired with an annotation that defines the validity criterion of the corresponding field. Default values for these configs can be found in defaults.yaml.
Note that you may put other configurations in any of the configs. Storm will ignore anything it doesn't recognize, but your topologies are free to make use of them by reading them in the prepare method of Bolts or the open method of Spouts.
AbstractMap.SimpleEntry<K,V>, AbstractMap.SimpleImmutableEntry<K,V>
Modifier and Type | Field and Description |
---|---|
static String |
BLOBSTORE_CLEANUP_ENABLE
Enable the blobstore cleaner.
|
static String |
BLOBSTORE_DIR
What directory to use for the blobstore.
|
static String |
BLOBSTORE_HDFS_KEYTAB
Deprecated.
|
static String |
BLOBSTORE_HDFS_PRINCIPAL
Deprecated.
|
static String |
CLIENT_BLOBSTORE
What blobstore implementation the storm client should use.
|
static String |
DISABLE_SYMLINKS
On some systems (windows for example) symlinks require special privileges that not everyone wants to grant a headless user.
|
static String |
DRPC_AUTHORIZER_ACL
The Access Control List for the DRPC Authorizer.
|
static String |
DRPC_AUTHORIZER_ACL_FILENAME
File name of the DRPC Authorizer ACL.
|
static String |
DRPC_AUTHORIZER_ACL_STRICT
Whether the DRPCSimpleAclAuthorizer should deny requests for operations involving functions that have no explicit ACL entry.
|
static String |
DRPC_INVOCATIONS_PORT
This port on Storm DRPC is used by DRPC topologies to receive function invocations and send results back.
|
static String |
DRPC_INVOCATIONS_THREADS
DRPC invocations thrift server worker threads.
|
static String |
DRPC_INVOCATIONS_THRIFT_TRANSPORT_PLUGIN
The DRPC invocations transport plug-in for Thrift client/server communication.
|
static String |
DRPC_MAX_BUFFER_SIZE
The maximum buffer size thrift should use when reading messages for DRPC.
|
static String |
DRPC_PORT
This port is used by Storm DRPC for receiving DPRC requests from clients.
|
static String |
DRPC_QUEUE_SIZE
DRPC thrift server queue size.
|
static String |
DRPC_SERVERS
List of DRPC servers so that the DRPCSpout knows who to talk to.
|
static String |
DRPC_THRIFT_TRANSPORT_PLUGIN
The DRPC transport plug-in for Thrift client/server communication.
|
static String |
DRPC_WORKER_THREADS
DRPC thrift server worker threads.
|
static String |
EXECUTOR_METRICS_FREQUENCY_SECS
How often executor metrics should report to master, used for RPC heartbeat mode.
|
static String |
NIMBUS_ADMINS
A list of users that are cluster admins and can run any command.
|
static String |
NIMBUS_ADMINS_GROUPS
A list of groups that are cluster admins and can run any command.
|
static String |
NIMBUS_AUTO_CRED_PLUGINS
A list of plugins that nimbus should load during submit topology to populate credentials on user's behalf.
|
static String |
NIMBUS_CREDENTIAL_RENEWERS
A list of credential renewers that nimbus should load.
|
static String |
NIMBUS_DAEMON_USERS
A list of users that nimbus runs as and should be authorized to interact with the supervisor as nimbus would.
|
static String |
NIMBUS_GROUPS
A list of groups , users belong to these groups are the only ones allowed to run user operation on storm cluster.
|
static String |
NIMBUS_IMPERSONATION_ACL
Impersonation user ACL config entries.
|
static String |
NIMBUS_LOCAL_ASSIGNMENTS_BACKEND_CLASS
Nimbus assignments backend for storing local assignments.
|
static String |
NIMBUS_QUEUE_SIZE
Nimbus thrift server queue size, default is 100000.
|
static String |
NIMBUS_SCHEDULER_STRATEGY_CLASS_WHITELIST
A whitelist of the RAS scheduler strategies allowed by nimbus.
|
static String |
NIMBUS_SEEDS
List of seed nimbus hosts to use for leader nimbus discovery.
|
static String |
NIMBUS_SUPERVISOR_USERS
A list of users that run the supervisors and should be authorized to interact with nimbus as a supervisor would.
|
static String |
NIMBUS_THRIFT_MAX_BUFFER_SIZE
The maximum buffer size thrift should use when reading messages.
|
static String |
NIMBUS_THRIFT_PORT
Which port the Thrift interface of Nimbus should run on.
|
static String |
NIMBUS_THRIFT_THREADS
The number of threads that should be used by the nimbus thrift server.
|
static String |
NIMBUS_THRIFT_TRANSPORT_PLUGIN
The Nimbus transport plug-in for Thrift client/server communication.
|
static String |
NIMBUS_USERS
A list of users that are the only ones allowed to run user operation on storm cluster.
|
static String |
NUM_STAT_BUCKETS
The number of Buckets.
|
static String |
PACEMAKER_AUTH_METHOD
This should be one of "DIGEST", "KERBEROS", or "NONE" Determines the mode of authentication the pacemaker server and client use.
|
static String |
PACEMAKER_CLIENT_MAX_THREADS
The maximum number of threads that should be used by the Pacemaker client.
|
static String |
PACEMAKER_PORT
The port Pacemaker should run on.
|
static String |
PACEMAKER_SERVERS
The list of servers that Pacemaker is running on.
|
static String |
PACEMAKER_THRIFT_MESSAGE_SIZE_MAX
Pacemaker Thrift Max Message Size (bytes).
|
static String |
STORM_BLOBSTORE_ACL_VALIDATION_ENABLED
For secure mode we would want to turn on this config By default this is turned off assuming the default is insecure.
|
static String |
STORM_BLOBSTORE_DEPENDENCY_JAR_UPLOAD_CHUNK_SIZE_BYTES
What chunk size to use for storm client to upload dependency jars.
|
static String |
STORM_BLOBSTORE_INPUTSTREAM_BUFFER_SIZE_BYTES
What buffer size to use for the blobstore uploads.
|
static String |
STORM_BLOBSTORE_REPLICATION_FACTOR
Set replication factor for a blob in HDFS Blobstore Implementation.
|
static String |
STORM_CGROUP_HIERARCHY_DIR
root directory of the storm cgroup hierarchy.
|
static String |
STORM_CLUSTER_MODE
The mode this Storm cluster is running in.
|
static String |
STORM_CLUSTER_STATE_STORE
The ClusterState factory that worker will use to create a ClusterState to store state in.
|
static String |
STORM_DAEMON_METRICS_REPORTER_PLUGIN_DURATION_UNIT
Specify the duration unit in TimeUnit for daemon metrics reporter plugin.
|
static String |
STORM_DAEMON_METRICS_REPORTER_PLUGIN_LOCALE
Specify the Locale for daemon metrics reporter plugin.
|
static String |
STORM_DAEMON_METRICS_REPORTER_PLUGIN_RATE_UNIT
Specify the rate unit in TimeUnit for daemon metrics reporter plugin.
|
static String |
STORM_DO_AS_USER
The user as which the nimbus client should be acquired to perform the operation.
|
static String |
STORM_EXHIBITOR_POLL
How often to poll Exhibitor cluster in millis.
|
static String |
STORM_EXHIBITOR_PORT
The port Storm will use to connect to each of the exhibitor servers.
|
static String |
STORM_EXHIBITOR_RETRY_INTERVAL |
static String |
STORM_EXHIBITOR_RETRY_INTERVAL_CEILING
The ceiling of the interval between retries of an Exhibitor operation.
|
static String |
STORM_EXHIBITOR_RETRY_TIMES
The number of times to retry an Exhibitor operation.
|
static String |
STORM_EXHIBITOR_SERVERS
A list of hosts of Exhibitor servers used to discover/maintain connection to ZooKeeper cluster.
|
static String |
STORM_EXHIBITOR_URIPATH |
static String |
STORM_GROUP_MAPPING_SERVICE_CACHE_DURATION_SECS
Max no.of seconds group mapping service will cache user groups
|
static String |
STORM_GROUP_MAPPING_SERVICE_PARAMS
Initialization parameters for the group mapping service plugin.
|
static String |
STORM_GROUP_MAPPING_SERVICE_PROVIDER_PLUGIN
The plugin that will provide user groups service.
|
static String |
STORM_HDFS_LOGIN_KEYTAB
The keytab for nimbus/supervisor to use to access secure hdfs.
|
static String |
STORM_HDFS_LOGIN_PRINCIPAL
The principal for nimbus/supervisor to use to access secure hdfs.
|
static String |
STORM_ID
The id assigned to a running topology.
|
static String |
STORM_LOCAL_DIR
A directory on the local filesystem used by Storm for any local filesystem usage it needs.
|
static String |
STORM_LOCAL_HOSTNAME
The hostname the supervisors/workers should report to nimbus.
|
static String |
STORM_LOCAL_MODE_ZMQ
Whether or not to use ZeroMQ for messaging in local mode.
|
static String |
STORM_MESSAGING_NETTY_AUTHENTICATION
Netty based messaging: Is authentication required for Netty messaging from client worker process to server worker process.
|
static String |
STORM_MESSAGING_NETTY_BUFFER_HIGH_WATERMARK
Netty based messaging: The netty write buffer high watermark in bytes.
|
static String |
STORM_MESSAGING_NETTY_BUFFER_LOW_WATERMARK
Netty based messaging: The netty write buffer low watermark in bytes.
|
static String |
STORM_MESSAGING_NETTY_BUFFER_SIZE
Netty based messaging: The buffer size for send/recv buffer.
|
static String |
STORM_MESSAGING_NETTY_CLIENT_WORKER_THREADS
Netty based messaging: The # of worker threads for the client.
|
static String |
STORM_MESSAGING_NETTY_MAX_SLEEP_MS
Netty based messaging: The max # of milliseconds that a peer will wait.
|
static String |
STORM_MESSAGING_NETTY_MIN_SLEEP_MS
Netty based messaging: The min # of milliseconds that a peer will wait.
|
static String |
STORM_MESSAGING_NETTY_SERVER_WORKER_THREADS
Netty based messaging: The # of worker threads for the server.
|
static String |
STORM_MESSAGING_NETTY_SOCKET_BACKLOG
Netty based messaging: Sets the backlog value to specify when the channel binds to a local address.
|
static String |
STORM_MESSAGING_TRANSPORT
The transporter for communication among Storm tasks.
|
static String |
STORM_META_SERIALIZATION_DELEGATE
The delegate for serializing metadata, should be used for serialized objects stored in zookeeper and on disk.
|
static String |
STORM_METRICS_REPORTERS
Deprecated.
Use
TOPOLOGY_METRICS_REPORTERS instead. |
static String |
STORM_NETTY_MESSAGE_BATCH_SIZE
If the Netty messaging layer is busy, the Netty client will try to batch message as more as possible up to the size of
STORM_NETTY_MESSAGE_BATCH_SIZE bytes.
|
static String |
STORM_NETWORK_TOPOGRAPHY_PLUGIN
What Network Topography detection classes should we use.
|
static String |
STORM_NIMBUS_RETRY_INTERVAL
The starting interval between exponential backoff retries of a Nimbus operation.
|
static String |
STORM_NIMBUS_RETRY_INTERVAL_CEILING
The ceiling of the interval between retries of a client connect to Nimbus operation.
|
static String |
STORM_NIMBUS_RETRY_TIMES
The number of times to retry a Nimbus operation.
|
static String |
STORM_OCI_CGROUP_ROOT
The root of cgroup for oci to use.
|
static String |
STORM_PRINCIPAL_TO_LOCAL_PLUGIN
The plugin that will convert a principal to a local user.
|
static String |
STORM_THRIFT_SOCKET_TIMEOUT_MS
How long before a Thrift Client socket hangs before timeout and restart the socket.
|
static String |
STORM_THRIFT_TRANSPORT_PLUGIN
The default transport plug-in for Thrift client/server communication.
|
static String |
STORM_TOPOLOGY_METRICS_SYSTEM_REPORTERS
A list of system metrics reporters that will get added to each topology.
|
static String |
STORM_TOPOLOGY_SUBMISSION_NOTIFIER_PLUGIN
FQCN of a class that implements
ISubmitterHook @see ISubmitterHook for details. |
static String |
STORM_WORKERS_ARTIFACTS_DIR
The workers-artifacts directory (where we place all workers' logs), can be either absolute or relative.
|
static String |
STORM_ZOOKEEPER_AUTH_PAYLOAD
A string representing the payload for cluster Zookeeper authentication.
|
static String |
STORM_ZOOKEEPER_AUTH_SCHEME
The cluster Zookeeper authentication scheme to use, e.g.
|
static String |
STORM_ZOOKEEPER_CONNECTION_TIMEOUT
The connection timeout for clients to ZooKeeper.
|
static String |
STORM_ZOOKEEPER_DRPC_ACL
The ACL of the drpc user in zookeeper so the drpc servers can verify worker tokens.
|
static String |
STORM_ZOOKEEPER_PORT
The port Storm will use to connect to each of the ZooKeeper servers.
|
static String |
STORM_ZOOKEEPER_RETRY_INTERVAL
The interval between retries of a Zookeeper operation.
|
static String |
STORM_ZOOKEEPER_RETRY_INTERVAL_CEILING
The ceiling of the interval between retries of a Zookeeper operation.
|
static String |
STORM_ZOOKEEPER_RETRY_TIMES
The number of times to retry a Zookeeper operation.
|
static String |
STORM_ZOOKEEPER_ROOT
The root location at which Storm stores data in ZooKeeper.
|
static String |
STORM_ZOOKEEPER_SERVERS
A list of hosts of ZooKeeper servers used to manage the cluster.
|
static String |
STORM_ZOOKEEPER_SESSION_TIMEOUT
The session timeout for clients to ZooKeeper.
|
static String |
STORM_ZOOKEEPER_SUPERACL
This is part of a temporary workaround to a ZK bug, it is the 'scheme:acl' for the user Nimbus and Supervisors use to authenticate
with ZK.
|
static String |
STORM_ZOOKEEPER_TOPOLOGY_AUTH_PAYLOAD
A string representing the payload for topology Zookeeper authentication.
|
static String |
STORM_ZOOKEEPER_TOPOLOGY_AUTH_SCHEME
The topology Zookeeper authentication scheme to use, e.g.
|
static String |
SUPERVISOR_CPU_CAPACITY
The total amount of CPU resources a supervisor is allowed to give to its workers.
|
static String |
SUPERVISOR_MEMORY_CAPACITY_MB
The total amount of memory (in MiB) a supervisor is allowed to give to its workers.
|
static String |
SUPERVISOR_QUEUE_SIZE |
static String |
SUPERVISOR_RESOURCES_MAP
A map of resources the Supervisor has e.g {"cpu.pcore.percent" : 200.0.
|
static String |
SUPERVISOR_RUN_WORKER_AS_USER
Should the supervior try to run the worker as the lauching user or not.
|
static String |
SUPERVISOR_THRIFT_MAX_BUFFER_SIZE |
static String |
SUPERVISOR_THRIFT_PORT |
static String |
SUPERVISOR_THRIFT_SOCKET_TIMEOUT_MS
How long before a supervisor Thrift Client socket hangs before timeout and restart the socket.
|
static String |
SUPERVISOR_THRIFT_THREADS |
static String |
SUPERVISOR_THRIFT_TRANSPORT_PLUGIN |
static String |
SUPERVISOR_WORKER_DEFAULT_VERSION
The version of storm to assume a topology should run as if not version is given by the client when submitting the topology.
|
static String |
SUPERVISOR_WORKER_HEARTBEATS_MAX_TIMEOUT_SECS
max timeout for supervisor reported heartbeats when master gains leadership.
|
static String |
SUPERVISOR_WORKER_LAUNCHER
Full path to the worker-laucher executable that will be used to lauch workers when SUPERVISOR_RUN_WORKER_AS_USER is set to true.
|
static String |
SUPERVISOR_WORKER_SHUTDOWN_SLEEP_SECS
How many seconds to allow for graceful worker shutdown when killing workers before resorting to force kill.
|
static String |
SUPERVISOR_WORKER_TIMEOUT_SECS
How long a worker can go without heartbeating before the supervisor tries to restart the worker process.
|
static String |
SUPERVISOR_WORKER_VERSION_CLASSPATH_MAP
Map a version of storm to a worker classpath that can be used to run it.
|
static String |
SUPERVISOR_WORKER_VERSION_LOGWRITER_MAP
Map a version of storm to a worker's logwriter class.
|
static String |
SUPERVISOR_WORKER_VERSION_MAIN_MAP
Map a version of storm to a worker's main class.
|
static String |
TASK_CREDENTIALS_POLL_SECS
How often a task should sync credentials, worst case.
|
static String |
TASK_HEARTBEAT_FREQUENCY_SECS
How often a task should heartbeat its status to the Pacamker.
|
static String |
TASK_REFRESH_POLL_SECS
How often a task should sync its connections with other tasks (if a task is reassigned, the other tasks sending messages to it need
to refresh their connections).
|
static String |
TOPOLOGY_ACKER_CPU_PCORE_PERCENT
The config indicates the percentage of cpu for a core an instance(executor) of an acker will use.
|
static String |
TOPOLOGY_ACKER_EXECUTORS
How many executors to spawn for ackers.
|
static String |
TOPOLOGY_ACKER_RESOURCES_OFFHEAP_MEMORY_MB
The maximum amount of memory an instance of an acker will take off heap.
|
static String |
TOPOLOGY_ACKER_RESOURCES_ONHEAP_MEMORY_MB
The maximum amount of memory an instance of an acker will take on heap.
|
static String |
TOPOLOGY_AUTO_CREDENTIALS
A list of IAutoCredentials that the topology should load and use.
|
static String |
TOPOLOGY_AUTO_TASK_HOOKS
A list of task hooks that are automatically added to every spout and bolt in the topology.
|
static String |
TOPOLOGY_BACKPRESSURE_CHECK_MILLIS
How often a worker should check and notify upstream workers about its tasks that are no longer experiencing BP and able to receive
new messages.
|
static String |
TOPOLOGY_BACKPRESSURE_ENABLE
Deprecated.
|
static String |
TOPOLOGY_BACKPRESSURE_WAIT_PARK_MICROSEC
Configures park time if using WaitStrategyPark for BackPressure.
|
static String |
TOPOLOGY_BACKPRESSURE_WAIT_PROGRESSIVE_LEVEL1_COUNT
Configures steps used to determine progression to the next level of wait ..
|
static String |
TOPOLOGY_BACKPRESSURE_WAIT_PROGRESSIVE_LEVEL2_COUNT
Configures steps used to determine progression to the next level of wait ..
|
static String |
TOPOLOGY_BACKPRESSURE_WAIT_PROGRESSIVE_LEVEL3_SLEEP_MILLIS
Configures sleep time if using WaitStrategyProgressive for BackPressure.
|
static String |
TOPOLOGY_BACKPRESSURE_WAIT_STRATEGY
A class that implements a wait strategy for an upstream component (spout/bolt) trying to write to a downstream component whose recv
queue is full
|
static String |
TOPOLOGY_BATCH_FLUSH_INTERVAL_MILLIS
How often to send flush tuple to the executors for flushing out batched events.
|
static String |
TOPOLOGY_BLOBSTORE_MAP
A map with blobstore keys mapped to each filename the worker will have access to in the launch directory to the blob by local file
name, uncompress flag, and if the worker should restart when the blob is updated.
|
static String |
TOPOLOGY_BOLT_WAIT_PARK_MICROSEC
Configures park time for WaitStrategyPark.
|
static String |
TOPOLOGY_BOLT_WAIT_PROGRESSIVE_LEVEL1_COUNT
Configures number of iterations to spend in level 1 of WaitStrategyProgressive, before progressing to level 2.
|
static String |
TOPOLOGY_BOLT_WAIT_PROGRESSIVE_LEVEL2_COUNT
Configures number of iterations to spend in level 2 of WaitStrategyProgressive, before progressing to level 3.
|
static String |
TOPOLOGY_BOLT_WAIT_PROGRESSIVE_LEVEL3_SLEEP_MILLIS
Configures sleep time for WaitStrategyProgressive.
|
static String |
TOPOLOGY_BOLT_WAIT_STRATEGY
Selects the Bolt's Wait Strategy to use when there are no incoming msgs.
|
static String |
TOPOLOGY_BOLTS_LATE_TUPLE_STREAM
Bolt-specific configuration for windowed bolts to specify the name of the stream on which late tuples are going to be emitted.
|
static String |
TOPOLOGY_BOLTS_MESSAGE_ID_FIELD_NAME |
static String |
TOPOLOGY_BOLTS_SLIDING_INTERVAL_COUNT |
static String |
TOPOLOGY_BOLTS_SLIDING_INTERVAL_DURATION_MS |
static String |
TOPOLOGY_BOLTS_TUPLE_TIMESTAMP_MAX_LAG_MS
Bolt-specific configuration for windowed bolts to specify the maximum time lag of the tuple timestamp in milliseconds.
|
static String |
TOPOLOGY_BOLTS_WATERMARK_EVENT_INTERVAL_MS |
static String |
TOPOLOGY_BOLTS_WINDOW_LENGTH_COUNT |
static String |
TOPOLOGY_BOLTS_WINDOW_LENGTH_DURATION_MS |
static String |
TOPOLOGY_BUILTIN_METRICS_BUCKET_SIZE_SECS
The time period that builtin metrics data in bucketed into.
|
static String |
TOPOLOGY_CLASSPATH
Topology-specific classpath for the worker child process.
|
static String |
TOPOLOGY_CLASSPATH_BEGINNING
Topology-specific classpath for the worker child process.
|
static String |
TOPOLOGY_COMPONENT_CPU_PCORE_PERCENT
The config indicates the percentage of cpu for a core an instance(executor) of a component will use.
|
static String |
TOPOLOGY_COMPONENT_RESOURCES_MAP
A map of resources used by each component e.g {"cpu.pcore.percent" : 200.0.
|
static String |
TOPOLOGY_COMPONENT_RESOURCES_OFFHEAP_MEMORY_MB
The maximum amount of memory an instance of a spout/bolt will take off heap.
|
static String |
TOPOLOGY_COMPONENT_RESOURCES_ONHEAP_MEMORY_MB
The maximum amount of memory an instance of a spout/bolt will take on heap.
|
static String |
TOPOLOGY_DEBUG
When set to true, Storm will log every message that's emitted.
|
static String |
TOPOLOGY_DISABLE_LOADAWARE_MESSAGING
Disable load aware grouping support.
|
static String |
TOPOLOGY_ENABLE_MESSAGE_TIMEOUTS
True if Storm should timeout messages or not.
|
static String |
TOPOLOGY_ENABLE_SEND_ICONNECTION_METRICS
This config allows a topology to enable/disable reporting of __send-iconnection metrics.
|
static String |
TOPOLOGY_ENABLE_V2_METRICS_TICK
This config allows a topology to report metrics data points from the V2 metrics API through the metrics tick.
|
static String |
TOPOLOGY_ENVIRONMENT
Topology-specific environment variables for the worker child process.
|
static String |
TOPOLOGY_ERROR_THROTTLE_INTERVAL_SECS
The interval in seconds to use for determining whether to throttle error reported to Zookeeper.
|
static String |
TOPOLOGY_EVENT_LOGGER_REGISTER
A list of classes implementing IEventLogger (See storm.yaml.example for exact config format).
|
static String |
TOPOLOGY_EVENTLOGGER_EXECUTORS
How many executors to spawn for event logger.
|
static String |
TOPOLOGY_EXECUTOR_OVERFLOW_LIMIT
If number of items in task's overflowQ exceeds this, new messages coming from other workers to this task will be dropped This
prevents OutOfMemoryException that can occur in rare scenarios in the presence of BackPressure.
|
static String |
TOPOLOGY_EXECUTOR_RECEIVE_BUFFER_SIZE
The size of the receive queue for each executor.
|
static String |
TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION
Whether or not to use Java serialization in a topology.
|
static String |
TOPOLOGY_GROUPS
A list of groups that are allowed to interact with the topology.
|
static String |
TOPOLOGY_ISOLATED_MACHINES
The maximum number of machines that should be used by this topology.
|
static String |
TOPOLOGY_KRYO_DECORATORS
A list of classes that customize storm's kryo instance during start-up.
|
static String |
TOPOLOGY_KRYO_FACTORY
Class that specifies how to create a Kryo instance for serialization.
|
static String |
TOPOLOGY_KRYO_REGISTER
A list of serialization registrations for Kryo ( https://github.com/EsotericSoftware/kryo ), the underlying serialization framework
for Storm.
|
static String |
TOPOLOGY_LOCALITYAWARE_HIGHER_BOUND
This signifies the load congestion among target tasks in scope.
|
static String |
TOPOLOGY_LOCALITYAWARE_LOWER_BOUND
This signifies the load congestion among target tasks in scope.
|
static String |
TOPOLOGY_LOGGING_CONFIG_FILE
Log file the user can use to configure Log4j2.
|
static String |
TOPOLOGY_LOGGING_SENSITIVITY
Topology central logging sensitivity to determine who has access to logs in central logging system.
|
static String |
TOPOLOGY_MAX_ERROR_REPORT_PER_INTERVAL
See doc for
TOPOLOGY_ERROR_THROTTLE_INTERVAL_SECS . |
static String |
TOPOLOGY_MAX_REPLICATION_WAIT_TIME_SEC
Maximum wait time for the nimbus host replication to achieve the nimbus.min.replication.count.
|
static String |
TOPOLOGY_MAX_SPOUT_PENDING
The maximum number of tuples that can be pending on a spout task at any given time.
|
static String |
TOPOLOGY_MAX_TASK_PARALLELISM
The maximum parallelism allowed for a component in this topology.
|
static String |
TOPOLOGY_MESSAGE_TIMEOUT_SECS
The maximum amount of time given to the topology to fully process a message emitted by a spout.
|
static String |
TOPOLOGY_METRICS_CONSUMER_CPU_PCORE_PERCENT
The config indicates the percentage of cpu for a core an instance(executor) of a metrics consumer will use.
|
static String |
TOPOLOGY_METRICS_CONSUMER_REGISTER
A list of classes implementing IMetricsConsumer (See storm.yaml.example for exact config format).
|
static String |
TOPOLOGY_METRICS_CONSUMER_RESOURCES_OFFHEAP_MEMORY_MB
The maximum amount of memory an instance of a metrics consumer will take off heap.
|
static String |
TOPOLOGY_METRICS_CONSUMER_RESOURCES_ONHEAP_MEMORY_MB
The maximum amount of memory an instance of a metrics consumer will take on heap.
|
static String |
TOPOLOGY_METRICS_REPORTERS
Configure the topology metrics reporters to be used on workers.
|
static String |
TOPOLOGY_MIN_REPLICATION_COUNT
Minimum number of nimbus hosts where the code must be replicated before leader nimbus is allowed to perform topology activation tasks
like setting up heartbeats/assignments and marking the topology as active.
|
static String |
TOPOLOGY_MULTILANG_LOG_HANDLER
The fully qualified name of a
ShellLogHandler to handle output from non-JVM processes e.g. |
static String |
TOPOLOGY_MULTILANG_SERIALIZER
The serializer for communication between shell components and non-JVM processes.
|
static String |
TOPOLOGY_NAME
Name of the topology.
|
static String |
TOPOLOGY_OCI_IMAGE
Specify the oci image to use.
|
static String |
TOPOLOGY_PRIORITY
Sets the priority for a topology.
|
static String |
TOPOLOGY_PRODUCER_BATCH_SIZE
The number of tuples to batch before sending to the destination executor.
|
static String |
TOPOLOGY_RAS_ACKER_EXECUTORS_PER_WORKER
How many ackers to put in when launching a new worker until we run out of ackers.
|
static String |
TOPOLOGY_RAS_CONSTRAINT_MAX_STATE_SEARCH
The maximum number of states that will be searched looking for a solution in resource aware strategies, e.g.
|
static String |
TOPOLOGY_RAS_CONSTRAINT_MAX_TIME_SECS
The maximum number of seconds to spend scheduling a topology using resource aware strategies, e.g.
|
static String |
TOPOLOGY_RAS_CONSTRAINTS
Declare scheduling constraints for a topology used by the constraint solver strategy.
|
static String |
TOPOLOGY_RAS_ONE_COMPONENT_PER_WORKER
Whether to limit each worker to one component.
|
static String |
TOPOLOGY_RAS_ONE_EXECUTOR_PER_WORKER |
static String |
TOPOLOGY_RAS_ORDER_EXECUTORS_BY_PROXIMITY_NEEDS
If set to true, unassigned executors will be sorted by topological order with network proximity needs before being scheduled.
|
static String |
TOPOLOGY_READONLY_GROUPS
A list of readonly groups that are allowed to interact with the topology.
|
static String |
TOPOLOGY_READONLY_USERS
A list of readonly users that are allowed to interact with the topology.
|
static String |
TOPOLOGY_SCHEDULER_FAVORED_NODES
A list of host names that this topology would prefer to be scheduled on (no guarantee is given though).
|
static String |
TOPOLOGY_SCHEDULER_STRATEGY
The strategy to use when scheduling a topology with Resource Aware Scheduler.
|
static String |
TOPOLOGY_SCHEDULER_UNFAVORED_NODES
A list of host names that this topology would prefer to NOT be scheduled on (no guarantee is given though).
|
static String |
TOPOLOGY_SERIALIZED_MESSAGE_SIZE_METRICS
Enable tracking of network message byte counts per source-destination task.
|
static String |
TOPOLOGY_SHELLBOLT_MAX_PENDING
Max pending tuples in one ShellBolt.
|
static String |
TOPOLOGY_SKIP_MISSING_KRYO_REGISTRATIONS
Whether or not Storm should skip the loading of kryo registrations for which it does not know the class or have the serializer
implementation.
|
static String |
TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS
The amount of milliseconds the SleepEmptyEmitStrategy should sleep for.
|
static String |
TOPOLOGY_SPOUT_RECVQ_SKIPS
Check recvQ after every N invocations of Spout's nextTuple() [when ACKing is disabled].
|
static String |
TOPOLOGY_SPOUT_WAIT_PARK_MICROSEC
Configures park time for WaitStrategyPark for spout.
|
static String |
TOPOLOGY_SPOUT_WAIT_PROGRESSIVE_LEVEL1_COUNT
Configures number of iterations to spend in level 1 of WaitStrategyProgressive, before progressing to level 2.
|
static String |
TOPOLOGY_SPOUT_WAIT_PROGRESSIVE_LEVEL2_COUNT
Configures number of iterations to spend in level 2 of WaitStrategyProgressive, before progressing to level 3.
|
static String |
TOPOLOGY_SPOUT_WAIT_PROGRESSIVE_LEVEL3_SLEEP_MILLIS
Configures sleep time for WaitStrategyProgressive.
|
static String |
TOPOLOGY_SPOUT_WAIT_STRATEGY
A class that implements a wait strategy for spout.
|
static String |
TOPOLOGY_SPREAD_COMPONENTS
Deprecated.
|
static String |
TOPOLOGY_STATE_CHECKPOINT_INTERVAL
Topology configuration to specify the checkpoint interval (in millis) at which the topology state is saved when
IStatefulBolt bolts are involved. |
static String |
TOPOLOGY_STATE_KRYO_REGISTER
List of classes to register during state serialization.
|
static String |
TOPOLOGY_STATE_PROVIDER
The class name of the
StateProvider implementation. |
static String |
TOPOLOGY_STATE_PROVIDER_CONFIG
The configuration specific to the
StateProvider implementation. |
static String |
TOPOLOGY_STATE_SYNCHRONIZATION_TIMEOUT_SECS
The maximum amount of time a component gives a source of state to synchronize before it requests synchronization again.
|
static String |
TOPOLOGY_STATS_SAMPLE_RATE
The percentage of tuples to sample to produce stats for a task.
|
static String |
TOPOLOGY_SUBMITTER_PRINCIPAL
The principal who submitted a topology.
|
static String |
TOPOLOGY_SUBMITTER_USER
The local user name of the user who submitted a topology.
|
static String |
TOPOLOGY_SUBPROCESS_TIMEOUT_SECS
How long a subprocess can go without heartbeating before the ShellSpout/ShellBolt tries to suicide itself.
|
static String |
TOPOLOGY_TASKS
How many instances to create for a spout/bolt.
|
static String |
TOPOLOGY_TESTING_ALWAYS_TRY_SERIALIZE
Try to serialize all tuples, even for local transfers.
|
static String |
TOPOLOGY_TICK_TUPLE_FREQ_SECS
How often a tick tuple from the "__system" component and "__tick" stream should be sent to tasks.
|
static String |
TOPOLOGY_TRANSACTIONAL_ID
This config is available for TransactionalSpouts, and contains the id ( a String) for the transactional topology.
|
static String |
TOPOLOGY_TRANSFER_BATCH_SIZE
The size of the transfer queue for each worker.
|
static String |
TOPOLOGY_TRANSFER_BUFFER_SIZE
The size of the transfer queue for each worker.
|
static String |
TOPOLOGY_TRIDENT_BATCH_EMIT_INTERVAL_MILLIS
How often a batch can be emitted in a Trident topology.
|
static String |
TOPOLOGY_TRIDENT_WINDOWING_INMEMORY_CACHE_LIMIT
Maximum number of tuples that can be stored inmemory cache in windowing operators for fast access without fetching them from store.
|
static String |
TOPOLOGY_TUPLE_SERIALIZER
The serializer class for ListDelegate (tuple payload).
|
static String |
TOPOLOGY_USERS
A list of users that are allowed to interact with the topology.
|
static String |
TOPOLOGY_V2_METRICS_TICK_INTERVAL_SECONDS
Topology configuration to specify the V2 metrics tick interval in seconds.
|
static String |
TOPOLOGY_VERSION
User defined version of this topology.
|
static String |
TOPOLOGY_WORKER_CHILDOPTS
Topology-specific options for the worker child process.
|
static String |
TOPOLOGY_WORKER_GC_CHILDOPTS
Topology-specific options GC for the worker child process.
|
static String |
TOPOLOGY_WORKER_LOGWRITER_CHILDOPTS
Topology-specific options for the logwriter process of a worker.
|
static String |
TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB
A per topology config that specifies the maximum amount of memory a worker can use for that specific topology.
|
static String |
TOPOLOGY_WORKER_METRICS
A map of metric name to class name implementing IMetric that will be created once per worker JVM.
|
static String |
TOPOLOGY_WORKER_SHARED_THREAD_POOL_SIZE
The size of the shared thread pool for worker tasks to make use of.
|
static String |
TOPOLOGY_WORKER_TIMEOUT_SECS
Topology configurable worker heartbeat timeout before the supervisor tries to restart the worker process.
|
static String |
TOPOLOGY_WORKERS
How many processes should be spawned around the cluster to execute this topology.
|
static String |
TRANSACTIONAL_ZOOKEEPER_PORT
The port to use to connect to the transactional zookeeper servers.
|
static String |
TRANSACTIONAL_ZOOKEEPER_ROOT
The root directory in ZooKeeper for metadata about TransactionalSpouts.
|
static String |
TRANSACTIONAL_ZOOKEEPER_SERVERS
The list of zookeeper servers in which to keep the transactional state.
|
static String |
WORKER_BLOB_UPDATE_POLL_INTERVAL_SECS
Interval to check for the worker to check for updated blobs and refresh worker state accordingly.
|
static String |
WORKER_CHILDOPTS
The jvm opts provided to workers launched by this supervisor.
|
static String |
WORKER_GC_CHILDOPTS
The jvm opts provided to workers launched by this supervisor for GC.
|
static String |
WORKER_HEAP_MEMORY_MB
The default heap memory size in MB per worker, used in the jvm -Xmx opts for launching the worker.
|
static String |
WORKER_HEARTBEAT_FREQUENCY_SECS
How often this worker should heartbeat to the supervisor.
|
static String |
WORKER_LOG_LEVEL_RESET_POLL_SECS
How often a worker should check dynamic log level timeouts for expiration.
|
static String |
WORKER_MAX_TIMEOUT_SECS
Enforce maximum on
TOPOLOGY_WORKER_TIMEOUT_SECS . |
static String |
WORKER_METRICS
A map of metric name to class name implementing IMetric that will be created once per worker JVM.
|
Constructor and Description |
---|
Config() |
Modifier and Type | Method and Description |
---|---|
static String |
getBlobstoreHDFSPrincipal(Map conf)
Deprecated.
|
static String |
getHdfsKeytab(Map<String,Object> conf)
Get the hdfs keytab.
|
static String |
getHdfsPrincipal(Map<String,Object> conf)
Get the hostname substituted hdfs principal.
|
void |
registerDecorator(Class<? extends IKryoDecorator> klass) |
static void |
registerDecorator(Map<String,Object> conf,
Class<? extends IKryoDecorator> klass) |
void |
registerEventLogger(Class<? extends IEventLogger> klass) |
void |
registerEventLogger(Class<? extends IEventLogger> klass,
Map<String,Object> argument) |
static void |
registerEventLogger(Map<String,Object> conf,
Class<? extends IEventLogger> klass) |
static void |
registerEventLogger(Map<String,Object> conf,
Class<? extends IEventLogger> klass,
Map<String,Object> argument) |
void |
registerMetricsConsumer(Class klass) |
void |
registerMetricsConsumer(Class klass,
long parallelismHint) |
void |
registerMetricsConsumer(Class klass,
Object argument,
long parallelismHint) |
static void |
registerMetricsConsumer(Map<String,Object> conf,
Class klass) |
static void |
registerMetricsConsumer(Map<String,Object> conf,
Class klass,
long parallelismHint) |
static void |
registerMetricsConsumer(Map<String,Object> conf,
Class klass,
Object argument,
long parallelismHint) |
void |
registerSerialization(Class klass) |
void |
registerSerialization(Class klass,
Class<? extends com.esotericsoftware.kryo.Serializer> serializerClass) |
static void |
registerSerialization(Map<String,Object> conf,
Class klass) |
static void |
registerSerialization(Map<String,Object> conf,
Class klass,
Class<? extends com.esotericsoftware.kryo.Serializer> serializerClass) |
static void |
setClasspath(Map<String,Object> conf,
String cp) |
void |
setClasspath(String cp) |
void |
setDebug(boolean isOn) |
static void |
setDebug(Map<String,Object> conf,
boolean isOn) |
void |
setEnvironment(Map<String,Object> env) |
static void |
setEnvironment(Map<String,Object> conf,
Map<String,Object> env) |
void |
setFallBackOnJavaSerialization(boolean fallback) |
static void |
setFallBackOnJavaSerialization(Map<String,Object> conf,
boolean fallback) |
void |
setKryoFactory(Class<? extends IKryoFactory> klass) |
static void |
setKryoFactory(Map<String,Object> conf,
Class<? extends IKryoFactory> klass) |
void |
setMaxSpoutPending(int max) |
static void |
setMaxSpoutPending(Map<String,Object> conf,
int max) |
void |
setMaxTaskParallelism(int max) |
static void |
setMaxTaskParallelism(Map<String,Object> conf,
int max) |
void |
setMessageTimeoutSecs(int secs) |
static void |
setMessageTimeoutSecs(Map<String,Object> conf,
int secs) |
void |
setNumAckers(int numExecutors) |
static void |
setNumAckers(Map<String,Object> conf,
int numExecutors) |
void |
setNumEventLoggers(int numExecutors) |
static void |
setNumEventLoggers(Map<String,Object> conf,
int numExecutors) |
void |
setNumWorkers(int workers) |
static void |
setNumWorkers(Map<String,Object> conf,
int workers) |
void |
setSkipMissingKryoRegistrations(boolean skip) |
static void |
setSkipMissingKryoRegistrations(Map<String,Object> conf,
boolean skip) |
void |
setStatsSampleRate(double rate) |
static void |
setStatsSampleRate(Map<String,Object> conf,
double rate) |
void |
setTopologyComponentWorkerConstraints(String component1,
String component2)
Declares executors of component1 cannot be on the same worker as executors of component2.
|
void |
setTopologyConstraintsMaxStateSearch(int numStates)
Sets the maximum number of states that will be searched in the constraint solver strategy.
|
void |
setTopologyPriority(int priority)
Set the priority for a topology.
|
void |
setTopologyStrategy(String strategy) |
static void |
setTopologyVersion(Map<String,Object> conf,
String version) |
void |
setTopologyVersion(String version) |
void |
setTopologyWorkerMaxHeapSize(Number size)
Set the max heap size allow per worker for this topology.
|
clear, clone, compute, computeIfAbsent, computeIfPresent, containsKey, containsValue, entrySet, forEach, get, getOrDefault, isEmpty, keySet, merge, put, putAll, putIfAbsent, remove, remove, replace, replace, replaceAll, size, values
equals, hashCode, toString
public static final String TOPOLOGY_TUPLE_SERIALIZER
public static final String TOPOLOGY_DISABLE_LOADAWARE_MESSAGING
public static final String TOPOLOGY_LOCALITYAWARE_HIGHER_BOUND
public static final String TOPOLOGY_LOCALITYAWARE_LOWER_BOUND
public static final String TOPOLOGY_TESTING_ALWAYS_TRY_SERIALIZE
public static final String TOPOLOGY_BLOBSTORE_MAP
public static final String WORKER_LOG_LEVEL_RESET_POLL_SECS
public static final String TASK_CREDENTIALS_POLL_SECS
@Deprecated public static final String TOPOLOGY_BACKPRESSURE_ENABLE
public static final String TOPOLOGY_USERS
public static final String TOPOLOGY_GROUPS
public static final String TOPOLOGY_READONLY_USERS
public static final String TOPOLOGY_READONLY_GROUPS
public static final String TOPOLOGY_ENABLE_MESSAGE_TIMEOUTS
public static final String TOPOLOGY_DEBUG
public static final String TOPOLOGY_VERSION
public static final String TOPOLOGY_MULTILANG_LOG_HANDLER
ShellLogHandler
to handle output from non-JVM processes e.g.
"com.mycompany.CustomShellLogHandler". If not provided, org.apache.storm.utils.DefaultLogHandler will be used.public static final String TOPOLOGY_MULTILANG_SERIALIZER
public static final String TOPOLOGY_WORKERS
public static final String TOPOLOGY_TASKS
public static final String TOPOLOGY_COMPONENT_RESOURCES_MAP
public static final String TOPOLOGY_COMPONENT_RESOURCES_ONHEAP_MEMORY_MB
public static final String TOPOLOGY_COMPONENT_RESOURCES_OFFHEAP_MEMORY_MB
public static final String TOPOLOGY_COMPONENT_CPU_PCORE_PERCENT
public static final String TOPOLOGY_ACKER_RESOURCES_ONHEAP_MEMORY_MB
public static final String TOPOLOGY_ACKER_RESOURCES_OFFHEAP_MEMORY_MB
public static final String TOPOLOGY_ACKER_CPU_PCORE_PERCENT
public static final String TOPOLOGY_METRICS_CONSUMER_RESOURCES_ONHEAP_MEMORY_MB
public static final String TOPOLOGY_METRICS_CONSUMER_RESOURCES_OFFHEAP_MEMORY_MB
public static final String TOPOLOGY_METRICS_CONSUMER_CPU_PCORE_PERCENT
public static final String TOPOLOGY_ENABLE_V2_METRICS_TICK
public static final String TOPOLOGY_V2_METRICS_TICK_INTERVAL_SECONDS
public static final String TOPOLOGY_ENABLE_SEND_ICONNECTION_METRICS
public static final String TOPOLOGY_STATE_PROVIDER
StateProvider
implementation. If not specified defaults to InMemoryKeyValueStateProvider
. This can be overridden at the component level.public static final String TOPOLOGY_STATE_PROVIDER_CONFIG
StateProvider
implementation. This can be overridden at the
component level. The value and the interpretation of this config is based on the state provider implementation. For e.g. this could
be just a config file name which contains the config for the state provider implementation.public static final String TOPOLOGY_STATE_CHECKPOINT_INTERVAL
IStatefulBolt
bolts are involved.public static final String TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB
public static final String TOPOLOGY_SCHEDULER_STRATEGY
public static final String TOPOLOGY_RAS_ORDER_EXECUTORS_BY_PROXIMITY_NEEDS
public static final String TOPOLOGY_RAS_CONSTRAINTS
Old style Config.TOPOLOGY_RAS_CONSTRAINTS (ListOfListOfString) specified a list of components that cannot co-exist on the same Worker.
New style Config.TOPOLOGY_RAS_CONSTRAINTS is map where each component has a list of other incompatible components (which serves the same function as the old style configuration) and optional number that specifies the maximum co-location count for the component on a node.
comp-1 cannot exist on same worker as comp-2 or comp-3, and at most "2" comp-1 on same node
comp-2 and comp-4 cannot be on same worker (missing comp-1 is implied from comp-1 constraint)
{ "comp-1": { "maxNodeCoLocationCnt": 2, "incompatibleComponents": ["comp-2", "comp-3" ] }, "comp-2": { "incompatibleComponents": [ "comp-4" ] } }
@Deprecated public static final String TOPOLOGY_SPREAD_COMPONENTS
public static final String TOPOLOGY_RAS_CONSTRAINT_MAX_STATE_SEARCH
public static final String TOPOLOGY_RAS_ONE_EXECUTOR_PER_WORKER
public static final String TOPOLOGY_RAS_ONE_COMPONENT_PER_WORKER
public static final String TOPOLOGY_RAS_CONSTRAINT_MAX_TIME_SECS
public static final String TOPOLOGY_SCHEDULER_FAVORED_NODES
Favored nodes are moved to the front of the node selection list.
If the same node is also present in TOPOLOGY_SCHEDULER_UNFAVORED_NODES
then the node is considered only as a favored node and is removed from the unfavored list.
public static final String TOPOLOGY_SCHEDULER_UNFAVORED_NODES
Unfavored nodes are moved to the end of the node selection list.
If the same node is also present in TOPOLOGY_SCHEDULER_FAVORED_NODES
then the node is considered only as a favored node and is removed from the unfavored list.
public static final String TOPOLOGY_ACKER_EXECUTORS
1. If not setting this variable or setting it as null,
a. If RAS is not used:
Nimbus will set it to TOPOLOGY_WORKERS
.
b. If RAS is used:
Nimbus will set it to (the estimate number of workers * TOPOLOGY_RAS_ACKER_EXECUTORS_PER_WORKER
).
TOPOLOGY_RAS_ACKER_EXECUTORS_PER_WORKER
is default to be 1 if not set.
2. If this variable is set to 0,
then Storm will immediately ack tuples as soon as they come off the spout,
effectively disabling reliability.
3. If this variable is set to a positive integer,
Storm will not honor TOPOLOGY_RAS_ACKER_EXECUTORS_PER_WORKER
setting.
Instead, nimbus will set it as (this variable / estimate num of workers).
public static final String TOPOLOGY_RAS_ACKER_EXECUTORS_PER_WORKER
This setting is RAS specific.
If TOPOLOGY_ACKER_EXECUTORS
is not configured,
this setting will be used to calculate TOPOLOGY_ACKER_EXECUTORS
.
If TOPOLOGY_ACKER_EXECUTORS
is configured,
nimbus will ignore this and set it as (TOPOLOGY_ACKER_EXECUTORS
/ estimate num of workers).
public static final String TOPOLOGY_EVENT_LOGGER_REGISTER
Note that EventLoggerBolt takes care of all the implementations of IEventLogger, hence registering many implementations (especially they're implemented as 'blocking' manner) would slow down overall topology.
public static final String TOPOLOGY_EVENTLOGGER_EXECUTORS
By setting it as null, Storm will set the number of eventlogger executors to be equal to the number of workers configured for this topology (or the estimated number of workers if the Resource Aware Scheduler is used). If this variable is set to 0, event logging will be disabled.
public static final String TOPOLOGY_MESSAGE_TIMEOUT_SECS
public static final String TOPOLOGY_KRYO_REGISTER
See Kryo's documentation for more information about writing custom serializers.
public static final String TOPOLOGY_KRYO_DECORATORS
public static final String TOPOLOGY_KRYO_FACTORY
public static final String TOPOLOGY_SKIP_MISSING_KRYO_REGISTRATIONS
public static final String TOPOLOGY_STATE_KRYO_REGISTER
public static final String TOPOLOGY_METRICS_CONSUMER_REGISTER
public static final String TOPOLOGY_SERIALIZED_MESSAGE_SIZE_METRICS
public static final String TOPOLOGY_WORKER_METRICS
public static final String WORKER_METRICS
public static final String TOPOLOGY_MAX_TASK_PARALLELISM
public static final String TOPOLOGY_MAX_SPOUT_PENDING
A pending tuple is one that has been emitted from a spout but has not been acked or failed yet. Note that this config parameter has no effect for unreliable spouts that don't tag their tuples with a message id.
public static final String TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS
public static final String TOPOLOGY_STATE_SYNCHRONIZATION_TIMEOUT_SECS
public static final String TOPOLOGY_STATS_SAMPLE_RATE
public static final String TOPOLOGY_BUILTIN_METRICS_BUCKET_SIZE_SECS
public static final String TOPOLOGY_FALL_BACK_ON_JAVA_SERIALIZATION
public static final String TOPOLOGY_WORKER_CHILDOPTS
public static final String TOPOLOGY_WORKER_GC_CHILDOPTS
public static final String TOPOLOGY_WORKER_LOGWRITER_CHILDOPTS
public static final String TOPOLOGY_CLASSPATH
public static final String TOPOLOGY_CLASSPATH_BEGINNING
public static final String TOPOLOGY_ENVIRONMENT
public static final String TOPOLOGY_BOLTS_WINDOW_LENGTH_COUNT
public static final String TOPOLOGY_BOLTS_WINDOW_LENGTH_DURATION_MS
public static final String TOPOLOGY_BOLTS_SLIDING_INTERVAL_COUNT
public static final String TOPOLOGY_BOLTS_SLIDING_INTERVAL_DURATION_MS
public static final String TOPOLOGY_BOLTS_LATE_TUPLE_STREAM
public static final String TOPOLOGY_BOLTS_TUPLE_TIMESTAMP_MAX_LAG_MS
TimestampExtractor
is specified.public static final String TOPOLOGY_BOLTS_WATERMARK_EVENT_INTERVAL_MS
public static final String TOPOLOGY_BOLTS_MESSAGE_ID_FIELD_NAME
public static final String TOPOLOGY_TRANSACTIONAL_ID
public static final String TOPOLOGY_AUTO_TASK_HOOKS
public static final String TOPOLOGY_EXECUTOR_RECEIVE_BUFFER_SIZE
public static final String TOPOLOGY_TRANSFER_BUFFER_SIZE
public static final String TOPOLOGY_TRANSFER_BATCH_SIZE
public static final String TOPOLOGY_TICK_TUPLE_FREQ_SECS
public static final String TOPOLOGY_PRODUCER_BATCH_SIZE
public static final String TOPOLOGY_EXECUTOR_OVERFLOW_LIMIT
public static final String TOPOLOGY_BACKPRESSURE_CHECK_MILLIS
public static final String TOPOLOGY_BATCH_FLUSH_INTERVAL_MILLIS
public static final String TOPOLOGY_WORKER_SHARED_THREAD_POOL_SIZE
public static final String TOPOLOGY_ERROR_THROTTLE_INTERVAL_SECS
public static final String TOPOLOGY_MAX_ERROR_REPORT_PER_INTERVAL
TOPOLOGY_ERROR_THROTTLE_INTERVAL_SECS
.public static final String TOPOLOGY_TRIDENT_BATCH_EMIT_INTERVAL_MILLIS
public static final String TOPOLOGY_TRIDENT_WINDOWING_INMEMORY_CACHE_LIMIT
public static final String STORM_ID
public static final String TOPOLOGY_NAME
public static final String TOPOLOGY_SUBMITTER_PRINCIPAL
public static final String TOPOLOGY_SUBMITTER_USER
public static final String TOPOLOGY_AUTO_CREDENTIALS
public static final String TOPOLOGY_SHELLBOLT_MAX_PENDING
public static final String TOPOLOGY_SUBPROCESS_TIMEOUT_SECS
public static final String TOPOLOGY_LOGGING_SENSITIVITY
public static final String TOPOLOGY_LOGGING_CONFIG_FILE
public static final String TOPOLOGY_PRIORITY
public static final String TRANSACTIONAL_ZOOKEEPER_ROOT
public static final String TRANSACTIONAL_ZOOKEEPER_SERVERS
public static final String TRANSACTIONAL_ZOOKEEPER_PORT
public static final String STORM_DO_AS_USER
public static final String TOPOLOGY_ISOLATED_MACHINES
MultitenantScheduler
.
Round Robin Strategy uses this value to avoid spreading a topology too
thinly over a large number of machines - avoiding the the extreme case where the topology would be spread over
all workers and thus deny scheduling of other topologies. Round Robin scheduling will occupy all the workers on
this limited number of machines, forcing other topologies to be scheduled on other machines; thus isolating the
topology from other topologies.
Set storm.scheduler to MultitenantScheduler
Alternatively set storm.scheduler to ResourceAwareScheduler
using TOPOLOGY_SCHEDULER_STRATEGY
set to
RoundRobinResourceAwareStrategy
public static final String TOPOLOGY_SPOUT_WAIT_STRATEGY
1. nextTuple emits no tuples 2. The spout has hit maxSpoutPending and can't emit any more tuples
This class must implement IWaitStrategy
.
public static final String TOPOLOGY_SPOUT_WAIT_PARK_MICROSEC
public static final String TOPOLOGY_SPOUT_WAIT_PROGRESSIVE_LEVEL1_COUNT
public static final String TOPOLOGY_SPOUT_WAIT_PROGRESSIVE_LEVEL2_COUNT
public static final String TOPOLOGY_SPOUT_WAIT_PROGRESSIVE_LEVEL3_SLEEP_MILLIS
public static final String TOPOLOGY_BOLT_WAIT_STRATEGY
IWaitStrategy
.public static final String TOPOLOGY_BOLT_WAIT_PARK_MICROSEC
public static final String TOPOLOGY_BOLT_WAIT_PROGRESSIVE_LEVEL1_COUNT
public static final String TOPOLOGY_BOLT_WAIT_PROGRESSIVE_LEVEL2_COUNT
public static final String TOPOLOGY_BOLT_WAIT_PROGRESSIVE_LEVEL3_SLEEP_MILLIS
public static final String TOPOLOGY_BACKPRESSURE_WAIT_STRATEGY
1. nextTuple emits no tuples 2. The spout has hit maxSpoutPending and can't emit any more tuples
This class must implement IWaitStrategy
.
public static final String TOPOLOGY_BACKPRESSURE_WAIT_PARK_MICROSEC
public static final String TOPOLOGY_BACKPRESSURE_WAIT_PROGRESSIVE_LEVEL3_SLEEP_MILLIS
public static final String TOPOLOGY_BACKPRESSURE_WAIT_PROGRESSIVE_LEVEL1_COUNT
public static final String TOPOLOGY_BACKPRESSURE_WAIT_PROGRESSIVE_LEVEL2_COUNT
public static final String TOPOLOGY_SPOUT_RECVQ_SKIPS
public static final String TOPOLOGY_MIN_REPLICATION_COUNT
public static final String TOPOLOGY_MAX_REPLICATION_WAIT_TIME_SEC
public static final String PACEMAKER_SERVERS
public static final String PACEMAKER_PORT
public static final String PACEMAKER_CLIENT_MAX_THREADS
public static final String PACEMAKER_AUTH_METHOD
public static final String PACEMAKER_THRIFT_MESSAGE_SIZE_MAX
public static final String STORM_GROUP_MAPPING_SERVICE_CACHE_DURATION_SECS
public static final String DRPC_SERVERS
public static final String DRPC_INVOCATIONS_PORT
public static final String STORM_NIMBUS_RETRY_TIMES
public static final String STORM_NIMBUS_RETRY_INTERVAL
public static final String STORM_NIMBUS_RETRY_INTERVAL_CEILING
public static final String NIMBUS_THRIFT_TRANSPORT_PLUGIN
public static final String NIMBUS_THRIFT_PORT
public static final String NIMBUS_QUEUE_SIZE
public static final String NIMBUS_LOCAL_ASSIGNMENTS_BACKEND_CLASS
public static final String NIMBUS_THRIFT_THREADS
public static final String NIMBUS_THRIFT_MAX_BUFFER_SIZE
public static final String STORM_THRIFT_SOCKET_TIMEOUT_MS
public static final String DRPC_THRIFT_TRANSPORT_PLUGIN
public static final String DRPC_PORT
public static final String DRPC_QUEUE_SIZE
public static final String DRPC_WORKER_THREADS
public static final String DRPC_MAX_BUFFER_SIZE
public static final String DRPC_INVOCATIONS_THRIFT_TRANSPORT_PLUGIN
public static final String DRPC_INVOCATIONS_THREADS
public static final String STORM_GROUP_MAPPING_SERVICE_PARAMS
STORM_GROUP_MAPPING_SERVICE_PROVIDER_PLUGIN
implementation to access optional settings.public static final String STORM_THRIFT_TRANSPORT_PLUGIN
public static final String SUPERVISOR_WORKER_TIMEOUT_SECS
TOPOLOGY_WORKER_TIMEOUT_SECS
, if set.public static final String WORKER_MAX_TIMEOUT_SECS
TOPOLOGY_WORKER_TIMEOUT_SECS
.public static final String TOPOLOGY_WORKER_TIMEOUT_SECS
WORKER_MAX_TIMEOUT_SECS
.
When topology timeout is greater, the following configs are effectively overridden:
SUPERVISOR_WORKER_TIMEOUT_SECS
, SUPERVISOR_WORKER_START_TIMEOUT_SECS, NIMBUS_TASK_TIMEOUT_SECS and NIMBUS_TASK_LAUNCH_SECS.public static final String SUPERVISOR_WORKER_SHUTDOWN_SLEEP_SECS
public static final String STORM_ZOOKEEPER_SERVERS
public static final String STORM_ZOOKEEPER_PORT
public static final String STORM_ZOOKEEPER_SUPERACL
public static final String STORM_ZOOKEEPER_DRPC_ACL
Should be in the form 'scheme:acl' just like STORM_ZOOKEEPER_SUPERACL.
public static final String STORM_ZOOKEEPER_TOPOLOGY_AUTH_SCHEME
public static final String STORM_META_SERIALIZATION_DELEGATE
public static final String TOPOLOGY_METRICS_REPORTERS
public static final String STORM_TOPOLOGY_METRICS_SYSTEM_REPORTERS
@Deprecated public static final String STORM_METRICS_REPORTERS
TOPOLOGY_METRICS_REPORTERS
instead.public static final String CLIENT_BLOBSTORE
public static final String BLOBSTORE_DIR
public static final String BLOBSTORE_CLEANUP_ENABLE
@Deprecated public static final String BLOBSTORE_HDFS_PRINCIPAL
getBlobstoreHDFSPrincipal(Map conf)
method).@Deprecated public static final String BLOBSTORE_HDFS_KEYTAB
public static final String STORM_BLOBSTORE_REPLICATION_FACTOR
public static final String STORM_HDFS_LOGIN_PRINCIPAL
getHdfsPrincipal(java.util.Map<java.lang.String, java.lang.Object>)
method).public static final String STORM_HDFS_LOGIN_KEYTAB
public static final String STORM_LOCAL_HOSTNAME
InetAddress.getLocalHost().getCanonicalHostName()
.
You should set this config when you don't have a DNS which supervisors/workers can utilize to find each other
based on hostname got from calls to
InetAddress.getLocalHost().getCanonicalHostName()
.
public static final String NIMBUS_SEEDS
public static final String NIMBUS_USERS
public static final String NIMBUS_GROUPS
public static final String STORM_CLUSTER_MODE
public static final String STORM_ZOOKEEPER_ROOT
public static final String STORM_ZOOKEEPER_TOPOLOGY_AUTH_PAYLOAD
public static final String STORM_ZOOKEEPER_AUTH_SCHEME
public static final String STORM_ZOOKEEPER_AUTH_PAYLOAD
public static final String STORM_NETWORK_TOPOGRAPHY_PLUGIN
public static final String WORKER_GC_CHILDOPTS
public static final String WORKER_CHILDOPTS
public static final String WORKER_HEAP_MEMORY_MB
public static final String SUPERVISOR_MEMORY_CAPACITY_MB
public static final String SUPERVISOR_CPU_CAPACITY
public static final String SUPERVISOR_THRIFT_PORT
public static final String SUPERVISOR_THRIFT_TRANSPORT_PLUGIN
public static final String SUPERVISOR_QUEUE_SIZE
public static final String SUPERVISOR_THRIFT_THREADS
public static final String SUPERVISOR_THRIFT_MAX_BUFFER_SIZE
public static final String SUPERVISOR_THRIFT_SOCKET_TIMEOUT_MS
public static final String SUPERVISOR_RESOURCES_MAP
public static final String STORM_LOCAL_MODE_ZMQ
Defaults to false.
public static final String STORM_MESSAGING_TRANSPORT
public static final String STORM_MESSAGING_NETTY_AUTHENTICATION
public static final String STORM_MESSAGING_NETTY_BUFFER_SIZE
public static final String STORM_MESSAGING_NETTY_BUFFER_HIGH_WATERMARK
If the number of bytes queued in the netty's write buffer exceeds this value, the netty Channel.isWritable()
will start to
return false
. The client will wait until the value falls below the low water mark.
public static final String STORM_MESSAGING_NETTY_BUFFER_LOW_WATERMARK
Once the number of bytes queued in the write buffer exceeded the high water
mark and then dropped down below this value, the netty Channel.isWritable()
will start to return true.
public static final String STORM_MESSAGING_NETTY_SOCKET_BACKLOG
public static final String STORM_MESSAGING_NETTY_SERVER_WORKER_THREADS
public static final String STORM_NETTY_MESSAGE_BATCH_SIZE
public static final String STORM_MESSAGING_NETTY_MIN_SLEEP_MS
public static final String STORM_MESSAGING_NETTY_MAX_SLEEP_MS
public static final String STORM_MESSAGING_NETTY_CLIENT_WORKER_THREADS
public static final String SUPERVISOR_RUN_WORKER_AS_USER
public static final String SUPERVISOR_WORKER_HEARTBEATS_MAX_TIMEOUT_SECS
public static final String DISABLE_SYMLINKS
public static final String STORM_PRINCIPAL_TO_LOCAL_PLUGIN
public static final String STORM_GROUP_MAPPING_SERVICE_PROVIDER_PLUGIN
public static final String NIMBUS_CREDENTIAL_RENEWERS
public static final String NIMBUS_AUTO_CRED_PLUGINS
public static final String NIMBUS_SUPERVISOR_USERS
public static final String NIMBUS_DAEMON_USERS
public static final String NIMBUS_ADMINS
public static final String NIMBUS_ADMINS_GROUPS
public static final String STORM_BLOBSTORE_ACL_VALIDATION_ENABLED
public static final String STORM_BLOBSTORE_INPUTSTREAM_BUFFER_SIZE_BYTES
public static final String STORM_BLOBSTORE_DEPENDENCY_JAR_UPLOAD_CHUNK_SIZE_BYTES
public static final String STORM_TOPOLOGY_SUBMISSION_NOTIFIER_PLUGIN
ISubmitterHook
@see ISubmitterHook for details.public static final String NIMBUS_IMPERSONATION_ACL
public static final String NIMBUS_SCHEDULER_STRATEGY_CLASS_WHITELIST
public static final String SUPERVISOR_WORKER_LAUNCHER
public static final String SUPERVISOR_WORKER_VERSION_CLASSPATH_MAP
Only the major and minor version numbers are used, although this may change in the future. The code will first try to find a version that is the same or higher than the requested version, but with the same major version number. If it cannot it will fall back to using one with a lower minor version, but in some cases this might fail as some features may be missing.
Because of how this selection process works please don't include two releases with the same major and minor versions as it is undefined which will be selected. Also it is good practice to just include one release for each major version you want to support unless the minor versions are truly not compatible with each other. This is to avoid maintenance and testing overhead.
This config needs to be set on all supervisors and on nimbus. In general this can be the output of calling storm classpath on the version you want and adding in an entry for the config directory for that release. You should modify the storm.yaml of each of these versions to match the features and settings you want on the main version.
public static final String SUPERVISOR_WORKER_VERSION_MAIN_MAP
public static final String SUPERVISOR_WORKER_VERSION_LOGWRITER_MAP
public static final String SUPERVISOR_WORKER_DEFAULT_VERSION
public static final String STORM_LOCAL_DIR
public static final String STORM_WORKERS_ARTIFACTS_DIR
public static final String STORM_EXHIBITOR_SERVERS
public static final String STORM_EXHIBITOR_PORT
public static final String STORM_EXHIBITOR_URIPATH
public static final String STORM_EXHIBITOR_POLL
public static final String STORM_EXHIBITOR_RETRY_TIMES
public static final String STORM_EXHIBITOR_RETRY_INTERVAL
public static final String STORM_EXHIBITOR_RETRY_INTERVAL_CEILING
public static final String STORM_ZOOKEEPER_CONNECTION_TIMEOUT
public static final String STORM_ZOOKEEPER_SESSION_TIMEOUT
public static final String STORM_ZOOKEEPER_RETRY_INTERVAL
public static final String STORM_ZOOKEEPER_RETRY_INTERVAL_CEILING
public static final String STORM_ZOOKEEPER_RETRY_TIMES
public static final String STORM_CLUSTER_STATE_STORE
public static final String WORKER_HEARTBEAT_FREQUENCY_SECS
public static final String EXECUTOR_METRICS_FREQUENCY_SECS
public static final String TASK_HEARTBEAT_FREQUENCY_SECS
EXECUTOR_METRICS_FREQUENCY_SECS
.public static final String TASK_REFRESH_POLL_SECS
public static final String DRPC_AUTHORIZER_ACL
DRPCSimpleACLAuthorizer
,
Constant Field Valuespublic static final String DRPC_AUTHORIZER_ACL_FILENAME
DRPCSimpleACLAuthorizer
,
Constant Field Valuespublic static final String DRPC_AUTHORIZER_ACL_STRICT
DRPCSimpleACLAuthorizer
,
Constant Field Valuespublic static final String STORM_CGROUP_HIERARCHY_DIR
public static final String NUM_STAT_BUCKETS
public static String STORM_OCI_CGROUP_ROOT
public static String TOPOLOGY_OCI_IMAGE
public static final String WORKER_BLOB_UPDATE_POLL_INTERVAL_SECS
public static final String STORM_DAEMON_METRICS_REPORTER_PLUGIN_LOCALE
public static final String STORM_DAEMON_METRICS_REPORTER_PLUGIN_RATE_UNIT
public static final String STORM_DAEMON_METRICS_REPORTER_PLUGIN_DURATION_UNIT
public static void registerSerialization(Map<String,Object> conf, Class klass)
public static void registerSerialization(Map<String,Object> conf, Class klass, Class<? extends com.esotericsoftware.kryo.Serializer> serializerClass)
public static void registerEventLogger(Map<String,Object> conf, Class<? extends IEventLogger> klass, Map<String,Object> argument)
public static void registerEventLogger(Map<String,Object> conf, Class<? extends IEventLogger> klass)
public static void registerMetricsConsumer(Map<String,Object> conf, Class klass, Object argument, long parallelismHint)
public static void registerMetricsConsumer(Map<String,Object> conf, Class klass, long parallelismHint)
public static void registerMetricsConsumer(Map<String,Object> conf, Class klass)
public static void registerDecorator(Map<String,Object> conf, Class<? extends IKryoDecorator> klass)
public static void setKryoFactory(Map<String,Object> conf, Class<? extends IKryoFactory> klass)
public static void setSkipMissingKryoRegistrations(Map<String,Object> conf, boolean skip)
public static void setFallBackOnJavaSerialization(Map<String,Object> conf, boolean fallback)
public void setClasspath(String cp)
public void setDebug(boolean isOn)
public void setTopologyVersion(String version)
public void setNumWorkers(int workers)
public void setNumAckers(int numExecutors)
public void setNumEventLoggers(int numExecutors)
public void setMessageTimeoutSecs(int secs)
public void registerSerialization(Class klass)
public void registerSerialization(Class klass, Class<? extends com.esotericsoftware.kryo.Serializer> serializerClass)
public void registerEventLogger(Class<? extends IEventLogger> klass, Map<String,Object> argument)
public void registerEventLogger(Class<? extends IEventLogger> klass)
public void registerMetricsConsumer(Class klass, Object argument, long parallelismHint)
public void registerMetricsConsumer(Class klass, long parallelismHint)
public void registerMetricsConsumer(Class klass)
public void registerDecorator(Class<? extends IKryoDecorator> klass)
public void setKryoFactory(Class<? extends IKryoFactory> klass)
public void setSkipMissingKryoRegistrations(boolean skip)
public void setMaxTaskParallelism(int max)
public void setMaxSpoutPending(int max)
public void setStatsSampleRate(double rate)
public void setFallBackOnJavaSerialization(boolean fallback)
public void setTopologyWorkerMaxHeapSize(Number size)
size
- the maximum heap size for a worker.public void setTopologyComponentWorkerConstraints(String component1, String component2)
component1
- a component that should not coexist with component2component2
- a component that should not coexist with component1public void setTopologyConstraintsMaxStateSearch(int numStates)
numStates
- maximum number of stats to search.public void setTopologyPriority(int priority)
public void setTopologyStrategy(String strategy)
@Deprecated public static String getBlobstoreHDFSPrincipal(Map conf) throws UnknownHostException
UnknownHostException
public static String getHdfsPrincipal(Map<String,Object> conf) throws UnknownHostException
conf
- the storm ConfigurationUnknownHostException
- on UnknowHostExceptionCopyright © 2023 The Apache Software Foundation. All rights reserved.