Package | Description |
---|---|
org.apache.storm | |
org.apache.storm.hdfs.spout |
Modifier and Type | Field and Description |
---|---|
static String |
DaemonConfig.BLACKLIST_SCHEDULER_RESUME_TIME
The number of seconds that the blacklisted slots or supervisor will be resumed.
|
static String |
DaemonConfig.BLACKLIST_SCHEDULER_TOLERANCE_COUNT
The number of hit count that will trigger blacklist in tolerance time.
|
static String |
DaemonConfig.BLACKLIST_SCHEDULER_TOLERANCE_TIME
The number of seconds that the blacklist scheduler will concern of bad slots or supervisors.
|
static String |
Config.DRPC_INVOCATIONS_PORT
This port on Storm DRPC is used by DRPC topologies to receive function invocations and send results back.
|
static String |
Config.DRPC_INVOCATIONS_THREADS
DRPC invocations thrift server worker threads.
|
static String |
Config.DRPC_MAX_BUFFER_SIZE
The maximum buffer size thrift should use when reading messages for DRPC.
|
static String |
Config.DRPC_PORT
This port is used by Storm DRPC for receiving DPRC requests from clients.
|
static String |
Config.DRPC_QUEUE_SIZE
DRPC thrift server queue size.
|
static String |
DaemonConfig.DRPC_REQUEST_TIMEOUT_SECS
The timeout on DRPC requests within the DRPC server.
|
static String |
Config.DRPC_WORKER_THREADS
DRPC thrift server worker threads.
|
static String |
Config.EXECUTOR_METRICS_FREQUENCY_SECS
How often executor metrics should report to master, used for RPC heartbeat mode.
|
static String |
DaemonConfig.LOGVIEWER_CLEANUP_AGE_MINS
How many minutes since a log was last modified for the log to be considered for clean-up.
|
static String |
DaemonConfig.LOGVIEWER_CLEANUP_INTERVAL_SECS
How often to clean up old log files.
|
static String |
DaemonConfig.LOGVIEWER_HTTPS_PORT
Storm Logviewer HTTPS port.
|
static String |
DaemonConfig.LOGVIEWER_MAX_PER_WORKER_LOGS_SIZE_MB
The maximum number of bytes per worker’s files can take up in MB.
|
static String |
DaemonConfig.LOGVIEWER_MAX_SUM_WORKER_LOGS_SIZE_MB
The maximum number of bytes all worker log files can take up in MB.
|
static String |
DaemonConfig.LOGVIEWER_PORT
HTTP UI port for log viewer.
|
static String |
DaemonConfig.NIMBUS_BLOBSTORE_EXPIRATION_SECS
During operations with the blob store, via master, how long a connection is idle before nimbus considers it dead and drops the session and any associated connections.
|
static String |
DaemonConfig.NIMBUS_CLEANUP_INBOX_FREQ_SECS
How often nimbus should wake the cleanup thread to clean the inbox.
|
static String |
DaemonConfig.NIMBUS_CREDENTIAL_RENEW_FREQ_SECS
How often nimbus should wake up to renew credentials if needed.
|
static String |
DaemonConfig.NIMBUS_EXECUTORS_PER_TOPOLOGY
A number representing the maximum number of executors any single topology can acquire.
|
static String |
DaemonConfig.NIMBUS_MONITOR_FREQ_SECS
How often nimbus should wake up to check heartbeats and do reassignments.
|
static String |
Config.NIMBUS_QUEUE_SIZE
Nimbus thrift server queue size, default is 100000.
|
static String |
DaemonConfig.NIMBUS_SLOTS_PER_TOPOLOGY
A number representing the maximum number of workers any single topology can acquire.
|
static String |
DaemonConfig.NIMBUS_SUPERVISOR_TIMEOUT_SECS
How long before a supervisor can go without heartbeating before nimbus considers it dead and stops assigning new work to it.
|
static String |
DaemonConfig.NIMBUS_TASK_LAUNCH_SECS
A special timeout used when a task is initially launched.
|
static String |
DaemonConfig.NIMBUS_TASK_TIMEOUT_SECS
How long without heartbeating a task can go before nimbus will consider the task dead and reassign it to another location.
|
static String |
Config.NIMBUS_THRIFT_MAX_BUFFER_SIZE
The maximum buffer size thrift should use when reading messages.
|
static String |
Config.NIMBUS_THRIFT_PORT
Which port the Thrift interface of Nimbus should run on.
|
static String |
Config.NIMBUS_THRIFT_THREADS
The number of threads that should be used by the nimbus thrift server.
|
static String |
Config.NUM_STAT_BUCKETS
The number of Buckets.
|
static String |
Config.PACEMAKER_CLIENT_MAX_THREADS
The maximum number of threads that should be used by the Pacemaker client.
|
static String |
DaemonConfig.PACEMAKER_MAX_THREADS
The maximum number of threads that should be used by the Pacemaker.
|
static String |
Config.PACEMAKER_PORT
The port Pacemaker should run on.
|
static String |
Config.PACEMAKER_THRIFT_MESSAGE_SIZE_MAX
Pacemaker Thrift Max Message Size (bytes).
|
static String |
DaemonConfig.RESOURCE_AWARE_SCHEDULER_MAX_STATE_SEARCH |
static String |
DaemonConfig.RESOURCE_AWARE_SCHEDULER_MAX_TOPOLOGY_SCHEDULING_ATTEMPTS
The maximum number of times that the RAS will attempt to schedule a topology.
|
static String |
DaemonConfig.SCHEDULER_CONFIG_CACHE_EXPIRATION_SECS
How long before a scheduler considers its config cache expired.
|
static String |
DaemonConfig.SCHEDULER_CONFIG_LOADER_POLLTIME_SECS
It is the frequency at which the plugin will call out to artifactory instead of returning the most recently cached result.
|
static String |
DaemonConfig.SCHEDULER_CONFIG_LOADER_TIMEOUT_SECS
It is the amount of time an http connection to the artifactory server will wait before timing out.
|
static String |
DaemonConfig.SCHEDULING_TIMEOUT_SECONDS_PER_TOPOLOGY
Max time to attempt to schedule one topology.
|
static String |
Config.STORM_BLOBSTORE_DEPENDENCY_JAR_UPLOAD_CHUNK_SIZE_BYTES
What chunk size to use for storm client to upload dependency jars.
|
static String |
Config.STORM_BLOBSTORE_INPUTSTREAM_BUFFER_SIZE_BYTES
What buffer size to use for the blobstore uploads.
|
static String |
Config.STORM_BLOBSTORE_REPLICATION_FACTOR
Set replication factor for a blob in HDFS Blobstore Implementation.
|
static String |
DaemonConfig.STORM_CGROUP_MEMORY_LIMIT_TOLERANCE_MARGIN_MB
Please use STORM_SUPERVISOR_MEMORY_LIMIT_TOLERANCE_MARGIN_MB instead.
|
static String |
DaemonConfig.STORM_CLUSTER_METRICS_CONSUMER_PUBLISH_INTERVAL_SECS
How often cluster metrics data is published to metrics consumer.
|
static String |
Config.STORM_EXHIBITOR_PORT
The port Storm will use to connect to each of the exhibitor servers.
|
static String |
Config.STORM_MESSAGING_NETTY_BUFFER_HIGH_WATERMARK
Netty based messaging: The netty write buffer high watermark in bytes.
|
static String |
Config.STORM_MESSAGING_NETTY_BUFFER_LOW_WATERMARK
Netty based messaging: The netty write buffer low watermark in bytes.
|
static String |
Config.STORM_MESSAGING_NETTY_BUFFER_SIZE
Netty based messaging: The buffer size for send/recv buffer.
|
static String |
Config.STORM_MESSAGING_NETTY_MAX_SLEEP_MS
Netty based messaging: The max # of milliseconds that a peer will wait.
|
static String |
Config.STORM_MESSAGING_NETTY_MIN_SLEEP_MS
Netty based messaging: The min # of milliseconds that a peer will wait.
|
static String |
Config.STORM_MESSAGING_NETTY_SERVER_WORKER_THREADS
Netty based messaging: The # of worker threads for the server.
|
static String |
Config.STORM_MESSAGING_NETTY_SOCKET_BACKLOG
Netty based messaging: Sets the backlog value to specify when the channel binds to a local address.
|
static String |
DaemonConfig.STORM_SUPERVISOR_HARD_LIMIT_MEMORY_OVERAGE_MB
If the memory usage of a worker goes over its limit by this value is it shot immediately.
|
static String |
DaemonConfig.STORM_SUPERVISOR_HARD_MEMORY_LIMIT_MULTIPLIER
A multiplier for the memory limit of a worker that will have the supervisor shoot it immediately.
|
static String |
DaemonConfig.STORM_SUPERVISOR_LOW_MEMORY_THRESHOLD_MB
If the amount of memory that is free in the system (either on the box or in the supervisor’s cgroup) is below this number (in MB) consider the system to be in low memory mode and start shooting workers if they are over their limit.
|
static String |
DaemonConfig.STORM_SUPERVISOR_MEDIUM_MEMORY_GRACE_PERIOD_MS
The number of milliseconds that a worker is allowed to be over their limit when there is a medium amount of memory free in the system.
|
static String |
DaemonConfig.STORM_SUPERVISOR_MEDIUM_MEMORY_THRESHOLD_MB
If the amount of memory that is free in the system (either on the box or in the supervisor’s cgroup) is below this number (in MB) consider the system to be a little low on memory and start shooting workers if they are over their limit for a given grace period STORM_SUPERVISOR_MEDIUM_MEMORY_GRACE_PERIOD_MS.
|
static String |
DaemonConfig.STORM_SUPERVISOR_MEMORY_LIMIT_TOLERANCE_MARGIN_MB
Memory given to each worker for free (because java and storm have some overhead).
|
static String |
DaemonConfig.STORM_WORKER_CGROUP_CPU_LIMIT
the manually set cpu share for each CGroup on supervisor node.
|
static String |
DaemonConfig.STORM_WORKER_CGROUP_MEMORY_MB_LIMIT
the manually set memory limit (in MB) for each CGroup on supervisor node.
|
static String |
DaemonConfig.STORM_WORKER_MIN_CPU_PCORE_PERCENT
The config indicates the minimum percentage of cpu for a core that a worker will use.
|
static String |
DaemonConfig.STORM_WORKER_TOKEN_LIFE_TIME_HOURS
The number of hours a worker token is valid for.
|
static String |
Config.STORM_ZOOKEEPER_PORT
The port Storm will use to connect to each of the ZooKeeper servers.
|
static String |
DaemonConfig.SUPERVISOR_BLOBSTORE_DOWNLOAD_MAX_RETRIES
Maximum number of retries a supervisor is allowed to make for downloading a blob.
|
static String |
DaemonConfig.SUPERVISOR_BLOBSTORE_DOWNLOAD_THREAD_COUNT
What blobstore download parallelism the supervisor should use.
|
static String |
Config.SUPERVISOR_CPU_CAPACITY
The total amount of CPU resources a supervisor is allowed to give to its workers.
|
static String |
DaemonConfig.SUPERVISOR_LOCALIZER_CACHE_CLEANUP_INTERVAL_MS
The distributed cache cleanup interval.
|
static String |
DaemonConfig.SUPERVISOR_LOCALIZER_CACHE_TARGET_SIZE_MB
The distributed cache target size in MB.
|
static String |
DaemonConfig.SUPERVISOR_LOCALIZER_UPDATE_BLOB_INTERVAL_SECS
The distributed cache interval for checking for blobs to update.
|
static String |
Config.SUPERVISOR_MEMORY_CAPACITY_MB
The total amount of memory (in MiB) a supervisor is allowed to give to its workers.
|
static String |
DaemonConfig.SUPERVISOR_MONITOR_FREQUENCY_SECS
How often the supervisor checks the worker heartbeats to see if any of them need to be restarted.
|
static String |
Config.SUPERVISOR_QUEUE_SIZE |
static String |
Config.SUPERVISOR_THRIFT_MAX_BUFFER_SIZE |
static String |
Config.SUPERVISOR_THRIFT_PORT |
static String |
Config.SUPERVISOR_THRIFT_THREADS |
static String |
Config.SUPERVISOR_WORKER_SHUTDOWN_SLEEP_SECS
How many seconds to allow for graceful worker shutdown when killing workers before resorting to force kill.
|
static String |
DaemonConfig.SUPERVISOR_WORKER_START_TIMEOUT_SECS
How long a worker can go without heartbeating during the initial launch before the supervisor tries to restart the worker process.
|
static String |
Config.SUPERVISOR_WORKER_TIMEOUT_SECS
How long a worker can go without heartbeating before the supervisor tries to restart the worker process.
|
static String |
Config.TASK_CREDENTIALS_POLL_SECS
How often a task should sync credentials, worst case.
|
static String |
Config.TASK_HEARTBEAT_FREQUENCY_SECS
How often a task should heartbeat its status to the Pacamker.
|
static String |
Config.TASK_REFRESH_POLL_SECS
How often a task should sync its connections with other tasks (if a task is reassigned, the other tasks sending messages to it need to refresh their connections).
|
static String |
Config.TOPOLOGY_ACKER_CPU_PCORE_PERCENT
The config indicates the percentage of cpu for a core an instance(executor) of an acker will use.
|
static String |
Config.TOPOLOGY_ACKER_EXECUTORS
How many executors to spawn for ackers.
|
static String |
Config.TOPOLOGY_ACKER_RESOURCES_OFFHEAP_MEMORY_MB
The maximum amount of memory an instance of an acker will take off heap.
|
static String |
Config.TOPOLOGY_ACKER_RESOURCES_ONHEAP_MEMORY_MB
The maximum amount of memory an instance of an acker will take on heap.
|
static String |
Config.TOPOLOGY_BACKPRESSURE_CHECK_MILLIS
How often a worker should check and notify upstream workers about its tasks that are no longer experiencing BP and able to receive new messages.
|
static String |
Config.TOPOLOGY_BACKPRESSURE_WAIT_PARK_MICROSEC
Configures park time if using WaitStrategyPark for BackPressure.
|
static String |
Config.TOPOLOGY_BACKPRESSURE_WAIT_PROGRESSIVE_LEVEL1_COUNT
Configures steps used to determine progression to the next level of wait ..
|
static String |
Config.TOPOLOGY_BACKPRESSURE_WAIT_PROGRESSIVE_LEVEL2_COUNT
Configures steps used to determine progression to the next level of wait ..
|
static String |
Config.TOPOLOGY_BACKPRESSURE_WAIT_PROGRESSIVE_LEVEL3_SLEEP_MILLIS
Configures sleep time if using WaitStrategyProgressive for BackPressure.
|
static String |
Config.TOPOLOGY_BATCH_FLUSH_INTERVAL_MILLIS
How often to send flush tuple to the executors for flushing out batched events.
|
static String |
Config.TOPOLOGY_BOLT_WAIT_PARK_MICROSEC
Configures park time for WaitStrategyPark.
|
static String |
Config.TOPOLOGY_BOLT_WAIT_PROGRESSIVE_LEVEL1_COUNT
Configures number of iterations to spend in level 1 of WaitStrategyProgressive, before progressing to level 2.
|
static String |
Config.TOPOLOGY_BOLT_WAIT_PROGRESSIVE_LEVEL2_COUNT
Configures number of iterations to spend in level 2 of WaitStrategyProgressive, before progressing to level 3.
|
static String |
Config.TOPOLOGY_BOLT_WAIT_PROGRESSIVE_LEVEL3_SLEEP_MILLIS
Configures sleep time for WaitStrategyProgressive.
|
static String |
Config.TOPOLOGY_BOLTS_SLIDING_INTERVAL_COUNT |
static String |
Config.TOPOLOGY_BOLTS_SLIDING_INTERVAL_DURATION_MS |
static String |
Config.TOPOLOGY_BOLTS_TUPLE_TIMESTAMP_MAX_LAG_MS
Bolt-specific configuration for windowed bolts to specify the maximum time lag of the tuple timestamp in milliseconds.
|
static String |
Config.TOPOLOGY_BOLTS_WATERMARK_EVENT_INTERVAL_MS |
static String |
Config.TOPOLOGY_BOLTS_WINDOW_LENGTH_COUNT |
static String |
Config.TOPOLOGY_BOLTS_WINDOW_LENGTH_DURATION_MS |
static String |
Config.TOPOLOGY_COMPONENT_CPU_PCORE_PERCENT
The config indicates the percentage of cpu for a core an instance(executor) of a component will use.
|
static String |
Config.TOPOLOGY_COMPONENT_RESOURCES_OFFHEAP_MEMORY_MB
The maximum amount of memory an instance of a spout/bolt will take off heap.
|
static String |
Config.TOPOLOGY_COMPONENT_RESOURCES_ONHEAP_MEMORY_MB
The maximum amount of memory an instance of a spout/bolt will take on heap.
|
static String |
Config.TOPOLOGY_EVENTLOGGER_EXECUTORS
How many executors to spawn for event logger.
|
static String |
Config.TOPOLOGY_EXECUTOR_OVERFLOW_LIMIT
If number of items in task’s overflowQ exceeds this, new messages coming from other workers to this task will be dropped This prevents OutOfMemoryException that can occur in rare scenarios in the presence of BackPressure.
|
static String |
Config.TOPOLOGY_EXECUTOR_RECEIVE_BUFFER_SIZE
The size of the receive queue for each executor.
|
static String |
Config.TOPOLOGY_ISOLATED_MACHINES
The number of machines that should be used by this topology to isolate it from all others.
|
static String |
Config.TOPOLOGY_LOCALITYAWARE_HIGHER_BOUND
This signifies the load congestion among target tasks in scope.
|
static String |
Config.TOPOLOGY_LOCALITYAWARE_LOWER_BOUND
This signifies the load congestion among target tasks in scope.
|
static String |
Config.TOPOLOGY_MAX_ERROR_REPORT_PER_INTERVAL
See doc for
Config.TOPOLOGY_ERROR_THROTTLE_INTERVAL_SECS . |
static String |
Config.TOPOLOGY_MAX_SPOUT_PENDING
The maximum number of tuples that can be pending on a spout task at any given time.
|
static String |
Config.TOPOLOGY_MAX_TASK_PARALLELISM
The maximum parallelism allowed for a component in this topology.
|
static String |
Config.TOPOLOGY_MESSAGE_TIMEOUT_SECS
The maximum amount of time given to the topology to fully process a message emitted by a spout.
|
static String |
Config.TOPOLOGY_METRICS_CONSUMER_CPU_PCORE_PERCENT
The config indicates the percentage of cpu for a core an instance(executor) of a metrics consumer will use.
|
static String |
Config.TOPOLOGY_METRICS_CONSUMER_RESOURCES_OFFHEAP_MEMORY_MB
The maximum amount of memory an instance of a metrics consumer will take off heap.
|
static String |
Config.TOPOLOGY_METRICS_CONSUMER_RESOURCES_ONHEAP_MEMORY_MB
The maximum amount of memory an instance of a metrics consumer will take on heap.
|
static String |
Config.TOPOLOGY_PRIORITY
Sets the priority for a topology.
|
static String |
Config.TOPOLOGY_PRODUCER_BATCH_SIZE
The number of tuples to batch before sending to the destination executor.
|
static String |
Config.TOPOLOGY_RAS_ACKER_EXECUTORS_PER_WORKER
How many ackers to put in when launching a new worker until we run out of ackers.
|
static String |
Config.TOPOLOGY_RAS_CONSTRAINT_MAX_STATE_SEARCH
The maximum number of states that will be searched looking for a solution in resource aware strategies, e.g.
|
static String |
Config.TOPOLOGY_RAS_CONSTRAINT_MAX_TIME_SECS
The maximum number of seconds to spend scheduling a topology using resource aware strategies, e.g.
|
static String |
Config.TOPOLOGY_SHELLBOLT_MAX_PENDING
Max pending tuples in one ShellBolt.
|
static String |
Config.TOPOLOGY_SLEEP_SPOUT_WAIT_STRATEGY_TIME_MS
The amount of milliseconds the SleepEmptyEmitStrategy should sleep for.
|
static String |
Config.TOPOLOGY_SPOUT_RECVQ_SKIPS
Check recvQ after every N invocations of Spout’s nextTuple() [when ACKing is disabled].
|
static String |
Config.TOPOLOGY_SPOUT_WAIT_PARK_MICROSEC
Configures park time for WaitStrategyPark for spout.
|
static String |
Config.TOPOLOGY_SPOUT_WAIT_PROGRESSIVE_LEVEL1_COUNT
Configures number of iterations to spend in level 1 of WaitStrategyProgressive, before progressing to level 2.
|
static String |
Config.TOPOLOGY_SPOUT_WAIT_PROGRESSIVE_LEVEL2_COUNT
Configures number of iterations to spend in level 2 of WaitStrategyProgressive, before progressing to level 3.
|
static String |
Config.TOPOLOGY_SPOUT_WAIT_PROGRESSIVE_LEVEL3_SLEEP_MILLIS
Configures sleep time for WaitStrategyProgressive.
|
static String |
Config.TOPOLOGY_STATE_CHECKPOINT_INTERVAL
Topology configuration to specify the checkpoint interval (in millis) at which the topology state is saved when
IStatefulBolt bolts are involved. |
static String |
Config.TOPOLOGY_STATE_SYNCHRONIZATION_TIMEOUT_SECS
The maximum amount of time a component gives a source of state to synchronize before it requests synchronization again.
|
static String |
Config.TOPOLOGY_STATS_SAMPLE_RATE
The percentage of tuples to sample to produce stats for a task.
|
static String |
Config.TOPOLOGY_SUBPROCESS_TIMEOUT_SECS
How long a subprocess can go without heartbeating before the ShellSpout/ShellBolt tries to suicide itself.
|
static String |
Config.TOPOLOGY_TASKS
How many instances to create for a spout/bolt.
|
static String |
Config.TOPOLOGY_TRANSFER_BATCH_SIZE
The size of the transfer queue for each worker.
|
static String |
Config.TOPOLOGY_TRANSFER_BUFFER_SIZE
The size of the transfer queue for each worker.
|
static String |
Config.TOPOLOGY_TRIDENT_BATCH_EMIT_INTERVAL_MILLIS
How often a batch can be emitted in a Trident topology.
|
static String |
Config.TOPOLOGY_TRIDENT_WINDOWING_INMEMORY_CACHE_LIMIT
Maximum number of tuples that can be stored inmemory cache in windowing operators for fast access without fetching them from store.
|
static String |
Config.TOPOLOGY_V2_METRICS_TICK_INTERVAL_SECONDS
Topology configuration to specify the V2 metrics tick interval in seconds.
|
static String |
Config.TOPOLOGY_WORKER_MAX_HEAP_SIZE_MB
A per topology config that specifies the maximum amount of memory a worker can use for that specific topology.
|
static String |
Config.TOPOLOGY_WORKER_TIMEOUT_SECS
Topology configurable worker heartbeat timeout before the supervisor tries to restart the worker process.
|
static String |
Config.TOPOLOGY_WORKERS
How many processes should be spawned around the cluster to execute this topology.
|
static String |
Config.TRANSACTIONAL_ZOOKEEPER_PORT
The port to use to connect to the transactional zookeeper servers.
|
static String |
DaemonConfig.UI_HEADER_BUFFER_BYTES
The size of the header buffer for the UI in bytes.
|
static String |
DaemonConfig.UI_HTTPS_PORT
This port is used by Storm UI for receiving HTTPS (SSL) requests from clients.
|
static String |
DaemonConfig.UI_PORT
Storm UI binds to this port.
|
static String |
Config.WORKER_BLOB_UPDATE_POLL_INTERVAL_SECS
Interval to check for the worker to check for updated blobs and refresh worker state accordingly.
|
static String |
Config.WORKER_HEAP_MEMORY_MB
The default heap memory size in MB per worker, used in the jvm -Xmx opts for launching the worker.
|
static String |
Config.WORKER_HEARTBEAT_FREQUENCY_SECS
How often this worker should heartbeat to the supervisor.
|
static String |
Config.WORKER_LOG_LEVEL_RESET_POLL_SECS
How often a worker should check dynamic log level timeouts for expiration.
|
static String |
Config.WORKER_MAX_TIMEOUT_SECS
Enforce maximum on
Config.TOPOLOGY_WORKER_TIMEOUT_SECS . |
Modifier and Type | Field and Description |
---|---|
static String |
Configs.COMMIT_FREQ_COUNT
Deprecated.
please use
HdfsSpout.setCommitFrequencyCount(int) |
static String |
Configs.COMMIT_FREQ_SEC
Deprecated.
please use
HdfsSpout.setCommitFrequencySec(int) |
static String |
Configs.LOCK_TIMEOUT
Deprecated.
please use
HdfsSpout.setLockTimeoutSec(int) |
static String |
Configs.MAX_OUTSTANDING
Deprecated.
please use
HdfsSpout.setMaxOutstanding(int) |
Copyright © 2022 The Apache Software Foundation. All rights reserved.