public class DaemonConfig extends Object implements Validated
Storm configs are specified as a plain old map. This class provides constants for all the configurations possible on a Storm cluster. Each constant is paired with an annotation that defines the validity criterion of the corresponding field. Default values for these configs can be found in defaults.yaml.
This class extends Config
for supporting Storm Daemons.
Modifier and Type | Field and Description |
---|---|
static String |
BLACKLIST_SCHEDULER_ASSUME_SUPERVISOR_BAD_BASED_ON_BAD_SLOT
Whether
BlacklistScheduler will assume the supervisor is bad based on bad slots or not. |
static String |
BLACKLIST_SCHEDULER_ENABLE_SEND_ASSIGNMENT_FAILURES
Enables blacklisting support for supervisors with failed send assignment calls.
|
static String |
BLACKLIST_SCHEDULER_REPORTER
The class that the blacklist scheduler will report the blacklist.
|
static String |
BLACKLIST_SCHEDULER_RESUME_TIME
The number of seconds that the blacklisted slots or supervisor will be resumed.
|
static String |
BLACKLIST_SCHEDULER_STRATEGY
The class that specifies the eviction strategy to use in blacklist scheduler.
|
static String |
BLACKLIST_SCHEDULER_TOLERANCE_COUNT
The number of hit count that will trigger blacklist in tolerance time.
|
static String |
BLACKLIST_SCHEDULER_TOLERANCE_TIME
The number of seconds that the blacklist scheduler will concern of bad slots or supervisors.
|
static String |
DEV_ZOOKEEPER_PATH
The path to use as the zookeeper dir when running a zookeeper server via “storm dev-zookeeper”.
|
static String |
DRPC_AUTHORIZER
Class name for authorization plugin for DRPC client.
|
static String |
DRPC_CHILDOPTS
Childopts for Storm DRPC Java process.
|
static String |
DRPC_DISABLE_HTTP_BINDING
This controls wheather Storm DRPC should bind to http port even if drpc.http.port is > 0.
|
static String |
DRPC_HTTP_CREDS_PLUGIN
Class name of the HTTP credentials plugin for DRPC.
|
static String |
DRPC_HTTP_FILTER
A class implementing javax.servlet.Filter for DRPC HTTP requests.
|
static String |
DRPC_HTTP_FILTER_PARAMS
Initialization parameters for the javax.servlet.Filter of the DRPC HTTP service.
|
static String |
DRPC_HTTP_PORT
This port is used by Storm DRPC for receiving HTTP DPRC requests from clients.
|
static String |
DRPC_HTTPS_ENABLE_SSL_RELOAD
If set to true, keystore and truststore for DRPC Server will be automatically reloaded when modified.
|
static String |
DRPC_HTTPS_KEY_PASSWORD
Password to the private key in the keystore for setting up HTTPS (SSL).
|
static String |
DRPC_HTTPS_KEYSTORE_PASSWORD
Password to the keystore used by Storm DRPC for setting up HTTPS (SSL).
|
static String |
DRPC_HTTPS_KEYSTORE_PATH
Path to the keystore used by Storm DRPC for setting up HTTPS (SSL).
|
static String |
DRPC_HTTPS_KEYSTORE_TYPE
Type of keystore used by Storm DRPC for setting up HTTPS (SSL).
|
static String |
DRPC_HTTPS_NEED_CLIENT_AUTH |
static String |
DRPC_HTTPS_PORT
This port is used by Storm DRPC for receiving HTTPS (SSL) DPRC requests from clients.
|
static String |
DRPC_HTTPS_TRUSTSTORE_PASSWORD
Password to the truststore used by Storm DRPC setting up HTTPS (SSL).
|
static String |
DRPC_HTTPS_TRUSTSTORE_PATH
Path to the truststore used by Storm DRPC setting up HTTPS (SSL).
|
static String |
DRPC_HTTPS_TRUSTSTORE_TYPE
Type of truststore used by Storm DRPC for setting up HTTPS (SSL).
|
static String |
DRPC_HTTPS_WANT_CLIENT_AUTH
Password to the truststore used by Storm DRPC setting up HTTPS (SSL).
|
static String |
DRPC_REQUEST_TIMEOUT_SECS
The timeout on DRPC requests within the DRPC server.
|
static String |
ISOLATION_SCHEDULER_MACHINES
A map from topology name to the number of machines that should be dedicated for that topology.
|
static String |
JAVA_LIBRARY_PATH
This value is passed to spawned JVMs (e.g., Nimbus, Supervisor, and Workers) for the java.library.path value.
|
static String |
LOGS_GROUPS
A list of groups allowed to view logs via the Log Viewer.
|
static String |
LOGS_USERS
A list of users allowed to view logs via the Log Viewer.
|
static String |
LOGVIEWER_APPENDER_NAME
Appender name used by log viewer to determine log directory.
|
static String |
LOGVIEWER_CHILDOPTS
Childopts for log viewer java process.
|
static String |
LOGVIEWER_CLEANUP_AGE_MINS
How many minutes since a log was last modified for the log to be considered for clean-up.
|
static String |
LOGVIEWER_CLEANUP_INTERVAL_SECS
How often to clean up old log files.
|
static String |
LOGVIEWER_DISABLE_HTTP_BINDING
This controls wheather Storm Logviewer should bind to http port even if logviewer.port is > 0.
|
static String |
LOGVIEWER_FILTER
A class implementing javax.servlet.Filter for authenticating/filtering Logviewer requests.
|
static String |
LOGVIEWER_FILTER_PARAMS
Initialization parameters for the javax.servlet.Filter for Logviewer.
|
static String |
LOGVIEWER_HTTPS_ENABLE_SSL_RELOAD
If set to true, keystore and truststore for Logviewer will be automatically reloaded when modified.
|
static String |
LOGVIEWER_HTTPS_KEY_PASSWORD
Password to the private key in the keystore for setting up HTTPS (SSL).
|
static String |
LOGVIEWER_HTTPS_KEYSTORE_PASSWORD
Password for the keystore for HTTPS for Storm Logviewer.
|
static String |
LOGVIEWER_HTTPS_KEYSTORE_PATH
Path to the keystore containing the certs used by Storm Logviewer for HTTPS communications.
|
static String |
LOGVIEWER_HTTPS_KEYSTORE_TYPE
Type of the keystore for HTTPS for Storm Logviewer.
|
static String |
LOGVIEWER_HTTPS_NEED_CLIENT_AUTH |
static String |
LOGVIEWER_HTTPS_PORT
Storm Logviewer HTTPS port.
|
static String |
LOGVIEWER_HTTPS_TRUSTSTORE_PASSWORD
Password for the truststore for HTTPS for Storm Logviewer.
|
static String |
LOGVIEWER_HTTPS_TRUSTSTORE_PATH
Path to the truststore containing the certs used by Storm Logviewer for HTTPS communications.
|
static String |
LOGVIEWER_HTTPS_TRUSTSTORE_TYPE
Type of the truststore for HTTPS for Storm Logviewer.
|
static String |
LOGVIEWER_HTTPS_WANT_CLIENT_AUTH
Password to the truststore used by Storm Logviewer setting up HTTPS (SSL).
|
static String |
LOGVIEWER_MAX_PER_WORKER_LOGS_SIZE_MB
The maximum number of bytes per worker’s files can take up in MB.
|
static String |
LOGVIEWER_MAX_SUM_WORKER_LOGS_SIZE_MB
The maximum number of bytes all worker log files can take up in MB.
|
static String |
LOGVIEWER_PORT
HTTP UI port for log viewer.
|
static String |
MULTITENANT_SCHEDULER_USER_POOLS
A map from the user name to the number of machines that should that user is allowed to use.
|
static String |
NIMBUS_ASSIGNMENTS_SERVICE_THREAD_QUEUE_SIZE
This controls the number of working thread queue size of assignment service.
|
static String |
NIMBUS_ASSIGNMENTS_SERVICE_THREADS
This controls the number of working threads for distributing master assignments to supervisors.
|
static String |
NIMBUS_AUTHORIZER
Class name for authorization plugin for Nimbus.
|
static String |
NIMBUS_BLOBSTORE
What blobstore implementation nimbus should use.
|
static String |
NIMBUS_BLOBSTORE_EXPIRATION_SECS
During operations with the blob store, via master, how long a connection is idle before nimbus considers it dead and drops the session and any associated connections.
|
static String |
NIMBUS_CHILDOPTS
This parameter is used by the storm-deploy project to configure the jvm options for the nimbus daemon.
|
static String |
NIMBUS_CLEANUP_INBOX_FREQ_SECS
How often nimbus should wake the cleanup thread to clean the inbox.
|
static String |
NIMBUS_CODE_SYNC_FREQ_SECS
How often nimbus’s background thread to sync code for missing topologies should run.
|
static String |
NIMBUS_CREDENTIAL_RENEW_FREQ_SECS
How often nimbus should wake up to renew credentials if needed.
|
static String |
NIMBUS_DAEMON_USER
This is the user that the Nimbus daemon process is running as.
|
static String |
NIMBUS_EXECUTORS_PER_TOPOLOGY
A number representing the maximum number of executors any single topology can acquire.
|
static String |
NIMBUS_FILE_COPY_EXPIRATION_SECS
During upload/download with the master, how long an upload or download connection is idle before nimbus considers it dead and drops the connection.
|
static String |
NIMBUS_IMPERSONATION_AUTHORIZER
Impersonation user ACL config entries.
|
static String |
NIMBUS_INBOX_JAR_EXPIRATION_SECS
The length of time a jar file lives in the inbox before being deleted by the cleanup thread.
|
static String |
NIMBUS_MONITOR_FREQ_SECS
How often nimbus should wake up to check heartbeats and do reassignments.
|
static String |
NIMBUS_SLOTS_PER_TOPOLOGY
A number representing the maximum number of workers any single topology can acquire.
|
static String |
NIMBUS_SUPERVISOR_TIMEOUT_SECS
How long before a supervisor can go without heartbeating before nimbus considers it dead and stops assigning new work to it.
|
static String |
NIMBUS_TASK_LAUNCH_SECS
A special timeout used when a task is initially launched.
|
static String |
NIMBUS_TASK_TIMEOUT_SECS
How long without heartbeating a task can go before nimbus will consider the task dead and reassign it to another location.
|
static String |
NIMBUS_TOPOLOGY_ACTION_NOTIFIER_PLUGIN
FQCN of a class that implements
I @see org.apache.storm.nimbus.ITopologyActionNotifierPlugin for details. |
static String |
NIMBUS_TOPOLOGY_BLOBSTORE_DELETION_DELAY_MS
This controls the number of milliseconds nimbus will wait before deleting a topology blobstore once detected it is able to delete.
|
static String |
NIMBUS_TOPOLOGY_VALIDATOR
A custom class that implements ITopologyValidator that is run whenever a topology is submitted.
|
static String |
NIMBUS_WORKER_HEARTBEATS_RECOVERY_STRATEGY_CLASS
class controls heartbeats recovery strategy.
|
static String |
PACEMAKER_CHILDOPTS
This parameter is used by the storm-deploy project to configure the jvm options for the pacemaker daemon.
|
static String |
PACEMAKER_MAX_THREADS
The maximum number of threads that should be used by the Pacemaker.
|
static String |
RESOURCE_AWARE_SCHEDULER_MAX_STATE_SEARCH |
static String |
RESOURCE_AWARE_SCHEDULER_MAX_TOPOLOGY_SCHEDULING_ATTEMPTS
The maximum number of times that the RAS will attempt to schedule a topology.
|
static String |
RESOURCE_AWARE_SCHEDULER_PRIORITY_STRATEGY
the class that specifies the scheduling priority strategy to use in ResourceAwareScheduler.
|
static String |
RESOURCE_AWARE_SCHEDULER_USER_POOLS
A map of users to another map of the resource guarantees of the user.
|
static String |
SCHEDULER_CONFIG_LOADER_ARTIFACTORY_BASE_DIRECTORY
It is the part of the uri, configurable in Artifactory, which represents the top of the directory tree.
|
static String |
SCHEDULER_CONFIG_LOADER_POLLTIME_SECS
It is the frequency at which the plugin will call out to artifactory instead of returning the most recently cached result.
|
static String |
SCHEDULER_CONFIG_LOADER_TIMEOUT_SECS
It is the amount of time an http connection to the artifactory server will wait before timing out.
|
static String |
SCHEDULER_CONFIG_LOADER_URI
For ArtifactoryConfigLoader, this can either be a reference to an individual file in Artifactory or to a directory.
|
static String |
SCHEDULER_DISPLAY_RESOURCE
Whether we want to display all the resource capacity and scheduled usage on the UI page.
|
static String |
SCHEDULING_TIMEOUT_SECONDS_PER_TOPOLOGY
Max time to attempt to schedule one topology.
|
static String |
STORM_CGROUP_CGEXEC_CMD
full path to cgexec command.
|
static String |
STORM_CGROUP_HIERARCHY_NAME
name for the cgroup hierarchy.
|
static String |
STORM_CGROUP_INHERIT_CPUSET_CONFIGS
To determine whether or not to cgroups should inherit cpuset.cpus and cpuset.mems config values form parent cgroup Note that cpuset.cpus and cpuset.mems configs in a cgroup must be initialized (i.e.
|
static String |
STORM_CGROUP_MEMORY_ENFORCEMENT_ENABLE
Java does not always play nicely with cgroups.
|
static String |
STORM_CGROUP_MEMORY_LIMIT_TOLERANCE_MARGIN_MB
Please use STORM_SUPERVISOR_MEMORY_LIMIT_TOLERANCE_MARGIN_MB instead.
|
static String |
STORM_CGROUP_RESOURCES
resources to to be controlled by cgroups.
|
static String |
STORM_CLUSTER_METRICS_CONSUMER_PUBLISH_INTERVAL_SECS
How often cluster metrics data is published to metrics consumer.
|
static String |
STORM_CLUSTER_METRICS_CONSUMER_REGISTER
A list of classes implementing IClusterMetricsConsumer (See storm.yaml.example for exact config format).
|
static String |
STORM_DAEMON_METRICS_REPORTER_CSV_LOG_DIR
A specify csv reporter directory for CvsPreparableReporter daemon metrics reporter.
|
static String |
STORM_DAEMON_METRICS_REPORTER_PLUGIN_DOMAIN
A specify domain for daemon metrics reporter plugin to limit reporting to specific domain.
|
static String |
STORM_DAEMON_METRICS_REPORTER_PLUGINS
A list of daemon metrics reporter plugin class names.
|
static String |
STORM_HEALTH_CHECK_DIR
The directory where storm’s health scripts go.
|
static String |
STORM_HEALTH_CHECK_FAIL_ON_TIMEOUTS
Boolean setting to configure if health checks should fail when timeouts occur or not.
|
static String |
STORM_HEALTH_CHECK_TIMEOUT_MS
The time to allow any given healthcheck script to run before it is marked failed due to timeout.
|
static String |
STORM_LOG4J2_CONF_DIR
A directory that holds configuration files for log4j2.
|
static String |
STORM_METRIC_PROCESSOR_CLASS
Class implementing WorkerMetricsProcessor.
|
static String |
STORM_METRIC_STORE_CLASS
Class implementing MetricStore.
|
static String |
STORM_NETTY_FLUSH_CHECK_INTERVAL_MS
We check with this interval that whether the Netty channel is writable and try to write pending messages.
|
static String |
STORM_NIMBUS_ZOOKEEPER_ACLS_CHECK
In nimbus on startup check if all of the zookeeper ACLs are correct before starting.
|
static String |
STORM_NIMBUS_ZOOKEEPER_ACLS_FIXUP
In nimbus on startup check if all of the zookeeper ACLs are correct before starting.
|
static String |
STORM_RESOURCE_ISOLATION_PLUGIN
The plugin to be used for resource isolation.
|
static String |
STORM_RESOURCE_ISOLATION_PLUGIN_ENABLE
flag to determine whether to use a resource isolation plugin Also determines whether the unit tests for cgroup runs.
|
static String |
STORM_ROCKSDB_CREATE_IF_MISSING
RocksDB create if missing flag.
|
static String |
STORM_ROCKSDB_LOCATION
RocksDB file location.
|
static String |
STORM_ROCKSDB_METADATA_STRING_CACHE_CAPACITY
RocksDB metadata cache capacity.
|
static String |
STORM_ROCKSDB_METRIC_DELETION_PERIOD_HOURS
RocksDB setting for period of metric deletion thread.
|
static String |
STORM_ROCKSDB_METRIC_RETENTION_HOURS
RocksDB setting for length of metric retention.
|
static String |
STORM_SCHEDULER
A global task scheduler used to assign topologies’s tasks to supervisors’ workers.
|
static String |
STORM_SUPERVISOR_CGROUP_ROOTDIR
root directory for cgoups.
|
static String |
STORM_SUPERVISOR_HARD_LIMIT_MEMORY_OVERAGE_MB
If the memory usage of a worker goes over its limit by this value is it shot immediately.
|
static String |
STORM_SUPERVISOR_HARD_MEMORY_LIMIT_MULTIPLIER
A multiplier for the memory limit of a worker that will have the supervisor shoot it immediately.
|
static String |
STORM_SUPERVISOR_LOW_MEMORY_THRESHOLD_MB
If the amount of memory that is free in the system (either on the box or in the supervisor’s cgroup) is below this number (in MB) consider the system to be in low memory mode and start shooting workers if they are over their limit.
|
static String |
STORM_SUPERVISOR_MEDIUM_MEMORY_GRACE_PERIOD_MS
The number of milliseconds that a worker is allowed to be over their limit when there is a medium amount of memory free in the system.
|
static String |
STORM_SUPERVISOR_MEDIUM_MEMORY_THRESHOLD_MB
If the amount of memory that is free in the system (either on the box or in the supervisor’s cgroup) is below this number (in MB) consider the system to be a little low on memory and start shooting workers if they are over their limit for a given grace period STORM_SUPERVISOR_MEDIUM_MEMORY_GRACE_PERIOD_MS.
|
static String |
STORM_SUPERVISOR_MEMORY_LIMIT_TOLERANCE_MARGIN_MB
Memory given to each worker for free (because java and storm have some overhead).
|
static String |
STORM_TOPOLOGY_CLASSPATH_BEGINNING_ENABLED
Enables user-first classpath.
|
static String |
STORM_WORKER_CGROUP_CPU_LIMIT
the manually set cpu share for each CGroup on supervisor node.
|
static String |
STORM_WORKER_CGROUP_MEMORY_MB_LIMIT
the manually set memory limit (in MB) for each CGroup on supervisor node.
|
static String |
STORM_WORKER_MIN_CPU_PCORE_PERCENT
The config indicates the minimum percentage of cpu for a core that a worker will use.
|
static String |
STORM_WORKER_TOKEN_LIFE_TIME_HOURS
The number of hours a worker token is valid for.
|
static String |
SUPERVISOR_AUTHORIZER
Class name for authorization plugin for supervisor.
|
static String |
SUPERVISOR_BLOBSTORE
What blobstore implementation the supervisor should use.
|
static String |
SUPERVISOR_BLOBSTORE_DOWNLOAD_MAX_RETRIES
Maximum number of retries a supervisor is allowed to make for downloading a blob.
|
static String |
SUPERVISOR_BLOBSTORE_DOWNLOAD_THREAD_COUNT
What blobstore download parallelism the supervisor should use.
|
static String |
SUPERVISOR_CHILDOPTS
This parameter is used by the storm-deploy project to configure the jvm options for the supervisor daemon.
|
static String |
SUPERVISOR_ENABLE
Whether or not the supervisor should launch workers assigned to it.
|
static String |
SUPERVISOR_HEARTBEAT_FREQUENCY_SECS
how often the supervisor sends a heartbeat to the master.
|
static String |
SUPERVISOR_LOCALIZER_CACHE_CLEANUP_INTERVAL_MS
The distributed cache cleanup interval.
|
static String |
SUPERVISOR_LOCALIZER_CACHE_TARGET_SIZE_MB
The distributed cache target size in MB.
|
static String |
SUPERVISOR_MONITOR_FREQUENCY_SECS
How often the supervisor checks the worker heartbeats to see if any of them need to be restarted.
|
static String |
SUPERVISOR_NUMA_META
A map with keys mapped to each NUMA Node on the supervisor that will be used by scheduler.
|
static String |
SUPERVISOR_SCHEDULER_META
the metadata configured on the supervisor.
|
static String |
SUPERVISOR_SLOTS_PORTS
A list of ports that can run workers on this supervisor.
|
static String |
SUPERVISOR_WORKER_START_TIMEOUT_SECS
How long a worker can go without heartbeating during the initial launch before the supervisor tries to restart the worker process.
|
static String |
UI_CENTRAL_LOGGING_URL
Storm UI Central Logging URL.
|
static String |
UI_CHILDOPTS
Childopts for Storm UI Java process.
|
static String |
UI_DISABLE_HTTP_BINDING
This controls wheather Storm UI should bind to http port even if ui.port is > 0.
|
static String |
UI_DISABLE_SPOUT_LAG_MONITORING
This controls whether Storm UI displays spout lag for the Kafka spout.
|
static String |
UI_FILTER
A class implementing javax.servlet.Filter for authenticating/filtering UI requests.
|
static String |
UI_FILTER_PARAMS
Initialization parameters for the javax.servlet.Filter for UI.
|
static String |
UI_HEADER_BUFFER_BYTES
The size of the header buffer for the UI in bytes.
|
static String |
UI_HOST
Storm UI binds to this host/interface.
|
static String |
UI_HTTP_CREDS_PLUGIN
Class name of the HTTP credentials plugin for the UI.
|
static String |
UI_HTTPS_ENABLE_SSL_RELOAD
If set to true, keystore and truststore for UI will be automatically reloaded when modified.
|
static String |
UI_HTTPS_KEY_PASSWORD
Password to the private key in the keystore for setting up HTTPS (SSL).
|
static String |
UI_HTTPS_KEYSTORE_PASSWORD
Password to the keystore used by Storm UI for setting up HTTPS (SSL).
|
static String |
UI_HTTPS_KEYSTORE_PATH
Path to the keystore used by Storm UI for setting up HTTPS (SSL).
|
static String |
UI_HTTPS_KEYSTORE_TYPE
Type of keystore used by Storm UI for setting up HTTPS (SSL).
|
static String |
UI_HTTPS_NEED_CLIENT_AUTH |
static String |
UI_HTTPS_PORT
This port is used by Storm UI for receiving HTTPS (SSL) requests from clients.
|
static String |
UI_HTTPS_TRUSTSTORE_PASSWORD
Password to the truststore used by Storm UI setting up HTTPS (SSL).
|
static String |
UI_HTTPS_TRUSTSTORE_PATH
Path to the truststore used by Storm UI setting up HTTPS (SSL).
|
static String |
UI_HTTPS_TRUSTSTORE_TYPE
Type of truststore used by Storm UI for setting up HTTPS (SSL).
|
static String |
UI_HTTPS_WANT_CLIENT_AUTH
Password to the truststore used by Storm DRPC setting up HTTPS (SSL).
|
static String |
UI_PAGINATION
Storm UI drop-down pagination value.
|
static String |
UI_PORT
Storm UI binds to this port.
|
static String |
UI_PROJECT_BUGTRACKER_URL
Storm UI Project BUGTRACKER Link for reporting issue.
|
static String |
VALIDATE_TOPOLOGY_SCHEDULER_STRATEGY
Server side validation that @{see Config#TOPOLOGY_SCHEDULER_STRATEGY} is set ot a subclass of IStrategy.
|
static String |
WORKER_PROFILER_CHILDOPTS
The jvm profiler opts provided to workers launched by this supervisor.
|
static String |
WORKER_PROFILER_COMMAND
The command launched supervisor with worker arguments pid, action and [target_directory] Where action is - start profile, stop profile, jstack, heapdump and kill against pid.
|
static String |
WORKER_PROFILER_ENABLED
Enable profiling of worker JVMs using Oracle’s Java Flight Recorder.
|
Constructor and Description |
---|
DaemonConfig() |
Modifier and Type | Method and Description |
---|---|
static String |
getCgroupRootDir(Map<String,Object> conf) |
static String |
getCgroupStormHierarchyDir(Map<String,Object> conf) |
static String |
getCgroupStormHierarchyName(Map<String,Object> conf) |
static ArrayList<String> |
getCgroupStormResources(Map<String,Object> conf)
Get the cgroup resources from the conf.
|
public static final String STORM_NETTY_FLUSH_CHECK_INTERVAL_MS
We check with this interval that whether the Netty channel is writable and try to write pending messages.
public static final String STORM_DAEMON_METRICS_REPORTER_PLUGINS
A list of daemon metrics reporter plugin class names. These plugins must implement PreparableReporter
interface.
public static final String STORM_DAEMON_METRICS_REPORTER_PLUGIN_DOMAIN
A specify domain for daemon metrics reporter plugin to limit reporting to specific domain.
public static final String STORM_DAEMON_METRICS_REPORTER_CSV_LOG_DIR
A specify csv reporter directory for CvsPreparableReporter daemon metrics reporter.
public static final String STORM_LOG4J2_CONF_DIR
A directory that holds configuration files for log4j2. It can be either a relative or an absolute directory. If relative, it is relative to the storm’s home directory.
public static final String STORM_SCHEDULER
A global task scheduler used to assign topologies’s tasks to supervisors’ workers.
If this is not set, a default system scheduler will be used.
public static final String SCHEDULING_TIMEOUT_SECONDS_PER_TOPOLOGY
Max time to attempt to schedule one topology. The default is 60 seconds
public static final String BLACKLIST_SCHEDULER_TOLERANCE_TIME
The number of seconds that the blacklist scheduler will concern of bad slots or supervisors.
public static final String BLACKLIST_SCHEDULER_TOLERANCE_COUNT
The number of hit count that will trigger blacklist in tolerance time.
public static final String BLACKLIST_SCHEDULER_RESUME_TIME
The number of seconds that the blacklisted slots or supervisor will be resumed.
public static final String BLACKLIST_SCHEDULER_ENABLE_SEND_ASSIGNMENT_FAILURES
Enables blacklisting support for supervisors with failed send assignment calls.
public static final String BLACKLIST_SCHEDULER_REPORTER
The class that the blacklist scheduler will report the blacklist.
public static final String BLACKLIST_SCHEDULER_STRATEGY
The class that specifies the eviction strategy to use in blacklist scheduler. If you are using the RAS scheduler please set this to “org.apache.storm.scheduler.blacklist.strategies.RasBlacklistStrategy” or you may get odd behavior when the cluster is full and there are blacklisted nodes.
public static final String BLACKLIST_SCHEDULER_ASSUME_SUPERVISOR_BAD_BASED_ON_BAD_SLOT
Whether BlacklistScheduler
will assume the supervisor is bad based on bad slots or not. A bad slot indicates the situation where the nimbus doesn’t receive heartbeat from the worker in time, it’s hard to differentiate if it’s because of the supervisor node or the worker itself. If this is set to true, the scheduler will consider a supervisor is bad when seeing bad slots in it. Otherwise, the scheduler will assume a supervisor is bad only when it does not receive supervisor heartbeat in time.
public static final String SCHEDULER_DISPLAY_RESOURCE
Whether we want to display all the resource capacity and scheduled usage on the UI page. You MUST have this variable set if you are using any kind of resource-related scheduler.
If this is not set, we will not display resource capacity and usage on the UI.public static final String STORM_HEALTH_CHECK_DIR
The directory where storm’s health scripts go.
public static final String STORM_HEALTH_CHECK_TIMEOUT_MS
The time to allow any given healthcheck script to run before it is marked failed due to timeout.
public static final String STORM_HEALTH_CHECK_FAIL_ON_TIMEOUTS
Boolean setting to configure if health checks should fail when timeouts occur or not.
public static final String NIMBUS_DAEMON_USER
This is the user that the Nimbus daemon process is running as. May be used when security is enabled to authorize actions in the cluster.
public static final String NIMBUS_CHILDOPTS
This parameter is used by the storm-deploy project to configure the jvm options for the nimbus daemon.
public static final String NIMBUS_TASK_TIMEOUT_SECS
How long without heartbeating a task can go before nimbus will consider the task dead and reassign it to another location. Can be exceeded when Config.TOPOLOGY_WORKER_TIMEOUT_SECS
is set.
public static final String NIMBUS_MONITOR_FREQ_SECS
How often nimbus should wake up to check heartbeats and do reassignments. Note that if a machine ever goes down Nimbus will immediately wake up and take action. This parameter is for checking for failures when there’s no explicit event like that occurring.
public static final String NIMBUS_CLEANUP_INBOX_FREQ_SECS
How often nimbus should wake the cleanup thread to clean the inbox.
public static final String NIMBUS_INBOX_JAR_EXPIRATION_SECS
The length of time a jar file lives in the inbox before being deleted by the cleanup thread.
Probably keep this value greater than or equal to NIMBUS_CLEANUP_INBOX_JAR_EXPIRATION_SECS. Note that the time it takes to delete an inbox jar file is going to be somewhat more than NIMBUS_CLEANUP_INBOX_JAR_EXPIRATION_SECS (depending on how often NIMBUS_CLEANUP_FREQ_SECS is set to).
public static final String NIMBUS_SUPERVISOR_TIMEOUT_SECS
How long before a supervisor can go without heartbeating before nimbus considers it dead and stops assigning new work to it.
public static final String NIMBUS_TASK_LAUNCH_SECS
A special timeout used when a task is initially launched. During launch, this is the timeout used until the first heartbeat, overriding nimbus.task.timeout.secs.
A separate timeout exists for launch because there can be quite a bit of overhead to launching new JVM's and configuring them.
Can be exceeded when Config.TOPOLOGY_WORKER_TIMEOUT_SECS
is set.
public static final String NIMBUS_FILE_COPY_EXPIRATION_SECS
During upload/download with the master, how long an upload or download connection is idle before nimbus considers it dead and drops the connection.
public static final String NIMBUS_TOPOLOGY_VALIDATOR
A custom class that implements ITopologyValidator that is run whenever a topology is submitted. Can be used to provide business-specific logic for whether topologies are allowed to run or not.
public static final String NIMBUS_AUTHORIZER
Class name for authorization plugin for Nimbus.
public static final String SUPERVISOR_AUTHORIZER
Class name for authorization plugin for supervisor.
public static final String NIMBUS_IMPERSONATION_AUTHORIZER
Impersonation user ACL config entries.
public static final String NIMBUS_CREDENTIAL_RENEW_FREQ_SECS
How often nimbus should wake up to renew credentials if needed.
public static final String NIMBUS_TOPOLOGY_ACTION_NOTIFIER_PLUGIN
FQCN of a class that implements I
@see org.apache.storm.nimbus.ITopologyActionNotifierPlugin for details.
public static final String NIMBUS_ASSIGNMENTS_SERVICE_THREADS
This controls the number of working threads for distributing master assignments to supervisors.
public static final String NIMBUS_ASSIGNMENTS_SERVICE_THREAD_QUEUE_SIZE
This controls the number of working thread queue size of assignment service.
public static final String NIMBUS_WORKER_HEARTBEATS_RECOVERY_STRATEGY_CLASS
class controls heartbeats recovery strategy.
public static final String NIMBUS_TOPOLOGY_BLOBSTORE_DELETION_DELAY_MS
This controls the number of milliseconds nimbus will wait before deleting a topology blobstore once detected it is able to delete.
public static final String UI_HOST
Storm UI binds to this host/interface.
public static final String UI_PORT
Storm UI binds to this port.
public static final String UI_DISABLE_HTTP_BINDING
This controls wheather Storm UI should bind to http port even if ui.port is > 0.
public static final String UI_DISABLE_SPOUT_LAG_MONITORING
This controls whether Storm UI displays spout lag for the Kafka spout.
public static final String LOGVIEWER_DISABLE_HTTP_BINDING
This controls wheather Storm Logviewer should bind to http port even if logviewer.port is > 0.
public static final String DRPC_DISABLE_HTTP_BINDING
This controls wheather Storm DRPC should bind to http port even if drpc.http.port is > 0.
public static final String UI_PROJECT_BUGTRACKER_URL
Storm UI Project BUGTRACKER Link for reporting issue.
public static final String UI_CENTRAL_LOGGING_URL
Storm UI Central Logging URL.
public static final String UI_PAGINATION
Storm UI drop-down pagination value. Set ui.pagination to be a positive integer or -1 (displays all entries). Valid values: -1, 10, 20, 25 etc.
public static final String LOGVIEWER_PORT
HTTP UI port for log viewer.
public static final String LOGVIEWER_CHILDOPTS
Childopts for log viewer java process.
public static final String LOGVIEWER_CLEANUP_INTERVAL_SECS
How often to clean up old log files.
public static final String LOGVIEWER_CLEANUP_AGE_MINS
How many minutes since a log was last modified for the log to be considered for clean-up.
public static final String LOGVIEWER_MAX_SUM_WORKER_LOGS_SIZE_MB
The maximum number of bytes all worker log files can take up in MB.
public static final String LOGVIEWER_MAX_PER_WORKER_LOGS_SIZE_MB
The maximum number of bytes per worker’s files can take up in MB.
public static final String LOGVIEWER_HTTPS_PORT
Storm Logviewer HTTPS port. Logviewer must use HTTPS if Storm UI is using HTTPS.
public static final String LOGVIEWER_HTTPS_KEYSTORE_PATH
Path to the keystore containing the certs used by Storm Logviewer for HTTPS communications.
public static final String LOGVIEWER_HTTPS_KEYSTORE_PASSWORD
Password for the keystore for HTTPS for Storm Logviewer.
public static final String LOGVIEWER_HTTPS_KEYSTORE_TYPE
Type of the keystore for HTTPS for Storm Logviewer. see http://docs.oracle.com/javase/8/docs/api/java/security/KeyStore.html for more details.
public static final String LOGVIEWER_HTTPS_KEY_PASSWORD
Password to the private key in the keystore for setting up HTTPS (SSL).
public static final String LOGVIEWER_HTTPS_TRUSTSTORE_PATH
Path to the truststore containing the certs used by Storm Logviewer for HTTPS communications.
public static final String LOGVIEWER_HTTPS_TRUSTSTORE_PASSWORD
Password for the truststore for HTTPS for Storm Logviewer.
public static final String LOGVIEWER_HTTPS_TRUSTSTORE_TYPE
Type of the truststore for HTTPS for Storm Logviewer. see http://docs.oracle.com/javase/8/docs/api/java/security/Truststore.html for more details.
public static final String LOGVIEWER_HTTPS_WANT_CLIENT_AUTH
Password to the truststore used by Storm Logviewer setting up HTTPS (SSL).
public static final String LOGVIEWER_HTTPS_NEED_CLIENT_AUTH
public static final String LOGVIEWER_HTTPS_ENABLE_SSL_RELOAD
If set to true, keystore and truststore for Logviewer will be automatically reloaded when modified.
public static final String LOGS_USERS
A list of users allowed to view logs via the Log Viewer.
public static final String LOGS_GROUPS
A list of groups allowed to view logs via the Log Viewer.
public static final String LOGVIEWER_APPENDER_NAME
Appender name used by log viewer to determine log directory.
public static final String LOGVIEWER_FILTER
A class implementing javax.servlet.Filter for authenticating/filtering Logviewer requests.
public static final String LOGVIEWER_FILTER_PARAMS
Initialization parameters for the javax.servlet.Filter for Logviewer.
public static final String UI_CHILDOPTS
Childopts for Storm UI Java process.
public static final String UI_FILTER
A class implementing javax.servlet.Filter for authenticating/filtering UI requests.
public static final String UI_FILTER_PARAMS
Initialization parameters for the javax.servlet.Filter for UI.
public static final String UI_HEADER_BUFFER_BYTES
The size of the header buffer for the UI in bytes.
public static final String UI_HTTPS_PORT
This port is used by Storm UI for receiving HTTPS (SSL) requests from clients.
public static final String UI_HTTPS_KEYSTORE_PATH
Path to the keystore used by Storm UI for setting up HTTPS (SSL).
public static final String UI_HTTPS_KEYSTORE_PASSWORD
Password to the keystore used by Storm UI for setting up HTTPS (SSL).
public static final String UI_HTTPS_KEYSTORE_TYPE
Type of keystore used by Storm UI for setting up HTTPS (SSL). see http://docs.oracle.com/javase/7/docs/api/java/security/KeyStore .html for more details.
public static final String UI_HTTPS_KEY_PASSWORD
Password to the private key in the keystore for setting up HTTPS (SSL).
public static final String UI_HTTPS_TRUSTSTORE_PATH
Path to the truststore used by Storm UI setting up HTTPS (SSL).
public static final String UI_HTTPS_TRUSTSTORE_PASSWORD
Password to the truststore used by Storm UI setting up HTTPS (SSL).
public static final String UI_HTTPS_TRUSTSTORE_TYPE
Type of truststore used by Storm UI for setting up HTTPS (SSL). see http://docs.oracle .com/javase/7/docs/api/java/security/KeyStore.html for more details.
public static final String UI_HTTPS_WANT_CLIENT_AUTH
Password to the truststore used by Storm DRPC setting up HTTPS (SSL).
public static final String UI_HTTPS_NEED_CLIENT_AUTH
public static final String UI_HTTPS_ENABLE_SSL_RELOAD
If set to true, keystore and truststore for UI will be automatically reloaded when modified.
public static final String PACEMAKER_MAX_THREADS
The maximum number of threads that should be used by the Pacemaker. When Pacemaker gets loaded it will spawn new threads, up to this many total, to handle the load.
public static final String PACEMAKER_CHILDOPTS
This parameter is used by the storm-deploy project to configure the jvm options for the pacemaker daemon.
public static final String DRPC_HTTP_PORT
This port is used by Storm DRPC for receiving HTTP DPRC requests from clients.
public static final String DRPC_HTTPS_PORT
This port is used by Storm DRPC for receiving HTTPS (SSL) DPRC requests from clients.
public static final String DRPC_HTTPS_KEYSTORE_PATH
Path to the keystore used by Storm DRPC for setting up HTTPS (SSL).
public static final String DRPC_HTTPS_KEYSTORE_PASSWORD
Password to the keystore used by Storm DRPC for setting up HTTPS (SSL).
public static final String DRPC_HTTPS_KEYSTORE_TYPE
Type of keystore used by Storm DRPC for setting up HTTPS (SSL). see http://docs.oracle .com/javase/7/docs/api/java/security/KeyStore.html for more details.
public static final String DRPC_HTTPS_KEY_PASSWORD
Password to the private key in the keystore for setting up HTTPS (SSL).
public static final String DRPC_HTTPS_TRUSTSTORE_PATH
Path to the truststore used by Storm DRPC setting up HTTPS (SSL).
public static final String DRPC_HTTPS_TRUSTSTORE_PASSWORD
Password to the truststore used by Storm DRPC setting up HTTPS (SSL).
public static final String DRPC_HTTPS_TRUSTSTORE_TYPE
Type of truststore used by Storm DRPC for setting up HTTPS (SSL). see http://docs.oracle .com/javase/7/docs/api/java/security/KeyStore.html for more details.
public static final String DRPC_HTTPS_WANT_CLIENT_AUTH
Password to the truststore used by Storm DRPC setting up HTTPS (SSL).
public static final String DRPC_HTTPS_NEED_CLIENT_AUTH
public static final String DRPC_HTTPS_ENABLE_SSL_RELOAD
If set to true, keystore and truststore for DRPC Server will be automatically reloaded when modified.
public static final String DRPC_AUTHORIZER
Class name for authorization plugin for DRPC client.
public static final String DRPC_REQUEST_TIMEOUT_SECS
The timeout on DRPC requests within the DRPC server. Defaults to 10 minutes. Note that requests can also timeout based on the socket timeout on the DRPC client, and separately based on the topology message timeout for the topology implementing the DRPC function.
public static final String DRPC_CHILDOPTS
Childopts for Storm DRPC Java process.
public static final String SUPERVISOR_SCHEDULER_META
the metadata configured on the supervisor.
public static final String SUPERVISOR_SLOTS_PORTS
A list of ports that can run workers on this supervisor. Each worker uses one port, and the supervisor will only run one worker per port. Use this configuration to tune how many workers run on each machine.
public static final String SUPERVISOR_BLOBSTORE
What blobstore implementation the supervisor should use.
public static final String SUPERVISOR_LOCALIZER_CACHE_TARGET_SIZE_MB
The distributed cache target size in MB. This is a soft limit to the size of the distributed cache contents.
public static final String SUPERVISOR_LOCALIZER_CACHE_CLEANUP_INTERVAL_MS
The distributed cache cleanup interval. Controls how often it scans to attempt to cleanup anything over the cache target size.
public static final String SUPERVISOR_BLOBSTORE_DOWNLOAD_THREAD_COUNT
What blobstore download parallelism the supervisor should use.
public static final String SUPERVISOR_BLOBSTORE_DOWNLOAD_MAX_RETRIES
Maximum number of retries a supervisor is allowed to make for downloading a blob.
public static final String SUPERVISOR_NUMA_META
A map with keys mapped to each NUMA Node on the supervisor that will be used by scheduler. CPUs, memory and ports available on each NUMA node will be provided. Each supervisor will have different map of NUMAs. Example: “supervisor.numa.meta”: { “0”: { “numa.memory.mb”: 122880, “numa.cores”: [ 0, 12, 1, 13, 2, 14, 3, 15, 4, 16, 5, 17], “numa.ports”: [6700, 6701]}, “1” : {“numa.memory.mb”: 122880, “numa.cores”: [ 6, 18, 7, 19, 8, 20, 9, 21, 10, 22, 11, 23], “numa.ports”: [6702, 6703], “numa.generic.resources.map”: {“gpu.count” : 1}} }
public static final String NIMBUS_BLOBSTORE
What blobstore implementation nimbus should use.
public static final String NIMBUS_BLOBSTORE_EXPIRATION_SECS
During operations with the blob store, via master, how long a connection is idle before nimbus considers it dead and drops the session and any associated connections.
public static final String NIMBUS_SLOTS_PER_TOPOLOGY
A number representing the maximum number of workers any single topology can acquire. This will be ignored if the Resource Aware Scheduler is used.
public static final String DRPC_HTTP_FILTER
A class implementing javax.servlet.Filter for DRPC HTTP requests.
public static final String DRPC_HTTP_FILTER_PARAMS
Initialization parameters for the javax.servlet.Filter of the DRPC HTTP service.
public static final String NIMBUS_EXECUTORS_PER_TOPOLOGY
A number representing the maximum number of executors any single topology can acquire.
public static final String SUPERVISOR_CHILDOPTS
This parameter is used by the storm-deploy project to configure the jvm options for the supervisor daemon.
public static final String SUPERVISOR_WORKER_START_TIMEOUT_SECS
How long a worker can go without heartbeating during the initial launch before the supervisor tries to restart the worker process. This value override supervisor.worker.timeout.secs during launch because there is additional overhead to starting and configuring the JVM on launch. Can be exceeded when Config.TOPOLOGY_WORKER_TIMEOUT_SECS
is set.
public static final String SUPERVISOR_ENABLE
Whether or not the supervisor should launch workers assigned to it. Defaults to true – and you should probably never change this value. This configuration is used in the Storm unit tests.
public static final String SUPERVISOR_HEARTBEAT_FREQUENCY_SECS
how often the supervisor sends a heartbeat to the master.
public static final String SUPERVISOR_MONITOR_FREQUENCY_SECS
How often the supervisor checks the worker heartbeats to see if any of them need to be restarted.
public static final String WORKER_PROFILER_CHILDOPTS
The jvm profiler opts provided to workers launched by this supervisor.
public static final String WORKER_PROFILER_ENABLED
Enable profiling of worker JVMs using Oracle’s Java Flight Recorder. Unlocking commercial features requires a special license from Oracle. See http://www.oracle.com/technetwork/java/javase/terms/products/index.html
public static final String WORKER_PROFILER_COMMAND
The command launched supervisor with worker arguments pid, action and [target_directory] Where action is - start profile, stop profile, jstack, heapdump and kill against pid.
public static final String STORM_CLUSTER_METRICS_CONSUMER_REGISTER
A list of classes implementing IClusterMetricsConsumer (See storm.yaml.example for exact config format). Each listed class will be routed cluster related metrics data. Each listed class maps 1:1 to a ClusterMetricsConsumerExecutor and they’re executed in Nimbus. Only consumers which run in leader Nimbus receives metrics data.
public static final String STORM_CLUSTER_METRICS_CONSUMER_PUBLISH_INTERVAL_SECS
How often cluster metrics data is published to metrics consumer.
public static final String STORM_TOPOLOGY_CLASSPATH_BEGINNING_ENABLED
Enables user-first classpath. See topology.classpath.beginning.
public static final String JAVA_LIBRARY_PATH
This value is passed to spawned JVMs (e.g., Nimbus, Supervisor, and Workers) for the java.library.path value. java.library.path tells the JVM where to look for native libraries. It is necessary to set this config correctly since Storm uses the ZeroMQ and JZMQ native libs.
public static final String DEV_ZOOKEEPER_PATH
The path to use as the zookeeper dir when running a zookeeper server via “storm dev-zookeeper”. This zookeeper instance is only intended for development; it is not a production grade zookeeper setup.
public static final String ISOLATION_SCHEDULER_MACHINES
A map from topology name to the number of machines that should be dedicated for that topology. Set storm.scheduler to org.apache.storm.scheduler.IsolationScheduler to make use of the isolation scheduler.
public static final String SCHEDULER_CONFIG_LOADER_URI
For ArtifactoryConfigLoader, this can either be a reference to an individual file in Artifactory or to a directory. If it is a directory, the file with the largest lexographic name will be returned. Users need to add “artifactory+” to the beginning of the real URI to use ArtifactoryConfigLoader. For FileConfigLoader, this is the URI pointing to a file.
public static final String SCHEDULER_CONFIG_LOADER_POLLTIME_SECS
It is the frequency at which the plugin will call out to artifactory instead of returning the most recently cached result. Currently it’s only used in ArtifactoryConfigLoader.
public static final String SCHEDULER_CONFIG_LOADER_TIMEOUT_SECS
It is the amount of time an http connection to the artifactory server will wait before timing out. Currently it’s only used in ArtifactoryConfigLoader.
public static final String SCHEDULER_CONFIG_LOADER_ARTIFACTORY_BASE_DIRECTORY
It is the part of the uri, configurable in Artifactory, which represents the top of the directory tree. It’s only used in ArtifactoryConfigLoader.
public static final String MULTITENANT_SCHEDULER_USER_POOLS
A map from the user name to the number of machines that should that user is allowed to use. Set storm.scheduler to org.apache.storm.scheduler.multitenant.MultitenantScheduler
public static final String RESOURCE_AWARE_SCHEDULER_USER_POOLS
A map of users to another map of the resource guarantees of the user. Used by Resource Aware Scheduler to ensure per user resource guarantees.
public static final String RESOURCE_AWARE_SCHEDULER_PRIORITY_STRATEGY
the class that specifies the scheduling priority strategy to use in ResourceAwareScheduler.
public static final String RESOURCE_AWARE_SCHEDULER_MAX_TOPOLOGY_SCHEDULING_ATTEMPTS
The maximum number of times that the RAS will attempt to schedule a topology. The default is 5.
public static final String RESOURCE_AWARE_SCHEDULER_MAX_STATE_SEARCH
public static final String NIMBUS_CODE_SYNC_FREQ_SECS
How often nimbus’s background thread to sync code for missing topologies should run.
public static final String STORM_RESOURCE_ISOLATION_PLUGIN
The plugin to be used for resource isolation.
public static final String STORM_CGROUP_RESOURCES
resources to to be controlled by cgroups.
public static final String STORM_CGROUP_HIERARCHY_NAME
name for the cgroup hierarchy.
public static final String STORM_RESOURCE_ISOLATION_PLUGIN_ENABLE
flag to determine whether to use a resource isolation plugin Also determines whether the unit tests for cgroup runs. If storm.resource.isolation.plugin.enable is set to false the unit tests for cgroups will not run
public static final String STORM_METRIC_STORE_CLASS
Class implementing MetricStore. Runs on Nimbus.
public static final String STORM_METRIC_PROCESSOR_CLASS
Class implementing WorkerMetricsProcessor. Runs on Supervisors.
public static final String STORM_ROCKSDB_LOCATION
RocksDB file location. This setting is specific to the org.apache.storm.metricstore.rocksdb.RocksDbStore implementation for the storm.metricstore.class.
public static final String STORM_ROCKSDB_CREATE_IF_MISSING
RocksDB create if missing flag. This setting is specific to the org.apache.storm.metricstore.rocksdb.RocksDbStore implementation for the storm.metricstore.class.
public static final String STORM_ROCKSDB_METADATA_STRING_CACHE_CAPACITY
RocksDB metadata cache capacity. This setting is specific to the org.apache.storm.metricstore.rocksdb.RocksDbStore implementation for the storm.metricstore.class.
public static final String STORM_ROCKSDB_METRIC_RETENTION_HOURS
RocksDB setting for length of metric retention. This setting is specific to the org.apache.storm.metricstore.rocksdb.RocksDbStore implementation for the storm.metricstore.class.
public static final String STORM_ROCKSDB_METRIC_DELETION_PERIOD_HOURS
RocksDB setting for period of metric deletion thread. This setting is specific to the org.apache.storm.metricstore.rocksdb .RocksDbStore implementation for the storm.metricstore.class.
public static final String STORM_NIMBUS_ZOOKEEPER_ACLS_CHECK
In nimbus on startup check if all of the zookeeper ACLs are correct before starting. If not don’t start nimbus.
public static final String STORM_NIMBUS_ZOOKEEPER_ACLS_FIXUP
In nimbus on startup check if all of the zookeeper ACLs are correct before starting. If not do your best to fix them before nimbus starts, if it cannot fix them nimbus will not start. This overrides any value set for storm.nimbus.zookeeper.acls.check.
public static final String VALIDATE_TOPOLOGY_SCHEDULER_STRATEGY
Server side validation that @{see Config#TOPOLOGY_SCHEDULER_STRATEGY} is set ot a subclass of IStrategy.
public static final String UI_HTTP_CREDS_PLUGIN
Class name of the HTTP credentials plugin for the UI.
public static final String DRPC_HTTP_CREDS_PLUGIN
Class name of the HTTP credentials plugin for DRPC.
public static String STORM_SUPERVISOR_CGROUP_ROOTDIR
root directory for cgoups.
public static String STORM_WORKER_CGROUP_MEMORY_MB_LIMIT
the manually set memory limit (in MB) for each CGroup on supervisor node.
public static String STORM_WORKER_CGROUP_CPU_LIMIT
the manually set cpu share for each CGroup on supervisor node.
public static String STORM_CGROUP_CGEXEC_CMD
full path to cgexec command.
public static String STORM_CGROUP_MEMORY_LIMIT_TOLERANCE_MARGIN_MB
Please use STORM_SUPERVISOR_MEMORY_LIMIT_TOLERANCE_MARGIN_MB instead. The amount of memory a worker can exceed its allocation before cgroup will kill it.
public static String STORM_CGROUP_INHERIT_CPUSET_CONFIGS
To determine whether or not to cgroups should inherit cpuset.cpus and cpuset.mems config values form parent cgroup Note that cpuset.cpus and cpuset.mems configs in a cgroup must be initialized (i.e. contain a valid value) prior to being able to launch processes in that cgroup. The common use case for this config is when the linux distribution that is used does not support the cgroup.clone_children config.
public static String STORM_CGROUP_MEMORY_ENFORCEMENT_ENABLE
Java does not always play nicely with cgroups. It is coming but not fully implemented and not for the way storm uses cgroups. In the short term you can disable the hard memory enforcement by cgroups and let the supervisor handle shooting workers going over their limit in a kinder way.
public static String STORM_SUPERVISOR_MEMORY_LIMIT_TOLERANCE_MARGIN_MB
Memory given to each worker for free (because java and storm have some overhead). This is memory on the box that the workers can use. This should not be included in SUPERVISOR_MEMORY_CAPACITY_MB, as nimbus does not use this memory for scheduling.
public static String STORM_SUPERVISOR_HARD_MEMORY_LIMIT_MULTIPLIER
A multiplier for the memory limit of a worker that will have the supervisor shoot it immediately. 1.0 means shoot the worker as soon as it goes over. 2.0 means shoot the worker if its usage is double what was requested. This value is combined with STORM_SUPERVISOR_HARD_MEMORY_LIMIT_OVERAGE and which ever is greater is used for enforcement. This allows small workers to not be shot.
public static String STORM_SUPERVISOR_HARD_LIMIT_MEMORY_OVERAGE_MB
If the memory usage of a worker goes over its limit by this value is it shot immediately. This value is combined with STORM_SUPERVISOR_HARD_LIMIT_MEMORY_MULTIPLIER and which ever is greater is used for enforcement. This allows small workers to not be shot.
public static String STORM_SUPERVISOR_LOW_MEMORY_THRESHOLD_MB
If the amount of memory that is free in the system (either on the box or in the supervisor’s cgroup) is below this number (in MB) consider the system to be in low memory mode and start shooting workers if they are over their limit.
public static String STORM_SUPERVISOR_MEDIUM_MEMORY_THRESHOLD_MB
If the amount of memory that is free in the system (either on the box or in the supervisor’s cgroup) is below this number (in MB) consider the system to be a little low on memory and start shooting workers if they are over their limit for a given grace period STORM_SUPERVISOR_MEDIUM_MEMORY_GRACE_PERIOD_MS.
public static String STORM_SUPERVISOR_MEDIUM_MEMORY_GRACE_PERIOD_MS
The number of milliseconds that a worker is allowed to be over their limit when there is a medium amount of memory free in the system.
public static String STORM_WORKER_MIN_CPU_PCORE_PERCENT
The config indicates the minimum percentage of cpu for a core that a worker will use. Assuming the a core value to be 100, a value of 10 indicates 10% of the core. The P in PCORE represents the term “physical”. A default value will be set for this config if user does not override.
Workers in containers or cgroups may require a minimum amount of CPU in order to launch within the supervisor timeout. This setting allows configuring this to occur.public static String STORM_WORKER_TOKEN_LIFE_TIME_HOURS
The number of hours a worker token is valid for. This also sets how frequently worker tokens will be renewed.
Copyright © 2022 The Apache Software Foundation. All rights reserved.