Package org.apache.storm.kafka.spout
Class KafkaSpoutConfig.Builder<K,V>
java.lang.Object
org.apache.storm.kafka.spout.internal.CommonKafkaSpoutConfig.Builder<K,V,KafkaSpoutConfig.Builder<K,V>>
org.apache.storm.kafka.spout.KafkaSpoutConfig.Builder<K,V>
- Enclosing class:
- KafkaSpoutConfig<K,
V>
public static class KafkaSpoutConfig.Builder<K,V>
extends CommonKafkaSpoutConfig.Builder<K,V,KafkaSpoutConfig.Builder<K,V>>
-
Constructor Summary
ConstructorDescriptionBuilder
(String bootstrapServers, TopicFilter topicFilter, ManualPartitioner topicPartitioner) Create a KafkaSpoutConfig builder with default property values and no key/value deserializers. -
Method Summary
Modifier and TypeMethodDescriptionbuild()
setEmitNullTuples
(boolean emitNullTuples) Specifies if the spout should emit null tuples to the component downstream, or rather not emit and directly ack them.setGroupId
(String groupId) Specifies the group id.setMaxUncommittedOffsets
(int maxUncommittedOffsets) Defines the max number of polled offsets (records) that can be pending commit, before another poll can take place.setMetricsTimeBucketSizeInSecs
(int metricsTimeBucketSizeInSecs) The time period that metrics data in bucketed into.setOffsetCommitPeriodMs
(long offsetCommitPeriodMs) Specifies the period, in milliseconds, the offset commit task is periodically called.setProcessingGuarantee
(KafkaSpoutConfig.ProcessingGuarantee processingGuarantee) Specifies which processing guarantee the spout should offer.setRetry
(KafkaSpoutRetryService retryService) Sets the retry service for the spout to use.setTupleListener
(KafkaTupleListener tupleListener) Sets the tuple listener for the spout to use.setTupleTrackingEnforced
(boolean tupleTrackingEnforced) Specifies whether the spout should require Storm to track emitted tuples when using aKafkaSpoutConfig.ProcessingGuarantee
other thanKafkaSpoutConfig.ProcessingGuarantee.AT_LEAST_ONCE
.Methods inherited from class org.apache.storm.kafka.spout.internal.CommonKafkaSpoutConfig.Builder
getKafkaProps, setFirstPollOffsetStrategy, setPartitionRefreshPeriodMs, setPollTimeoutMs, setProp, setProp, setProp, setRecordTranslator, setRecordTranslator, setRecordTranslator, setStartTimeStamp
-
Constructor Details
-
Builder
-
Builder
-
Builder
-
Builder
public Builder(String bootstrapServers, TopicFilter topicFilter, ManualPartitioner topicPartitioner) Create a KafkaSpoutConfig builder with default property values and no key/value deserializers.- Parameters:
bootstrapServers
- The bootstrap servers the consumer will usetopicFilter
- The topic filter defining which topics and partitions the spout will readtopicPartitioner
- The topic partitioner defining which topics and partitions are assinged to each spout task
-
-
Method Details
-
setOffsetCommitPeriodMs
Specifies the period, in milliseconds, the offset commit task is periodically called. Default is 15s.This setting only has an effect if the configured
KafkaSpoutConfig.ProcessingGuarantee
isKafkaSpoutConfig.ProcessingGuarantee.AT_LEAST_ONCE
orKafkaSpoutConfig.ProcessingGuarantee.NO_GUARANTEE
.- Parameters:
offsetCommitPeriodMs
- time in ms
-
setGroupId
Specifies the group id.- Parameters:
groupId
- the group id
-
setMaxUncommittedOffsets
Defines the max number of polled offsets (records) that can be pending commit, before another poll can take place. Once this limit is reached, no more offsets (records) can be polled until the next successful commit(s) sets the number of pending offsets below the threshold. The default isKafkaSpoutConfig.DEFAULT_MAX_UNCOMMITTED_OFFSETS
. This limit is per partition and may in some cases be exceeded, but each partition cannot exceed this limit by more than maxPollRecords - 1.This setting only has an effect if the configured
KafkaSpoutConfig.ProcessingGuarantee
isKafkaSpoutConfig.ProcessingGuarantee.AT_LEAST_ONCE
.- Parameters:
maxUncommittedOffsets
- max number of records that can be be pending commit
-
setRetry
Sets the retry service for the spout to use.This setting only has an effect if the configured
KafkaSpoutConfig.ProcessingGuarantee
isKafkaSpoutConfig.ProcessingGuarantee.AT_LEAST_ONCE
.- Parameters:
retryService
- the new retry service- Returns:
- the builder (this).
-
setTupleListener
Sets the tuple listener for the spout to use.- Parameters:
tupleListener
- the tuple listener- Returns:
- the builder (this).
-
setEmitNullTuples
Specifies if the spout should emit null tuples to the component downstream, or rather not emit and directly ack them. By default this parameter is set to false, which means that null tuples are not emitted.- Parameters:
emitNullTuples
- sets if null tuples should or not be emitted downstream
-
setProcessingGuarantee
public KafkaSpoutConfig.Builder<K,V> setProcessingGuarantee(KafkaSpoutConfig.ProcessingGuarantee processingGuarantee) Specifies which processing guarantee the spout should offer. Refer to the documentation forKafkaSpoutConfig.ProcessingGuarantee
.- Parameters:
processingGuarantee
- The processing guarantee the spout should offer.
-
setTupleTrackingEnforced
Specifies whether the spout should require Storm to track emitted tuples when using aKafkaSpoutConfig.ProcessingGuarantee
other thanKafkaSpoutConfig.ProcessingGuarantee.AT_LEAST_ONCE
. The spout will always track emitted tuples when offering at-least-once guarantees regardless of this setting. This setting is false by default.Enabling tracking can be useful even in cases where reliability is not a concern, because it allows
Config.TOPOLOGY_MAX_SPOUT_PENDING
to have an effect, and enables some spout metrics (e.g. complete-latency) that would otherwise be disabled.- Parameters:
tupleTrackingEnforced
- true if Storm should track emitted tuples, false otherwise
-
setMetricsTimeBucketSizeInSecs
public KafkaSpoutConfig.Builder<K,V> setMetricsTimeBucketSizeInSecs(int metricsTimeBucketSizeInSecs) The time period that metrics data in bucketed into.- Parameters:
metricsTimeBucketSizeInSecs
- time in seconds
-
build
- Specified by:
build
in classCommonKafkaSpoutConfig.Builder<K,
V, KafkaSpoutConfig.Builder<K, V>>
-