org.apache.storm.kafka.spout.internal

Class CommonKafkaSpoutConfig.Builder<K,V,T extends CommonKafkaSpoutConfig.Builder<K,V,T>>

java.lang.Object

org.apache.storm.kafka.spout.internal.CommonKafkaSpoutConfig.Builder<K,V,T>

Direct Known Subclasses:

KafkaSpoutConfig.Builder, KafkaTridentSpoutConfig.Builder

Enclosing class:

CommonKafkaSpoutConfig<K,V>

public abstract static class CommonKafkaSpoutConfig.Builder<K,V,T extends CommonKafkaSpoutConfig.Builder<K,V,T>>
extends Object

Constructor Summary

Constructors

Constructor and Description
Builder(String bootstrapServers, Pattern topics)
Builder(String bootstrapServers, Set<String> topics)
Builder(String bootstrapServers, String... topics)
Builder(String bootstrapServers, TopicFilter topicFilter, ManualPartitioner topicPartitioner) Create a KafkaSpoutConfig builder with default property values and no key/value deserializers.

Method Summary

Modifier and Type	Method and Description
abstract CommonKafkaSpoutConfig<K,V>	build()
protected Map<String,Object>	getKafkaProps()
T	setFirstPollOffsetStrategy(FirstPollOffsetStrategy firstPollOffsetStrategy) Sets the offset used by the Kafka spout in the first poll to Kafka broker upon process start.
T	setPartitionRefreshPeriodMs(long partitionRefreshPeriodMs) Sets partition refresh period in milliseconds.
T	setPollTimeoutMs(long pollTimeoutMs) Specifies the time, in milliseconds, spent waiting in poll if data is not available.
T	setProp(Map<String,Object> props) Set multiple KafkaConsumer properties.
T	setProp(Properties props) Set multiple KafkaConsumer properties.
T	setProp(String key, Object value) Set a KafkaConsumer property.
T	setRecordTranslator(Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func, Fields fields) Configure a translator with tuples to be emitted on the default stream.
T	setRecordTranslator(Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func, Fields fields, String stream) Configure a translator with tuples to be emitted to a given stream.
T	setRecordTranslator(RecordTranslator<K,V> translator)
T	setStartTimeStamp(long startTimeStamp) Specifies the startTimeStamp if the first poll strategy is TIMESTAMP or UNCOMMITTED_TIMESTAMP.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Builder

public Builder(String bootstrapServers,
               String... topics)

Builder

public Builder(String bootstrapServers,
               Set<String> topics)

Builder

public Builder(String bootstrapServers,
               Pattern topics)

Builder

public Builder(String bootstrapServers,
               TopicFilter topicFilter,
               ManualPartitioner topicPartitioner)

Create a KafkaSpoutConfig builder with default property values and no key/value deserializers.

Parameters:

bootstrapServers - The bootstrap servers the consumer will use

topicFilter - The topic filter defining which topics and partitions the spout will read

topicPartitioner - The topic partitioner defining which topics and partitions are assinged to each spout task

Method Detail

setProp

public T setProp(String key,
                 Object value)

Set a KafkaConsumer property.

setProp

public T setProp(Map<String,Object> props)

Set multiple KafkaConsumer properties.

setProp

public T setProp(Properties props)

Set multiple KafkaConsumer properties.

setPollTimeoutMs

public T setPollTimeoutMs(long pollTimeoutMs)

Specifies the time, in milliseconds, spent waiting in poll if data is not available. Default is 200ms.

Parameters:

pollTimeoutMs - time in ms

setFirstPollOffsetStrategy

public T setFirstPollOffsetStrategy(FirstPollOffsetStrategy firstPollOffsetStrategy)

Sets the offset used by the Kafka spout in the first poll to Kafka broker upon process start. Please refer to to the documentation in FirstPollOffsetStrategy

Parameters:

firstPollOffsetStrategy - Offset used by Kafka spout first poll

setRecordTranslator

public T setRecordTranslator(RecordTranslator<K,V> translator)

setRecordTranslator

public T setRecordTranslator(Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func,
                             Fields fields)

Configure a translator with tuples to be emitted on the default stream.

Parameters:

func - extracts and turns a Kafka ConsumerRecord into a list of objects to be emitted

fields - the names of the fields extracted

Returns:

this to be able to chain configuration

setRecordTranslator

public T setRecordTranslator(Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func,
                             Fields fields,
                             String stream)

Configure a translator with tuples to be emitted to a given stream.

Parameters:

func - extracts and turns a Kafka ConsumerRecord into a list of objects to be emitted

fields - the names of the fields extracted

stream - the stream to emit the tuples on

Returns:

this to be able to chain configuration

setPartitionRefreshPeriodMs

public T setPartitionRefreshPeriodMs(long partitionRefreshPeriodMs)

Sets partition refresh period in milliseconds. This is how often Kafka will be polled to check for new topics and/or new partitions.

Parameters:

partitionRefreshPeriodMs - time in milliseconds

Returns:

the builder (this)

setStartTimeStamp

public T setStartTimeStamp(long startTimeStamp)

Specifies the startTimeStamp if the first poll strategy is TIMESTAMP or UNCOMMITTED_TIMESTAMP.

Parameters:

startTimeStamp - time in ms

getKafkaProps

protected Map<String,Object> getKafkaProps()

build

public abstract CommonKafkaSpoutConfig<K,V> build()