Package org.apache.storm.kafka.spout.internal

Class CommonKafkaSpoutConfig.Builder<K,V,T extends CommonKafkaSpoutConfig.Builder<K,V,T>>

java.lang.Object
org.apache.storm.kafka.spout.internal.CommonKafkaSpoutConfig.Builder<K,V,T>
Direct Known Subclasses:
KafkaSpoutConfig.Builder, KafkaTridentSpoutConfig.Builder
Enclosing class:
CommonKafkaSpoutConfig<K,V>
public abstract static class CommonKafkaSpoutConfig.Builder<K,V,T extends CommonKafkaSpoutConfig.Builder<K,V,T>> extends Object

  • Constructor Details

    • Builder

      public Builder(String bootstrapServers, String... topics)

    • Builder

      public Builder(String bootstrapServers, Set<String> topics)

    • Builder

      public Builder(String bootstrapServers, Pattern topics)

    • Builder

      public Builder(String bootstrapServers, TopicFilter topicFilter, ManualPartitioner topicPartitioner)
      Create a KafkaSpoutConfig builder with default property values and no key/value deserializers.
      Parameters:
      bootstrapServers - The bootstrap servers the consumer will use
      topicFilter - The topic filter defining which topics and partitions the spout will read
      topicPartitioner - The topic partitioner defining which topics and partitions are assinged to each spout task

  • Method Details

    • setProp

      public T setProp(String key, Object value)
      Set a KafkaConsumer property.

    • setProp

      public T setProp(Map<String,Object> props)
      Set multiple KafkaConsumer properties.

    • setProp

      public T setProp(Properties props)
      Set multiple KafkaConsumer properties.

    • setPollTimeoutMs

      public T setPollTimeoutMs(long pollTimeoutMs)
      Specifies the time, in milliseconds, spent waiting in poll if data is not available. Default is 200ms.
      Parameters:
      pollTimeoutMs - time in ms

    • setFirstPollOffsetStrategy

      public T setFirstPollOffsetStrategy(FirstPollOffsetStrategy firstPollOffsetStrategy)
      Sets the offset used by the Kafka spout in the first poll to Kafka broker upon process start. Please refer to to the documentation in FirstPollOffsetStrategy
      Parameters:
      firstPollOffsetStrategy - Offset used by Kafka spout first poll

    • setRecordTranslator

      public T setRecordTranslator(RecordTranslator<K,V> translator)

    • setRecordTranslator

      public T setRecordTranslator(Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func, Fields fields)
      Configure a translator with tuples to be emitted on the default stream.
      Parameters:
      func - extracts and turns a Kafka ConsumerRecord into a list of objects to be emitted
      fields - the names of the fields extracted
      Returns:
      this to be able to chain configuration

    • setRecordTranslator

      public T setRecordTranslator(Func<org.apache.kafka.clients.consumer.ConsumerRecord<K,V>,List<Object>> func, Fields fields, String stream)
      Configure a translator with tuples to be emitted to a given stream.
      Parameters:
      func - extracts and turns a Kafka ConsumerRecord into a list of objects to be emitted
      fields - the names of the fields extracted
      stream - the stream to emit the tuples on
      Returns:
      this to be able to chain configuration

    • setPartitionRefreshPeriodMs

      public T setPartitionRefreshPeriodMs(long partitionRefreshPeriodMs)
      Sets partition refresh period in milliseconds. This is how often Kafka will be polled to check for new topics and/or new partitions.
      Parameters:
      partitionRefreshPeriodMs - time in milliseconds
      Returns:
      the builder (this)

    • setStartTimeStamp

      public T setStartTimeStamp(long startTimeStamp)
      Specifies the startTimeStamp if the first poll strategy is TIMESTAMP or UNCOMMITTED_TIMESTAMP.
      Parameters:
      startTimeStamp - time in ms

    • getKafkaProps

      protected Map<String,Object> getKafkaProps()

    • build

      public abstract CommonKafkaSpoutConfig<K,V> build()