org.apache.storm.kafka.spout

Class KafkaSpoutConfig<K,V>

java.lang.Object

org.apache.storm.kafka.spout.KafkaSpoutConfig<K,V>

All Implemented Interfaces:

Serializable

public class KafkaSpoutConfig<K,V>
extends Object
implements Serializable

KafkaSpoutConfig defines the required configuration to connect a consumer to a consumer group, as well as the subscribing topics

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	KafkaSpoutConfig.Builder<K,V>
static class	KafkaSpoutConfig.FirstPollOffsetStrategy Defines how the KafkaSpout seeks the offset to be used in the first poll to Kafka upon topology deployment.
static class	KafkaSpoutConfig.ProcessingGuarantee This enum controls when the tuple with the ConsumerRecord for an offset is marked as processed, i.e.

Field Summary

Fields

Modifier and Type	Field and Description
static KafkaSpoutConfig.FirstPollOffsetStrategy	DEFAULT_FIRST_POLL_OFFSET_STRATEGY
static int	DEFAULT_MAX_RETRIES
static int	DEFAULT_MAX_UNCOMMITTED_OFFSETS
static int	DEFAULT_METRICS_TIME_BUCKET_SIZE_SECONDS
static long	DEFAULT_OFFSET_COMMIT_PERIOD_MS
static long	DEFAULT_PARTITION_REFRESH_PERIOD_MS
static long	DEFAULT_POLL_TIMEOUT_MS
static KafkaSpoutConfig.ProcessingGuarantee	DEFAULT_PROCESSING_GUARANTEE
static KafkaSpoutRetryService	DEFAULT_RETRY_SERVICE
static KafkaTupleListener	DEFAULT_TUPLE_LISTENER

Constructor Summary

Constructors

Constructor and Description
KafkaSpoutConfig(KafkaSpoutConfig.Builder<K,V> builder) Creates a new KafkaSpoutConfig using a Builder.

Method Summary

Modifier and Type	Method and Description
static KafkaSpoutConfig.Builder<String,String>	builder(String bootstrapServers, Collection<String> topics) Factory method that creates a Builder with String key/value deserializers.
static KafkaSpoutConfig.Builder<String,String>	builder(String bootstrapServers, Pattern topics) Factory method that creates a Builder with String key/value deserializers.
static KafkaSpoutConfig.Builder<String,String>	builder(String bootstrapServers, String... topics) Factory method that creates a Builder with String key/value deserializers.
String	getConsumerGroupId()
KafkaSpoutConfig.FirstPollOffsetStrategy	getFirstPollOffsetStrategy()
Map<String,Object>	getKafkaProps() Gets the properties that will be passed to the KafkaConsumer.
org.apache.kafka.common.serialization.Deserializer<K>	getKeyDeserializer() Deprecated. Please use getKafkaProps() and look up the entry for ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG instead
int	getMaxUncommittedOffsets()
int	getMetricsTimeBucketSizeInSecs()
long	getOffsetsCommitPeriodMs()
long	getPartitionRefreshPeriodMs()
long	getPollTimeoutMs()
KafkaSpoutConfig.ProcessingGuarantee	getProcessingGuarantee()
KafkaSpoutRetryService	getRetryService()
Subscription	getSubscription()
RecordTranslator<K,V>	getTranslator()
KafkaTupleListener	getTupleListener()
org.apache.kafka.common.serialization.Deserializer<V>	getValueDeserializer() Deprecated. Please use getKafkaProps() and look up the entry for ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG instead
boolean	isConsumerAutoCommitMode() Deprecated. Use getProcessingGuarantee() instead.
boolean	isEmitNullTuples()
boolean	isTupleTrackingEnforced()
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_POLL_TIMEOUT_MS

public static final long DEFAULT_POLL_TIMEOUT_MS

See Also:

Constant Field Values

DEFAULT_OFFSET_COMMIT_PERIOD_MS

public static final long DEFAULT_OFFSET_COMMIT_PERIOD_MS

See Also:

Constant Field Values

DEFAULT_MAX_RETRIES

public static final int DEFAULT_MAX_RETRIES

See Also:

Constant Field Values

DEFAULT_MAX_UNCOMMITTED_OFFSETS

public static final int DEFAULT_MAX_UNCOMMITTED_OFFSETS

See Also:

Constant Field Values

DEFAULT_PARTITION_REFRESH_PERIOD_MS

public static final long DEFAULT_PARTITION_REFRESH_PERIOD_MS

See Also:

Constant Field Values

DEFAULT_FIRST_POLL_OFFSET_STRATEGY

public static final KafkaSpoutConfig.FirstPollOffsetStrategy DEFAULT_FIRST_POLL_OFFSET_STRATEGY

DEFAULT_RETRY_SERVICE

public static final KafkaSpoutRetryService DEFAULT_RETRY_SERVICE

DEFAULT_PROCESSING_GUARANTEE

public static final KafkaSpoutConfig.ProcessingGuarantee DEFAULT_PROCESSING_GUARANTEE

DEFAULT_TUPLE_LISTENER

public static final KafkaTupleListener DEFAULT_TUPLE_LISTENER

DEFAULT_METRICS_TIME_BUCKET_SIZE_SECONDS

public static final int DEFAULT_METRICS_TIME_BUCKET_SIZE_SECONDS

See Also:

Constant Field Values

Constructor Detail

KafkaSpoutConfig

public KafkaSpoutConfig(KafkaSpoutConfig.Builder<K,V> builder)

Creates a new KafkaSpoutConfig using a Builder.

Parameters:

builder - The Builder to construct the KafkaSpoutConfig from

Method Detail

builder

public static KafkaSpoutConfig.Builder<String,String> builder(String bootstrapServers,
                                                              String... topics)

Factory method that creates a Builder with String key/value deserializers.

Parameters:

bootstrapServers - The bootstrap servers for the consumer

topics - The topics to subscribe to

Returns:

The new builder

builder

public static KafkaSpoutConfig.Builder<String,String> builder(String bootstrapServers,
                                                              Collection<String> topics)

Factory method that creates a Builder with String key/value deserializers.

Parameters:

bootstrapServers - The bootstrap servers for the consumer

topics - The topics to subscribe to

Returns:

The new builder

builder

public static KafkaSpoutConfig.Builder<String,String> builder(String bootstrapServers,
                                                              Pattern topics)

Factory method that creates a Builder with String key/value deserializers.

Parameters:

bootstrapServers - The bootstrap servers for the consumer

topics - The topic pattern to subscribe to

Returns:

The new builder

getKafkaProps

public Map<String,Object> getKafkaProps()

Gets the properties that will be passed to the KafkaConsumer.

Returns:

The Kafka properties map

getKeyDeserializer

@Deprecated
public org.apache.kafka.common.serialization.Deserializer<K> getKeyDeserializer()

Deprecated. Please use getKafkaProps() and look up the entry for ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG instead

getValueDeserializer

@Deprecated
public org.apache.kafka.common.serialization.Deserializer<V> getValueDeserializer()

Deprecated. Please use getKafkaProps() and look up the entry for ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG instead

getSubscription

public Subscription getSubscription()

getTranslator

public RecordTranslator<K,V> getTranslator()

getPollTimeoutMs

public long getPollTimeoutMs()

getOffsetsCommitPeriodMs

public long getOffsetsCommitPeriodMs()

isConsumerAutoCommitMode

@Deprecated
public boolean isConsumerAutoCommitMode()

Deprecated. Use getProcessingGuarantee() instead.

getProcessingGuarantee

public KafkaSpoutConfig.ProcessingGuarantee getProcessingGuarantee()

isTupleTrackingEnforced

public boolean isTupleTrackingEnforced()

getConsumerGroupId

public String getConsumerGroupId()

getFirstPollOffsetStrategy

public KafkaSpoutConfig.FirstPollOffsetStrategy getFirstPollOffsetStrategy()

getMaxUncommittedOffsets

public int getMaxUncommittedOffsets()

getRetryService

public KafkaSpoutRetryService getRetryService()

getTupleListener

public KafkaTupleListener getTupleListener()

getPartitionRefreshPeriodMs

public long getPartitionRefreshPeriodMs()

isEmitNullTuples

public boolean isEmitNullTuples()

getMetricsTimeBucketSizeInSecs

public int getMetricsTimeBucketSizeInSecs()

toString

public String toString()

Overrides:

toString in class Object