org.apache.storm.kafka.spout.trident

Class KafkaTridentSpoutEmitter<K,V>

java.lang.Object

org.apache.storm.kafka.spout.trident.KafkaTridentSpoutEmitter<K,V>

All Implemented Interfaces:

Serializable, IOpaquePartitionedTridentSpout.Emitter<List<Map<String,Object>>,KafkaTridentSpoutTopicPartition,Map<String,Object>>

public class KafkaTridentSpoutEmitter<K,V>
extends Object
implements IOpaquePartitionedTridentSpout.Emitter<List<Map<String,Object>>,KafkaTridentSpoutTopicPartition,Map<String,Object>>, Serializable

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
KafkaTridentSpoutEmitter(KafkaTridentSpoutManager<K,V> kafkaManager, TopologyContext topologyContext) Creates instance of this class with default 500 millisecond refresh subscription timer
KafkaTridentSpoutEmitter(KafkaTridentSpoutManager<K,V> kafkaManager, TopologyContext topologyContext, Timer refreshSubscriptionTimer) Create a new Kafka spout emitter.

Method Summary

Modifier and Type	Method and Description
void	close()
Map<String,Object>	emitPartitionBatch(TransactionAttempt tx, TridentCollector collector, KafkaTridentSpoutTopicPartition currBatchPartition, Map<String,Object> lastBatch) Emit a batch of tuples for a partition/transaction.
List<KafkaTridentSpoutTopicPartition>	getOrderedPartitions(List<Map<String,Object>> allPartitionInfo) Computes ordered list of topic-partitions for this task taking into consideration that topic-partitions for this task must be assigned to the Kafka consumer running on this task.
List<KafkaTridentSpoutTopicPartition>	getPartitionsForTask(int taskId, int numTasks, List<KafkaTridentSpoutTopicPartition> allPartitionInfo)
void	refreshPartitions(List<KafkaTridentSpoutTopicPartition> partitionResponsibilities) This method is called when this task is responsible for a new set of partitions.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

KafkaTridentSpoutEmitter

public KafkaTridentSpoutEmitter(KafkaTridentSpoutManager<K,V> kafkaManager,
                                TopologyContext topologyContext,
                                Timer refreshSubscriptionTimer)

Create a new Kafka spout emitter.

Parameters:

kafkaManager - The Kafka consumer manager to use

topologyContext - The topology context

refreshSubscriptionTimer - The timer for deciding when to recheck the subscription

KafkaTridentSpoutEmitter

public KafkaTridentSpoutEmitter(KafkaTridentSpoutManager<K,V> kafkaManager,
                                TopologyContext topologyContext)

Creates instance of this class with default 500 millisecond refresh subscription timer

Method Detail

emitPartitionBatch

public Map<String,Object> emitPartitionBatch(TransactionAttempt tx,
                                             TridentCollector collector,
                                             KafkaTridentSpoutTopicPartition currBatchPartition,
                                             Map<String,Object> lastBatch)

Description copied from interface: IOpaquePartitionedTridentSpout.Emitter

Emit a batch of tuples for a partition/transaction.

Return the metadata describing this batch that will be used as lastPartitionMeta for defining the parameters of the next batch.

Specified by:

emitPartitionBatch in interface IOpaquePartitionedTridentSpout.Emitter<List<Map<String,Object>>,KafkaTridentSpoutTopicPartition,Map<String,Object>>

refreshPartitions

public void refreshPartitions(List<KafkaTridentSpoutTopicPartition> partitionResponsibilities)

Description copied from interface: IOpaquePartitionedTridentSpout.Emitter

This method is called when this task is responsible for a new set of partitions. Should be used to manage things like connections to brokers.

Specified by:

refreshPartitions in interface IOpaquePartitionedTridentSpout.Emitter<List<Map<String,Object>>,KafkaTridentSpoutTopicPartition,Map<String,Object>>

getOrderedPartitions

public List<KafkaTridentSpoutTopicPartition> getOrderedPartitions(List<Map<String,Object>> allPartitionInfo)

Computes ordered list of topic-partitions for this task taking into consideration that topic-partitions for this task must be assigned to the Kafka consumer running on this task.

Specified by:

getOrderedPartitions in interface IOpaquePartitionedTridentSpout.Emitter<List<Map<String,Object>>,KafkaTridentSpoutTopicPartition,Map<String,Object>>

Parameters:

allPartitionInfo - list of all partitions as returned by KafkaTridentSpoutOpaqueCoordinator

Returns:

ordered list of topic partitions for this task

getPartitionsForTask

public List<KafkaTridentSpoutTopicPartition> getPartitionsForTask(int taskId,
                                                                  int numTasks,
                                                                  List<KafkaTridentSpoutTopicPartition> allPartitionInfo)

Specified by:

getPartitionsForTask in interface IOpaquePartitionedTridentSpout.Emitter<List<Map<String,Object>>,KafkaTridentSpoutTopicPartition,Map<String,Object>>

Returns:

The list of partitions that are to be processed by the task with id taskId

close

public void close()

Specified by:

close in interface IOpaquePartitionedTridentSpout.Emitter<List<Map<String,Object>>,KafkaTridentSpoutTopicPartition,Map<String,Object>>

toString

public final String toString()

Overrides:

toString in class Object