org.apache.storm.starter

Class ReachTopology

java.lang.Object

org.apache.storm.starter.ReachTopology

public class ReachTopology
extends Object

This is a good example of doing complex Distributed RPC on top of Storm. This program creates a topology that can compute the reach for any URL on Twitter in realtime by parallelizing the whole computation.

Reach is the number of unique people exposed to a URL on Twitter. To compute reach, you have to get all the people who tweeted the URL, get all the followers of all those people, unique that set of followers, and then count the unique set. It’s an intense computation that can involve thousands of database calls and tens of millions of follower records.

This Storm topology does every piece of that computation in parallel, turning what would be a computation that takes minutes on a single machine into one that takes just a couple seconds.

For the purposes of demonstration, this topology replaces the use of actual DBs with in-memory hashmaps.

See Also:

Distributed RPC

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ReachTopology.CountAggregator
static class	ReachTopology.GetFollowers
static class	ReachTopology.GetTweeters
static class	ReachTopology.PartialUniquer

Field Summary

Fields

Modifier and Type	Field and Description
static Map<String,List<String>>	FOLLOWERS_DB
static Map<String,List<String>>	TWEETERS_DB

Constructor Summary

Constructors

Constructor and Description
ReachTopology()

Method Summary

Modifier and Type	Method and Description
static LinearDRPCTopologyBuilder	construct()
static void	main(String[] args)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TWEETERS_DB

public static Map<String,List<String>> TWEETERS_DB

FOLLOWERS_DB

public static Map<String,List<String>> FOLLOWERS_DB

Constructor Detail

ReachTopology

public ReachTopology()

Method Detail

construct

public static LinearDRPCTopologyBuilder construct()

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception