org.apache.storm.metricstore.rocksdb

Class RocksDbMetricsWriter

java.lang.Object

org.apache.storm.metricstore.rocksdb.RocksDbMetricsWriter

All Implemented Interfaces:

AutoCloseable, Runnable

public class RocksDbMetricsWriter
extends Object
implements Runnable, AutoCloseable

Class designed to perform all metrics inserts into RocksDB. Metrics are processed from a blocking queue. Inserts to RocksDB are done using a single thread to simplify design (such as looking up existing metric data for aggregation, and fetching/evicting metadata from the cache). This class is not thread safe.

A writable LRU StringMetadataCache is used to minimize looking up metadata string Ids. As entries are added to the full cache, older entries are evicted from the cache and need to be written to the database. This happens as the handleEvictedMetadata() method callback.

The following issues would need to be addressed to implement a multithreaded metrics writer:

• Generation of unique unused IDs for new metadata strings needs to be thread safe.
• Ensuring newly created metadata strings are seen by all threads.
• Maintaining a properly cached state of metadata for multiple writers. The current LRU cache evicts data as new metadata is added.
• Processing the aggregation of a metric requires fetching and updating previous aggregates. A multithreaded design would need to ensure two metrics were not updating an aggregated metric at the same time.
• Investigate performance of multiple threads inserting into RocksDB versus a single ordered insert.

Method Summary

Modifier and Type	Method and Description
void	close()
void	run() Run routine to wait for metrics on a queue and insert into RocksDB.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

run

public void run()

Run routine to wait for metrics on a queue and insert into RocksDB.

Specified by:

run in interface Runnable

close

public void close()

Specified by:

close in interface AutoCloseable