Class RocksDbMetricsWriter
java.lang.Object
org.apache.storm.metricstore.rocksdb.RocksDbMetricsWriter
- All Implemented Interfaces:
AutoCloseable
,Runnable
Class designed to perform all metrics inserts into RocksDB. Metrics are processed from a blocking queue. Inserts
to RocksDB are done using a single thread to simplify design (such as looking up existing metric data for aggregation,
and fetching/evicting metadata from the cache). This class is not thread safe.
A writable LRU StringMetadataCache is used to minimize looking up metadata string Ids. As entries are added to the full cache, older
entries are evicted from the cache and need to be written to the database. This happens as the handleEvictedMetadata()
method callback.
The following issues would need to be addressed to implement a multithreaded metrics writer:
- Generation of unique unused IDs for new metadata strings needs to be thread safe.
- Ensuring newly created metadata strings are seen by all threads.
- Maintaining a properly cached state of metadata for multiple writers. The current LRU cache evicts data as new metadata is added.
- Processing the aggregation of a metric requires fetching and updating previous aggregates. A multithreaded design would need to ensure two metrics were not updating an aggregated metric at the same time.
- Investigate performance of multiple threads inserting into RocksDB versus a single ordered insert.
-
Method Summary
-
Method Details
-
run
public void run()Run routine to wait for metrics on a queue and insert into RocksDB. -
close
public void close()- Specified by:
close
in interfaceAutoCloseable
-