Package org.apache.storm.localizer
Class AsyncLocalizer
java.lang.Object
org.apache.storm.localizer.AsyncLocalizer
- All Implemented Interfaces:
AutoCloseable
Downloads and caches blobs locally.
-
Field Summary
Modifier and TypeFieldDescriptionprotected long
protected final ConcurrentHashMap<String,
ConcurrentHashMap<String, LocalizedResource>> protected final ConcurrentHashMap<String,
ConcurrentHashMap<String, LocalizedResource>> -
Constructor Summary
ConstructorDescriptionAsyncLocalizer
(Map<String, Object> conf, StormMetricsRegistry metricsRegistry) -
Method Summary
Modifier and TypeMethodDescriptionvoid
close()
protected ClientBlobStore
void
recoverRunningTopology
(LocalAssignment currentAssignment, int port, BlobChangingCallback cb) Do everything needed to recover the state in the AsyncLocalizer for a running topology.void
releaseSlotFor
(LocalAssignment assignment, int port) Remove this assignment/port as blocking resources from being cleaned up.requestDownloadTopologyBlobs
(LocalAssignment assignment, int port, BlobChangingCallback cb) Request that all of the blobs necessary for this topology be downloaded.void
start()
Start any background threads needed.
-
Field Details
-
userFiles
-
userArchives
-
cacheTargetSize
protected long cacheTargetSize
-
-
Constructor Details
-
AsyncLocalizer
public AsyncLocalizer(Map<String, Object> conf, StormMetricsRegistry metricsRegistry) throws IOException- Throws:
IOException
-
-
Method Details
-
requestDownloadTopologyBlobs
public CompletableFuture<Void> requestDownloadTopologyBlobs(LocalAssignment assignment, int port, BlobChangingCallback cb) throws IOException Request that all of the blobs necessary for this topology be downloaded. Note that this adds references to blobs asynchronously in background threads.- Parameters:
assignment
- the assignment that needs the blobsport
- the port the assignment is a part ofcb
- a callback for when the blobs change. This is only for blobs that are tied to the lifetime of the worker.- Returns:
- a Future that indicates when they are all downloaded.
- Throws:
IOException
- if there was an error while trying doing it.
-
start
public void start()Start any background threads needed. This includes updating blobs and cleaning up unused blobs over the configured size limit. -
close
- Specified by:
close
in interfaceAutoCloseable
- Throws:
InterruptedException
-
recoverRunningTopology
public void recoverRunningTopology(LocalAssignment currentAssignment, int port, BlobChangingCallback cb) throws IOException Do everything needed to recover the state in the AsyncLocalizer for a running topology.- Parameters:
currentAssignment
- the assignment for the topology.port
- the port the assignment is on.cb
- a callback for when the blobs are updated. This will only be for blobs that indicate that if they change the worker should be restarted.- Throws:
IOException
- on any error trying to recover the state.
-
releaseSlotFor
Remove this assignment/port as blocking resources from being cleaned up.- Parameters:
assignment
- the assignment the resources are forport
- the port the topology is running on- Throws:
IOException
- on any error
-
getClientBlobStore
-