Package org.apache.storm.hdfs.blobstore
Class HdfsClientBlobStore
java.lang.Object
org.apache.storm.blobstore.ClientBlobStore
org.apache.storm.hdfs.blobstore.HdfsClientBlobStore
- All Implemented Interfaces:
AutoCloseable
,Shutdownable
Client to access the HDFS blobStore. At this point, this is meant to only be used by the
supervisor. Don't trust who the client says they are so pass null for all Subjects.
The HdfsBlobStore implementation takes care of the null Subjects. It assigns Subjects based on what hadoop says who the users are. These users must be configured accordingly in the SUPERVISOR_ADMINS for ACL validation and for the supervisors to download the blobs. This API is only used by the supervisor in order to talk directly to HDFS.
-
Nested Class Summary
Nested classes/interfaces inherited from class org.apache.storm.blobstore.ClientBlobStore
ClientBlobStore.WithBlobstore
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionvoid
close()
createBlobToExtend
(String key, SettableBlobMeta meta) Client facing API to create a blob.void
Creates state inside a zookeeper.void
deleteBlob
(String key) Client facing API to delete a blob.Client facing API to read a blob.getBlobMeta
(String key) Client facing API to read the metadata information.int
getBlobReplication
(String key) Client facing API to read the replication of a blob.long
Client facing API to get the last update time of existing blobs in a blobstore.boolean
isRemoteBlobExists
(String blobKey) Decide if the blob is deleted from cluster.listKeys()
List keys.void
Sets up the client API by parsing the configs.void
setBlobMetaToExtend
(String key, SettableBlobMeta meta) Client facing API to set the metadata for a blob.boolean
setClient
(Map<String, Object> conf, NimbusClient client) Client facing API to set a nimbus client.void
shutdown()
updateBlob
(String key) Client facing API to update a blob.int
updateBlobReplication
(String key, int replication) Client facing API to update the replication of a blob.Methods inherited from class org.apache.storm.blobstore.ClientBlobStore
createBlob, setBlobMeta, withConfiguredClient
-
Constructor Details
-
HdfsClientBlobStore
public HdfsClientBlobStore()
-
-
Method Details
-
prepare
Description copied from class:ClientBlobStore
Sets up the client API by parsing the configs.- Specified by:
prepare
in classClientBlobStore
- Parameters:
conf
- The storm conf containing the config details
-
createBlobToExtend
public AtomicOutputStream createBlobToExtend(String key, SettableBlobMeta meta) throws AuthorizationException, KeyAlreadyExistsException Description copied from class:ClientBlobStore
Client facing API to create a blob.- Specified by:
createBlobToExtend
in classClientBlobStore
- Parameters:
key
- blob key namemeta
- contains ACL information- Returns:
- AtomicOutputStream returns an output stream into which data can be written
- Throws:
AuthorizationException
KeyAlreadyExistsException
-
updateBlob
public AtomicOutputStream updateBlob(String key) throws AuthorizationException, KeyNotFoundException Description copied from class:ClientBlobStore
Client facing API to update a blob.- Specified by:
updateBlob
in classClientBlobStore
- Parameters:
key
- blob key name- Returns:
- AtomicOutputStream returns an output stream into which data can be written
- Throws:
AuthorizationException
KeyNotFoundException
-
getBlobMeta
Description copied from class:ClientBlobStore
Client facing API to read the metadata information.- Specified by:
getBlobMeta
in classClientBlobStore
- Parameters:
key
- blob key name- Returns:
- AtomicOutputStream returns an output stream into which data can be written
- Throws:
AuthorizationException
KeyNotFoundException
-
isRemoteBlobExists
Description copied from class:ClientBlobStore
Decide if the blob is deleted from cluster.- Specified by:
isRemoteBlobExists
in classClientBlobStore
- Parameters:
blobKey
- blob key- Throws:
AuthorizationException
-
setBlobMetaToExtend
public void setBlobMetaToExtend(String key, SettableBlobMeta meta) throws AuthorizationException, KeyNotFoundException Description copied from class:ClientBlobStore
Client facing API to set the metadata for a blob.- Specified by:
setBlobMetaToExtend
in classClientBlobStore
- Parameters:
key
- blob key namemeta
- contains ACL information- Throws:
AuthorizationException
KeyNotFoundException
-
deleteBlob
Description copied from class:ClientBlobStore
Client facing API to delete a blob.- Specified by:
deleteBlob
in classClientBlobStore
- Parameters:
key
- blob key name- Throws:
AuthorizationException
KeyNotFoundException
-
getBlob
Description copied from class:ClientBlobStore
Client facing API to read a blob.- Specified by:
getBlob
in classClientBlobStore
- Parameters:
key
- blob key name- Returns:
- an InputStream to read the metadata for a blob
- Throws:
AuthorizationException
KeyNotFoundException
-
listKeys
Description copied from class:ClientBlobStore
List keys.- Specified by:
listKeys
in classClientBlobStore
- Returns:
- Iterator for a list of keys currently present in the blob store.
-
getBlobReplication
Description copied from class:ClientBlobStore
Client facing API to read the replication of a blob.- Specified by:
getBlobReplication
in classClientBlobStore
- Parameters:
key
- blob key name- Returns:
- int indicates the replication factor of a blob
- Throws:
AuthorizationException
KeyNotFoundException
-
updateBlobReplication
public int updateBlobReplication(String key, int replication) throws AuthorizationException, KeyNotFoundException Description copied from class:ClientBlobStore
Client facing API to update the replication of a blob.- Specified by:
updateBlobReplication
in classClientBlobStore
- Parameters:
key
- blob key namereplication
- int indicates the replication factor a blob has to be set- Returns:
- int indicates the replication factor of a blob
- Throws:
AuthorizationException
KeyNotFoundException
-
setClient
Description copied from class:ClientBlobStore
Client facing API to set a nimbus client.- Specified by:
setClient
in classClientBlobStore
- Parameters:
conf
- storm confclient
- NimbusClient- Returns:
- indicates where the client connection has been setup.
-
createStateInZookeeper
Description copied from class:ClientBlobStore
Creates state inside a zookeeper. Required for blobstore to write to zookeeper when Nimbus HA is turned on in order to maintain state consistency.- Specified by:
createStateInZookeeper
in classClientBlobStore
-
shutdown
public void shutdown() -
close
public void close()- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in classClientBlobStore
-
getRemoteBlobstoreUpdateTime
Description copied from class:ClientBlobStore
Client facing API to get the last update time of existing blobs in a blobstore. This is only required for use on supervisors.- Specified by:
getRemoteBlobstoreUpdateTime
in classClientBlobStore
- Returns:
- the timestamp of when the blobstore was last updated. -1L if the blobstore does not support this.
- Throws:
IOException
-