Package org.apache.storm.hdfs.blobstore
Class HdfsBlobStoreImpl
java.lang.Object
org.apache.storm.hdfs.blobstore.HdfsBlobStoreImpl
HDFS blob store impl.
-
Nested Class Summary
-
Field Summary
Modifier and TypeFieldDescriptionstatic final org.apache.hadoop.fs.permission.FsPermission
-
Constructor Summary
ConstructorDescriptionHdfsBlobStoreImpl
(org.apache.hadoop.fs.Path path, Map<String, Object> conf) HdfsBlobStoreImpl
(org.apache.hadoop.fs.Path path, Map<String, Object> conf, org.apache.hadoop.conf.Configuration hconf) -
Method Summary
Modifier and TypeMethodDescriptionprotected void
delete
(org.apache.hadoop.fs.Path path) void
Delete a key from the blob store.boolean
Check if the key exists in the blob store.void
fullCleanup
(long age) protected int
getBlobReplication
(String key) protected org.apache.hadoop.fs.Path
long
Get the last update time of any blob.protected Iterator<BlobStoreFile>
listBlobStoreFiles
(org.apache.hadoop.fs.Path path) listKeys()
List relevant keys.listKeys
(org.apache.hadoop.fs.Path path) Get an input stream for reading a part.void
shutdown()
protected int
updateBlobReplication
(String key, int replication) void
Updates the last updated time of existing blobstores to the current time.void
Validates that the last updated blob time of the blobstore is up to date with the current existing blobs.Get an object tied to writing the data.
-
Field Details
-
BLOBSTORE_DIR_PERMISSION
public static final org.apache.hadoop.fs.permission.FsPermission BLOBSTORE_DIR_PERMISSION
-
-
Constructor Details
-
HdfsBlobStoreImpl
public HdfsBlobStoreImpl(org.apache.hadoop.fs.Path path, Map<String, Object> conf) throws IOException- Throws:
IOException
-
HdfsBlobStoreImpl
public HdfsBlobStoreImpl(org.apache.hadoop.fs.Path path, Map<String, Object> conf, org.apache.hadoop.conf.Configuration hconf) throws IOException- Throws:
IOException
-
-
Method Details
-
listKeys
List relevant keys.- Returns:
- all keys that are available for reading
- Throws:
IOException
- on any error
-
listKeys
- Throws:
IOException
-
read
Get an input stream for reading a part.- Parameters:
key
- the key of the part to read- Returns:
- the where to read the data from
- Throws:
IOException
- on any error
-
write
Get an object tied to writing the data.- Parameters:
key
- the key of the part to write to.create
- whether the file needs to be new or not.- Returns:
- an object that can be used to both write to, but also commit/cancel the operation.
- Throws:
IOException
- on any error
-
exists
Check if the key exists in the blob store.- Parameters:
key
- the key to check for- Returns:
- true if it exists else false.
-
deleteKey
Delete a key from the blob store.- Parameters:
key
- the key to delete- Throws:
IOException
- on any error
-
getKeyDir
-
fullCleanup
- Throws:
IOException
-
listBlobStoreFiles
protected Iterator<BlobStoreFile> listBlobStoreFiles(org.apache.hadoop.fs.Path path) throws IOException - Throws:
IOException
-
getBlobReplication
- Throws:
IOException
-
updateBlobReplication
- Throws:
IOException
-
delete
- Throws:
IOException
-
shutdown
public void shutdown() -
getLastBlobUpdateTime
Get the last update time of any blob.- Returns:
- the last updated time of blobs within the blobstore.
- Throws:
IOException
- on any error
-
updateLastBlobUpdateTime
Updates the last updated time of existing blobstores to the current time.- Throws:
IOException
- on any error
-
validateBlobUpdateTime
Validates that the last updated blob time of the blobstore is up to date with the current existing blobs.- Throws:
IOException
- on any error
-