org.apache.storm.container

Interface ResourceIsolationInterface

All Known Implementing Classes:

CgroupManager, DefaultResourceIsolationManager, DockerManager, OciContainerManager, RuncLibContainerManager

public interface ResourceIsolationInterface

A plugin to support resource isolation and limitation within Storm.

Method Summary

Modifier and Type	Method and Description
boolean	areAllProcessesDead(String user, String workerId) Check if all the processes are dead.
void	cleanup(String user, String workerId, int port) This function will be called when the worker needs to shutdown.
void	forceKill(String user, String workerId) Kill the given worker forcefully.
long	getMemoryUsage(String user, String workerId, int port) Get the current memory usage of the a given worker.
long	getSystemFreeMemoryMb() Get the amount of free memory in MB.
boolean	isResourceManaged() Return true if resources are being managed.
void	kill(String user, String workerId) Kill the given worker.
void	launchWorkerProcess(String user, String topologyId, Map<String,Object> topoConf, int port, String workerId, List<String> command, Map<String,String> env, String logPrefix, ExitCodeCallback processExitCallback, File targetDir) After reserving resources for the worker (i.e.
void	prepare(Map<String,Object> conf) Called when starting up.
void	reserveResourcesForWorker(String workerId, Integer workerMemory, Integer workerCpu, String numaId) This function should be used prior to starting the worker to reserve resources for the worker.
boolean	runProfilingCommand(String user, String workerId, List<String> command, Map<String,String> env, String logPrefix, File targetDir) Run profiling command.

Method Detail

prepare

void prepare(Map<String,Object> conf)
      throws IOException

Called when starting up.

Parameters:

conf - the cluster config

Throws:

IOException - on any error.

reserveResourcesForWorker

void reserveResourcesForWorker(String workerId,
                               Integer workerMemory,
                               Integer workerCpu,
                               String numaId)

This function should be used prior to starting the worker to reserve resources for the worker.

Parameters:

workerId - worker id of the worker to start

workerMemory - the amount of memory for the worker or null if not enforced

workerCpu - the amount of cpu for the worker or null if not enforced

numaId - NUMA zone if applicable the worker should be bound to

cleanup

void cleanup(String user,
             String workerId,
             int port)
      throws IOException

This function will be called when the worker needs to shutdown. This function should include logic to clean up after a worker is shutdown.

Parameters:

user - the user of the worker

workerId - worker id to shutdown and clean up after

port - the port of the worker

Throws:

IOException

launchWorkerProcess

void launchWorkerProcess(String user,
                         String topologyId,
                         Map<String,Object> topoConf,
                         int port,
                         String workerId,
                         List<String> command,
                         Map<String,String> env,
                         String logPrefix,
                         ExitCodeCallback processExitCallback,
                         File targetDir)
                  throws IOException

After reserving resources for the worker (i.e. calling reserveResourcesForWorker), this function can be used to launch worker process.

Parameters:

user - the user who runs the command as

topologyId - the Id of the topology

topoConf - the topology configuration

port - the port where the worker is on

workerId - the Id of the worker

command - the command to run

env - the environment to run the command

logPrefix - the prefix to include in the logs

processExitCallback - a callback for when the process exits

targetDir - the working directory to run the command in

Throws:

IOException - on I/O exception

getMemoryUsage

long getMemoryUsage(String user,
                    String workerId,
                    int port)
             throws IOException

Get the current memory usage of the a given worker.

Parameters:

user - the user that the worker is running as

workerId - the id of the worker

port - the port of the worker

Returns:

the amount of memory the worker is using in bytes or -1 if not supported

Throws:

IOException - on I/O exception

getSystemFreeMemoryMb

long getSystemFreeMemoryMb()
                    throws IOException

Get the amount of free memory in MB. This might not be the entire box, it might be within a parent resource group.

Returns:

The amount of memory in MB that are free on the system.

Throws:

IOException - on I/O exception

kill

void kill(String user,
          String workerId)
   throws IOException

Kill the given worker.

Parameters:

user - the user that the worker is running as

workerId - the id of the worker to kill

Throws:

IOException - on I/O exception

forceKill

void forceKill(String user,
               String workerId)
        throws IOException

Kill the given worker forcefully.

Parameters:

user - the user that the worker is running as

workerId - the id of the worker to kill

Throws:

IOException - on I/O exception

areAllProcessesDead

boolean areAllProcessesDead(String user,
                            String workerId)
                     throws IOException

Check if all the processes are dead.

Parameters:

user - the user that the processes are running as

workerId - the id of the worker to kill

Returns:

true if all the processed are dead; false otherwise

Throws:

IOException - on I/O exception

runProfilingCommand

boolean runProfilingCommand(String user,
                            String workerId,
                            List<String> command,
                            Map<String,String> env,
                            String logPrefix,
                            File targetDir)
                     throws IOException,
                            InterruptedException

Run profiling command.

Parameters:

user - the user that the worker is running as

workerId - the id of the worker

command - the command to run

env - the environment to run the command

logPrefix - the prefix to include in the logs

targetDir - the working directory to run the command in

Returns:

true if succeeds; false otherwise

Throws:

IOException - on I/O exception

InterruptedException - if interrupted

isResourceManaged

boolean isResourceManaged()

Return true if resources are being managed. The DefaultResourceIsolationManager will have it return false since it doesn’t really manage resources.

Returns:

true if resources are being managed.