Class CgroupManager
java.lang.Object
org.apache.storm.container.DefaultResourceIsolationManager
org.apache.storm.container.cgroup.CgroupManager
- All Implemented Interfaces:
ResourceIsolationInterface
Class that implements ResourceIsolationInterface that manages cgroups.
-
Field Summary
Fields inherited from class org.apache.storm.container.DefaultResourceIsolationManager
conf, runAsUser
-
Constructor Summary
-
Method Summary
Modifier and TypeMethodDescriptionvoid
This function will be called when the worker needs to shutdown.getAllPids
(String workerId) Get all of the pids that are a part of this container.getLaunchCommand
(String workerId, List<String> existingCommand) To compose launch command based on workerId and existing command.long
getMemoryUsage
(String user, String workerId, int port) Get the current memory usage of the a given worker.long
Get the amount of free memory in MB.protected static boolean
Extracting out to mock it for tests.boolean
This class doesn't really manage resources.void
launchWorkerProcess
(String user, String topologyId, Map<String, Object> topoConf, int port, String workerId, List<String> command, Map<String, String> env, String logPrefix, ExitCodeCallback processExitCallback, File targetDir) After reserving resources for the worker (i.e. calling reserveResourcesForWorker), this function can be used to launch worker process.void
initialize data structures.void
reserveResourcesForWorker
(String workerId, Integer totalMem, Integer cpuNum, String numaId) This function should be used prior to starting the worker to reserve resources for the worker.Methods inherited from class org.apache.storm.container.DefaultResourceIsolationManager
areAllProcessesDead, forceKill, kill, runProfilingCommand
-
Constructor Details
-
CgroupManager
public CgroupManager()
-
-
Method Details
-
prepare
initialize data structures.- Specified by:
prepare
in interfaceResourceIsolationInterface
- Overrides:
prepare
in classDefaultResourceIsolationManager
- Parameters:
conf
- storm confs- Throws:
IOException
- on any error.
-
reserveResourcesForWorker
public void reserveResourcesForWorker(String workerId, Integer totalMem, Integer cpuNum, String numaId) throws SecurityException Description copied from interface:ResourceIsolationInterface
This function should be used prior to starting the worker to reserve resources for the worker.- Specified by:
reserveResourcesForWorker
in interfaceResourceIsolationInterface
- Overrides:
reserveResourcesForWorker
in classDefaultResourceIsolationManager
- Parameters:
workerId
- worker id of the worker to starttotalMem
- the amount of memory for the worker or null if not enforcedcpuNum
- the amount of cpu for the worker or null if not enforcednumaId
- NUMA zone if applicable the worker should be bound to- Throws:
SecurityException
-
cleanup
Description copied from interface:ResourceIsolationInterface
This function will be called when the worker needs to shutdown. This function should include logic to clean up after a worker is shutdown.- Specified by:
cleanup
in interfaceResourceIsolationInterface
- Overrides:
cleanup
in classDefaultResourceIsolationManager
- Parameters:
user
- the user of the workerworkerId
- worker id to shutdown and clean up afterport
- the port of the worker- Throws:
IOException
-
isOnLinux
protected static boolean isOnLinux()Extracting out to mock it for tests.- Returns:
- true if on Linux.
-
launchWorkerProcess
public void launchWorkerProcess(String user, String topologyId, Map<String, Object> topoConf, int port, String workerId, List<String> command, Map<String, throws IOExceptionString> env, String logPrefix, ExitCodeCallback processExitCallback, File targetDir) Description copied from interface:ResourceIsolationInterface
After reserving resources for the worker (i.e. calling reserveResourcesForWorker), this function can be used to launch worker process.- Specified by:
launchWorkerProcess
in interfaceResourceIsolationInterface
- Overrides:
launchWorkerProcess
in classDefaultResourceIsolationManager
- Parameters:
user
- the user who runs the command astopologyId
- the Id of the topologytopoConf
- the topology configurationport
- the port where the worker is onworkerId
- the Id of the workercommand
- the command to runenv
- the environment to run the commandlogPrefix
- the prefix to include in the logsprocessExitCallback
- a callback for when the process exitstargetDir
- the working directory to run the command in- Throws:
IOException
- on I/O exception
-
getLaunchCommand
To compose launch command based on workerId and existing command.- Parameters:
workerId
- the worker idexistingCommand
- the current command to run that may need to be modified- Returns:
- new commandline with necessary additions to launch worker
-
getAllPids
Get all of the pids that are a part of this container.- Overrides:
getAllPids
in classDefaultResourceIsolationManager
- Parameters:
workerId
- the worker id- Returns:
- all of the pids that are a part of this container
- Throws:
IOException
-
getMemoryUsage
Description copied from interface:ResourceIsolationInterface
Get the current memory usage of the a given worker.- Specified by:
getMemoryUsage
in interfaceResourceIsolationInterface
- Overrides:
getMemoryUsage
in classDefaultResourceIsolationManager
- Parameters:
user
- the user that the worker is running asworkerId
- the id of the workerport
- the port of the worker- Returns:
- the amount of memory the worker is using in bytes or -1 if not supported
- Throws:
IOException
- on I/O exception
-
getSystemFreeMemoryMb
Description copied from interface:ResourceIsolationInterface
Get the amount of free memory in MB. This might not be the entire box, it might be within a parent resource group.- Specified by:
getSystemFreeMemoryMb
in interfaceResourceIsolationInterface
- Overrides:
getSystemFreeMemoryMb
in classDefaultResourceIsolationManager
- Returns:
- The amount of memory in MB that are free on the system.
- Throws:
IOException
- on I/O exception
-
isResourceManaged
public boolean isResourceManaged()Description copied from class:DefaultResourceIsolationManager
This class doesn't really manage resources.- Specified by:
isResourceManaged
in interfaceResourceIsolationInterface
- Overrides:
isResourceManaged
in classDefaultResourceIsolationManager
- Returns:
- false
-