Skip to content

FiniteStateRedAgent

Bases: BaseAgent

A red agent that performs as a finite state automata, transitioning the hosts it is aware of between different states of knowledge.

Throughout the episode, the hosts will transition between the 8 different states. This will mainly occur via the state transition matrices, depending on action success or failure. However, other external factors may affect the state, such as Blue removing a session from a host or the host being outside the agent's area of influence (their assigned subnets).

Functions

__init__ 

__init__(name = None, np_random = None, agent_subnets = None)

Initialises the FSM red agent.

Creates the variables to store internal knowledge for the agent. Sets the state transitions (basic) and host priorities (none), for the agent.

Parameters:

Name Type Description Default
name str

agent name

 None
np_random Tuple[np.random.Generator, Any]

numpy random number generator

 None
agent_subnets List[IPv4Subnet]

list of subnet cidr bounds that this red agent can reach

 None

action_list 

action_list()

The possible actions that can be performed by the agent, in the order of the columns of the state transition matrices.

Returns:

Name Type Description
actions List[Action]

List of the 9 actions that a red agent can perform in CC4

get_action 

get_action(observation: dict, action_space: dict)

The choosing and returning of the action to be used for the current step.

In order to make an appropriate choice, the observations from the previous action must be processed. This is carried out through private functions, in the order listed below:

  1. _host_state_transition(action, success)
    • The host that was last acted on has its state changed based on the action success.
  2. _process_new_observations(observation)
    • The details of the observation is then processed for newly discovered hosts and decoy discoveries.
  3. _session_removal_state_change(observation)
    • The sessions are then checked to make sure none were lost in the last step, and changing their host states accordingly.

An textual output is available if the print attributes are set to True (function last_turn_summary).

The next action for the current step is then selected:

  1. If the previous action is still 'in progress' then Sleep is returned, as this action will not be used.
  2. _choose_host_and_action(action_space, known_hosts)
    • A host is chosen; either randomly or based on host state priority.
    • An action on that host is then selected according to the state_transition_probabilities matrix.
  3. If the action chosen is ExploitRemoteService_cc4, then the selector that takes into account the detected decoys is chosen.
  4. The action is stored for reference and returned.

Parameters:

Name Type Description Default
observation dict

The dictionary holding the observations made by the agent from the previous action

 required
action_space dict

The restricted space that the agent knows about and can act on, given by the environment.

 required

last_turn_summary 

last_turn_summary(observation: dict, action: str, success: str)

Prints action name, parameters, success and sometimes observation and host states.

If self.print_action_output is True, the function will run and the observed action and its success will be outputted to the terminal.

If self.print_obs_output is True, additionally the observation and internal host_states dictionaries will be outputted. This should only be True for debugging.

Parameters:

Name Type Description Default
observation dict

the observation that the agent just received about its previous action

 required
action str

the previous action that was taken

 required
success Union[bool, CyEnums.TrinaryEnum]

the success of the previous action

 required

set_host_state_priority_list 

set_host_state_priority_list()

Abstract function for child classes to overwrite with a host state priority list.

Each dictionary value must be an integer or float from 0 to 100, with the total sum of values equaling 100.

Example 
host_state_priority_list = {
    'K':12.5, 'KD':12.5, 
    'S':12.5, 'SD':12.5, 
    'U':12.5, 'UD':12.5, 
    'R':12.5, 'RD':12.5}

Returns:

Name Type Description
host_state_priority_list None

when used in variant child classes, a dict would be returned.

set_initial_values 

set_initial_values(action_space, observation)

The action parameter values in the action space are sanitised for internal use.

Parameters:

Name Type Description Default
action_space dict
required
observation dict
required

state_transitions_failure 

state_transitions_failure()

The state transition matrix for an unsuccessful action on a host.

There is a row for each of the host states: K, KD, S, SD, U, UD, R, RD. Then there is a column for each of the actions, in the order of the action_list.

All column 0 must have transition state as all hosts in subnet are transitioned

Example 
map = {
    'K'  : ['K' , 'K' , 'K' , None, None, None, None, None, None],
    'KD' : ['KD', 'KD', 'KD', None, None, None, None, None, None],
    'S'  : ['S' , None, None, 'S' , 'S' , None, None, None, None],
    'SD' : ['SD', None, None, 'SD', 'SD', None, None, None, None],
    'U'  : ['U' , None, None, None, None, 'U' , None, None, 'U' ],
    'UD' : ['UD', None, None, None, None, 'UD', None, None, 'UD'],
    'R'  : ['R' , None, None, None, None, None, 'R' , 'R' , 'R' ],
    'RD' : ['RD', None, None, None, None, None, 'RD', 'RD', 'RD'],
    'F'  : ['F',  None, None, None, None, None, None, None, None],
}

Returns:

Name Type Description
map Dict[str, List[float]]

state_transitions_probability 

state_transitions_probability()

Returns a state transitions probability matrix.

There is a row for each of the host states: K, KD, S, SD, U, UD, R, RD. Then there is a column for each of the actions, in the order of the action_list.

Example 
map = {
    'K'  : [0.5,  0.25, 0.25, None, None, None, None, None, None],
    'KD' : [None, 0.5,  0.5,  None, None, None, None, None, None],
    'S'  : [0.25, None, None, 0.25, 0.5 , None, None, None, None],
    'SD' : [None, None, None, 0.25, 0.75, None, None, None, None],
    'U'  : [0.5 , None, None, None, None, 0.5 , None, None, 0.0 ],
    'UD' : [None, None, None, None, None, 1.0 , None, None, 0.0 ],
    'R'  : [0.5,  None, None, None, None, None, 0.25, 0.25, 0.0 ],
    'RD' : [None, None, None, None, None, None, 0.5,  0.5,  0.0 ],
}

Returns:

Name Type Description
matrix Dict[str, List[float]]

state_transitions_success 

state_transitions_success()

The state transition matrix for a successful action on a host.

There is a row for each of the host states: K, KD, S, SD, U, UD, R, RD. Then there is a column for each of the actions, in the order of the action_list.

All column 0 must have transition state as all hosts in subnet are transitioned

Example 
map = {
    'K'  : ['KD', 'S',  'S',  None, None, None, None, None, None],
    'KD' : ['KD', 'SD', 'SD',  None, None, None, None, None, None],
    'S'  : ['SD', None, None, 'S' , 'U' , None, None, None, None],
    'SD' : ['SD', None, None, 'SD', 'UD', None, None, None, None],
    'U'  : ['UD', None, None, None, None, 'R' , None, None, 'S' ],
    'UD' : ['UD', None, None, None, None, 'RD', None, None, 'SD'],
    'R'  : ['RD', None, None, None, None, None, 'R' , 'R' , 'S' ],
    'RD' : ['RD', None, None, None, None, None, 'RD', 'RD', 'SD'],
    'F'  : ['F',  None, None, None, None, None, None, None, None],
}

Returns:

Name Type Description
map Dict[str, List[float]]