geneva.evolve¶
Main evolution driver for Geneva (GENetic EVAsion). This file performs the genetic algorithm, and relies on the evaluator (evaluator.py) to provide fitness evaluations of each individual.
-
evolve.
add_to_hof
(hof, population)¶ Iterates over the current population and updates the hall of fame. The hall of fame is a dictionary that tracks the fitness of every run of every strategy ever.
Parameters: - hof (dict) – Current hall of fame
- population (list) – Population list
Returns: Updated hall of fame
Return type: dict
-
evolve.
collect_plugin_args
(cmd, plugin, plugin_type, message=None)¶ Collects and prints arguments for a given plugin.
Parameters: - cmd (list) – sys.argv or a list of args to parse
- plugin (str) – Name of plugin to import (“http”)
- plugin_type (str) – Component of plugin to import (“client”)
- message (str) – message to override for printing
-
evolve.
collect_results
(hall_of_fame)¶ Collect final results from offspring.
Parameters: hall_of_fame (dict) – Hall of fame of individuals Returns: Formatted printout of the hall of fame Return type: str
-
evolve.
driver
(cmd)¶ Main workflow driver for the solver. Parses flags and input data, and initiates solving.
Parameters: cmd (list) – sys.argv or a list of arguments Returns: Hall of fame of individuals Return type: dict
-
evolve.
eval_only
(logger, requested, ga_evaluator, runs=1)¶ Parses a string representation of a given strategy and runs it through the evaluator.
Parameters: - logger (
logging.Logger
) – A logger to log with - requested (str) – String representation of requested strategy or filename of strategies
- ga_evaluator (
evaluator.Evaluator
) – An evaluator to evaluate with - runs (int) – Number of times each strategy should be evaluated
Returns: Success rate of tested strategies
Return type: float
- logger (
-
evolve.
fitness_function
(logger, population, ga_evaluator)¶ Calls the evaluator to evaluate a given population of strategies. Sets the .fitness attribute of each individual.
Parameters: - logger (
logging.Logger
) – A logger to log with - population (list) – List of individuals to evaluate
- ga_evaluator (
evaluator.Evaluator
) – An evaluator object to evaluate with
Returns: Population post-evaluation
Return type: list
- logger (
-
evolve.
generate_strategy
(logger, num_in_trees, num_out_trees, num_in_actions, num_out_actions, seed, environment_id=None, disabled=None)¶ Generates a strategy individual.
Parameters: - logger (
logging.Logger
) – A logger to log with - num_in_trees (int) – Number of trees to initialize in the inbound forest
- num_out_trees (int) – Number of trees to initialize in the outbound forest
- num_in_actions (int) – Number of actions to initialize in the each inbound tree
- num_out_actions (int) – Number of actions to initialize in the each outbound tree
- environment_id (str, optional) – Environment ID to assign to the new individual
- disabled (str, optional) – List of actions that should not be considered in building a new strategy
Returns: A new strategy object
Return type: actions.strategy.Strategy
- logger (
-
evolve.
genetic_solve
(logger, options, ga_evaluator)¶ Run genetic algorithm with given options.
Parameters: - logger (
logging.Logger
) – A logger to log with - options (dict) – Options to respect.
- ga_evaluator (
evaluator.Evaluator
) – Evaluator to evaluate strategies with
Returns: Hall of fame of individuals
Return type: dict
- logger (
-
evolve.
get_args
(cmd)¶ Sets up argparse and collects arguments.
Parameters: cmd (list) – sys.argv or a list of args to parse Returns: Parsed arguments Return type: namespace
-
evolve.
get_unique_population_size
(population)¶ Computes number of unique individuals in a population.
Parameters: population (list) – Population list
-
evolve.
initialize_population
(logger, options, canary_id, disabled=None)¶ Initializes the population from either random strategies or strategies located in a file.
Parameters: - logger (
logging.Logger
) – A logger to log with - options (dict) –
Options to respect in generating initial population. Options that can be specified as keys:
”load_from” (str, optional): File to load population from population_size (int): Size of population to initialize
”in-trees” (int): Number of trees to initialize in inbound forest of each individual
”out-trees” (int): Number of trees to initialize in outbound forest of each individual
”in-actions” (int): Number of actions to initialize in each inbound tree of each individual
”out-actions” (int): Number of actions to initialize in each outbound tree of each individual
”seed” (str): Strategy to seed this pool with
- canary_id (str) – ID of the canary strategy, used to associate each new strategy with the packets captured during the canary phase
- disabled (list, optional) – List of actions that are disabled
Returns: New population of individuals
Return type: list
- logger (
-
evolve.
load_generation
(logger, filename)¶ Loads strategies from a file
Parameters: - logger (
logger.Logger
) – A logger to log with - filename (str) – Filename of file containing newline separated strategies to read generation from
- logger (
-
evolve.
mutate_individual
(logger, ind)¶ Simply calls the mutate function of the given individual.
Parameters: - logger (
logging.Logger
) – A logger to log with - ind (
actions.strategy.Strategy
) – A strategy individual to mutate
Returns: Mutated individual
Return type: actions.strategy.Strategy
- logger (
-
evolve.
mutation_crossover
(logger, population, hall, options)¶ Apply crossover and mutation on the offspring.
Hall is a copy of the hall of fame, used to accept or reject mutations.
Parameters: - logger (
logging.Logger
) – A logger to log with - population (list) – Population of individuals
- hall (dict) – Current hall of fame
- options (dict) – Options to override settings. Accepted keys are: “crossover_pb” (float): probability of crossover “mutation_pb” (float): probability of mutation “allowed_retries” (int): number of times a strategy is allowed to exist in the hall of fame. “no_reject_empty” (bool): whether or not empty strategies should be rejected
Returns: New population after mutation
Return type: list
- logger (
-
evolve.
print_results
(hall_of_fame, logger)¶ Prints hall of fame.
Parameters: - hall_of_fame (dict) – Hall of fame to print
- logger (
logging.Logger
) – A logger to log results with
-
evolve.
restrict_headers
(logger, protos, filter_fields, disabled_fields)¶ Restricts which protocols/fields can be accessed by the algorithm.
Parameters: - logger (
logging.Logger
) – A logger to log with - protos (str) – Comma separated string of protocols that are allowed
- filter_fields (str) – Comma separated string of fields to allow
- disabled_fields (str) – Comma separated string of fields to disable
- logger (
-
evolve.
run_collection_phase
(logger, ga_evaluator)¶ Individual mutation works best when it has seen real packets to base action and trigger values off of, instead of blindly fuzzing packets. Usually, the 0th generation is useless because it hasn’t seen any real packets yet, and it bases everything off fuzzed data. To combat this, a canary phase is done instead.
In the canary phase, a single dummy individual is evaluated to capture packets. Once the packets are captured, they are associated with all of the initial population pool, so all of the individuals have some packets to base their data off of.
Since this phase by necessity requires the evaluator, this is only run if –no-eval is not specified.
Parameters: - logger (
logging.Logger
) – A logger to log with - ga_evaluator (
evaluator.Evaluator
) – An evaluator object to evaluate with
Returns: ID of the test ‘canary’ strategy evaluated to do initial collection
Return type: str
- logger (
-
evolve.
sel_random
(individuals, k)¶ Implementation credit to DEAP: https://github.com/DEAP/deap Select k individuals at random from the input individuals with replacement. The list returned contains references to the input individuals.
Parameters: - individuals (list) – A list of individuals to select from.
- k (int) – The number of individuals to select.
Returns: A list of selected individuals.
Return type: list
-
evolve.
selection_tournament
(individuals, k, tournsize, fit_attr='fitness')¶ Implementation credit to DEAP: https://github.com/DEAP/deap Select the best individual among tournsize randomly chosen individuals, k times. The list returned contains references to the input individuals.
Parameters: - individuals (list) – A list of individuals to select from.
- k (int) – The number of individuals to select.
- tournsize (int) – The number of individuals participating in each tournament.
- fit_attr – The attribute of individuals to use as selection criterion (defaults to “fitness”)
Returns: A list of selected individuals.
Return type: list
-
evolve.
setup_logger
(log_level)¶ Sets up the logger. This will log at the specified level to “ga.log” and at debug level to “ga_debug.log”. Logs are stored in the trials/ directory under a run-specific folder. Example: trials/2020-01-01_01:00:00/logs/ga.log
Parameters: log_level (str) – Log level to use in setting up the logger (“debug”)
-
evolve.
write_generation
(filename, population)¶ Writes the population pool for a specific generation.
Parameters: - filename (str) – Name of file to write the generation out to
- population (list) – List of individuals to write out
-
evolve.
write_hall
(filename, hall_of_fame)¶ Writes hall of fame out to a file.
Parameters: - filename (str) – Filename to write results to
- hall_of_fame (dict) – Hall of fame of individuals