MininetGym Manual

MininetGym is a browser-accessible, open-source framework for training and evaluating Reinforcement Learning (RL) agents on cybersecurity tasks inside a live Software-Defined Network (SDN) emulated with Mininet and controlled by OpenDayLight.

The web dashboard exposes three main panels - Configuration Setup, Training Dashboard, and Results Panel - that cover the full experiment lifecycle without requiring any command-line interaction.

---

Configuration Setup

This panel lets you define every aspect of an experiment before starting training. Changes are saved immediately with the Save button.

Network Topology

The switch (OVS) sits at the center. All hosts and IoT devices connect to it in a star topology, as shown in the schema below.

• Hosts - number of regular hosts in the Mininet topology (default 5). Each host can generate normal traffic and, depending on the scenario, act as an attacker or victim.
• IoT Devices - number of IoT nodes attached to the switch (default 5). IoT nodes generate lightweight traffic patterns distinct from regular hosts.
• Controller - OpenDayLight REST API address; typically 127.0.0.1:8181 for a local instance or the Docker host IP for a containerised deployment.

Screenshots - Configuration Panel

Gym Type (Scenario)

The gym_type field selects both the task and the data source. Every scenario has a live variant (traffic is generated in real time inside Mininet) and a _from_dataset variant (observations are replayed from a pre-recorded scenario.json file, which is much faster and fully reproducible).

gym_type value	UI label	Task	Action space	Data source
classification	Classification	Classify traffic: None / Ping / UDP / TCP	Alert only (4 classes)	Live Mininet
classification_from_dataset	Classification (dataset)	Same classification task	Alert only (4 classes)	Replay dataset
attacks	Attack-Net	Binary detection: Normal vs. Attack (global view)	Alert only (2 classes)	Live Mininet
attacks_from_dataset	Attack-Net (dataset)	Same binary detection	Alert only (2 classes)	Replay dataset
attacks_ho	Attack-PerHost	Per-host: Normal / Victim / Attacker	SDN link block per host	Live Mininet
attacks_ho_from_dataset	Attack-PerHost (dataset)	Same per-host detection	SDN link block per host	Replay dataset
marl_pz	MARL-PZ	PettingZoo multi-agent (Host agents + optional Coordinator), Dec-POMDP with CTDE	Distributed SDN block	Live Mininet
marl_pz_from_dataset	MARL-PZ (dataset)	Same MARL-PZ task	Distributed SDN block	Replay dataset
marl_attacks	MARL (legacy)	Hierarchical multi-agent (Coordinator + Host agents), threaded	Distributed SDN block	Live Mininet
marl_attacks_from_dataset	MARL (legacy, dataset)	Same legacy MARL task	Distributed SDN block	Replay dataset

Agent Configuration

Use the Add Agent button to include one or more agents in the experiment. Each agent entry has:

• Algorithm - choose from Q-Learning, SARSA, DQN, PPO, A2C, Supervised. PPO and A2C are not available when the scenario is marl_pz.
• Name - a free-form label shown in all charts and tables.
• Episodes - number of training episodes (overrides env_params.episodes if set per-agent).
• Steps per episode - maximum steps before an episode resets (overrides env_params.max_steps).

Multiple agents can run sequentially in the same experiment for easy comparison.

---

Full Configuration Reference

All parameters are stored in config/default.yaml and editable live from the Configuration panel. The table below documents every key available in the configuration file, organised by section.

Global Settings

Parameter	Default	Description
training_directory	_training	Root directory where all experiment outputs are written. Each run creates a timestamped sub-folder.
enable_web_interface	true	Enable or disable the web dashboard. Set to false to run headless from the command line only.
web_server_port	5000	TCP port the Flask web server listens on.
server_user	mininet-gym	Unix user that owns Mininet-related processes. Must have passwordless sudo for Mininet commands.
log_level	info	Logging verbosity: info for normal operation, debug for verbose per-step output.
random_seed	77	Integer seed passed to NumPy and Python's random module to make runs reproducible.

Environment Parameters (env_params)

Parameter	Default	Description
gym_type	attacks	Selects which scenario pipeline to execute. Use *_from_dataset variants to replay pre-recorded files; use plain variants for live traffic generation.
agent_execution_mode	parallel	Controls how multiple agents are scheduled within an episode. parallel is normal production behaviour; sequential is useful for debugging.
csv_file	—	Path to a pre-recorded traffic CSV file. Required only for classification_from_dataset + Supervised CSV workflows.
episodes	10	Number of training episodes. More episodes mean longer training but better convergence.
max_steps	20	Maximum number of environment steps per episode.
test_episodes	80	Number of evaluation episodes run after training completes.
n_bins	4	Number of bins used by log-scale state discretisation for tabular agents.
steps_min_percentage	0.9	Minimum fraction of max_steps that must elapse before early-stop is evaluated.
accuracy_min	0.9	Accuracy threshold for early-stop.
wait_after_read	1	Seconds to pause after querying OpenDayLight flow statistics before collecting the next observation.
show_normal_traffic	true	Include normal-traffic generation events in the logs and UI.
print_training_chart	true	Save per-episode reward and accuracy chart images to the run output folder during training.
must_check_env	false	Run Gymnasium's check_env() validator before starting.

Attack Parameters (env_params.attacks)

Active for attack-oriented scenarios: Attack-Net, Attack-PerHost, MARL (and their dataset variants).

Parameter	Default	Description
likely	0.45	Base probability that an idle host starts an attack at each step.
likely_train	0.9	Attack probability during training episodes.
likely_eval	0.3	Attack probability during evaluation episodes.
max_attack_percentage	0.9	Hard cap on effective attack probability regardless of dynamic scaling.
short_attack_duration	5	Length of a SHORT_ATTACK window.
long_attack_duration	25	Length of a LONG_ATTACK window.
no_attack_timeout	3	Cooldown period after any attack before the same host can attack again.
unblock_min_hold_rounds	2	Minimum rounds a host must remain blocked before unblocking is allowed.
unblock_required_normal_streak	2	Consecutive NORMAL decisions required before a blocked host is unblocked.
apply_drop_rules	false	When true, the environment pushes real OpenDayLight flow rules to drop traffic from an identified attacker.

Unblock Policy

Parameter	Default	Description
unblock_min_hold_rounds	2	Minimum number of full episode rounds a host must remain SDN-blocked before the agent is permitted to unblock it.
unblock_required_normal_streak	2	Number of consecutive NORMAL decisions the agent must make for a blocked host before the block is lifted.

SDN Enforcement

Parameter	Default	Description
apply_drop_rules	false	When true, the environment pushes real OpenDayLight flow rules to drop traffic from an identified attacker at the OVS switch.

Detection Thresholds (env_params.attacks.thresholds and env_params.classification.thresholds)

Thresholds are used by the rule-based labeller that converts raw OpenDayLight flow statistics into ground-truth labels for reward computation.

Parameter	Default	Description
attacks.thresholds.packets	22000	Absolute packet-count threshold per observation window.
attacks.thresholds.var_packets	50	Allowed packet-count variation (%) relative to a rolling baseline.
attacks.thresholds.bytes	423000000	Absolute byte-volume threshold.
attacks.thresholds.var_bytes	30	Allowed byte variation (%) relative to baseline.
classification.thresholds.packets	200	Packet count above which traffic is labelled as active (non-idle).
classification.thresholds.bytes	1000000	Byte volume used alongside packet count to distinguish traffic types.
classification.thresholds.var_packets	100	Allowed packet variation (%) for classification labelling.
classification.thresholds.var_bytes	100	Allowed byte variation (%) for classification labelling.

Network Topology (env_params.net_params)

Parameter	Default	Description
num_hosts	5	Number of regular hosts in the Mininet topology.
num_switches	1	Number of OVS switches.
num_iots	5	Number of IoT nodes.
traffic_types	["none","ping","udp","tcp"]	List of normal-traffic generators available to hosts during an episode.
start_cli	false	Open the interactive Mininet CLI after the network is created.

SDN Controller (env_params.net_params.controller)

Parameter	Default	Description
ip	—	IP address of the OpenDayLight REST API.
port	8181	TCP port for the OpenDayLight REST API.
usr	admin	OpenDayLight REST API username.
pwd	admin	OpenDayLight REST API password.

Agent Common Parameters (agents.*)

Each entry in the agents list represents one agent configuration. Multiple agents can be enabled for sequential comparison in the same run.

Parameter	Default	Description
name	—	Display name shown in charts, confusion matrices, the metrics table, and PDF exports.
algorithm	—	Algorithm identifier: Q-learning, SARSA, DQN, PPO, A2C, Supervised.
enabled	true	Toggle this agent on or off without removing its configuration block.
progress_bar	false	Show a tqdm progress bar in the server terminal while training.
skip_learn	false	Skip the learning phase and go directly to evaluation.
show_action	false	Log the action chosen by the agent at every step.
load	false	Attempt to load saved model weights or Q-table artefacts before training begins.
load_dir	—	Relative path from training_directory to a previously saved run folder.
save	true	Save trained model artefacts to the run output folder at the end of training.
state_input_mode	normalized	Input representation for deep agents. normalized is recommended.
net_arch	[8, 8]	Neural network hidden layer sizes for deep agents.

Algorithm-Specific Hyperparameters

Q-Learning & SARSA

Parameter	Default	Description
learning_rate	0.2	Step size α for Q-table updates.
discount_factor	0.3	γ — how much future rewards are discounted.
exploration_rate	1.0	Initial ε for ε-greedy exploration.
exploration_decay	0.995	Multiplicative factor applied to ε after each episode.

DQN (Deep Q-Network)

Parameter	Default	Description
learning_rate	0.005	Adam optimiser learning rate.
gamma	0.3	Discount factor γ for future rewards.
buffer_size	5000	Capacity of the experience replay buffer.
batch_size	16	Mini-batch size drawn from the replay buffer per gradient update.
target_update_interval	80	Number of steps between target network weight copies.
learning_starts	20	Number of environment steps collected before learning begins.
exploration_fraction	0.1	Fraction of total training steps over which ε is annealed.
exploration_initial_eps	1.0	Starting value of ε.
exploration_final_eps	0.1	Final (minimum) value of ε.

PPO (Proximal Policy Optimisation)

Parameter	Default	Description
learning_rate	0.005	Adam optimiser learning rate for both actor and critic networks.
gamma	0.3	Discount factor γ.
n_steps	20	Number of rollout steps collected per environment before each policy update.
batch_size	1	Mini-batch size for policy gradient updates.
ent_coef	0.1	Entropy coefficient added to the loss to encourage exploration.

A2C (Advantage Actor-Critic)

Parameter	Default	Description
learning_rate	0.005	Adam optimiser learning rate.
gamma	0.3	Discount factor γ.
n_steps	20	Rollout steps before each policy update.
ent_coef	0.1	Entropy coefficient. Same semantics as PPO.

Supervised Agent

Parameter	Default	Description
train_test_split_ratio	0.20	Fraction of the initial dataset held out as a test set for the first evaluation (0–1).

---

Training Dashboard

Start the experiment from the Configuration panel and switch to Training Dashboard to monitor progress in real time. All data is pushed via WebSocket (Socket.IO) - no page refresh needed.

Screenshots - Training in Action

Live Charts

• Reward per Episode — cumulative reward obtained by the agent in each episode. A rising trend indicates the agent is learning the correct classification policy.
• Accuracy per Episode — percentage of correct classifications (or block/no-block decisions) over the episode. Target: ≥ 90 % after convergence.
• ε (Epsilon) Decay — shows how exploration decreases over time for tabular agents (Q-Learning, SARSA) and DQN. Deep actor-critic agents (PPO, A2C) use entropy rather than ε.

Host Status Monitor

A live grid shows the current classification label for every host: Normal, Attacker, or Victim. When an SDN block is issued, the attacker cell displays a lock icon (🔒).

Controls

• Pause / Resume — suspends the training thread; the Mininet network keeps running and traffic continues.
• Stop — terminates the current experiment cleanly and saves all partial results to disk. The Results panel becomes available immediately.
• Reset — clears the live charts and console log without stopping training.

---

Results Panel

After training completes (or is stopped), switch to the Results Panel to inspect per-agent metrics and export a PDF report.

Screenshots - Results Panel

Metrics Summary Table

For each agent the table shows the following metrics, computed over all evaluation (test_episodes) episodes:

Metric	Description
Accuracy	Fraction of steps where the agent's decision matches the ground-truth label.
Precision	Of all steps the agent labelled as "Attack", what fraction were true attacks.
Recall	Of all true attack steps, what fraction the agent correctly detected.
F1-score	Harmonic mean of Precision and Recall.
Mitigation Ratio	Ratio of SDN block actions correctly issued to total true attack events detected.
False Negative Rate (FNR)	Fraction of actual attacks that the agent missed.
Attack Latency	Average number of environment steps between the start of an attack and the first correct SDN block action.

Comparison Charts

• Bar chart — side-by-side accuracy and F1 across all agents.
• Radar chart — multi-dimensional agent profile covering accuracy, precision, recall, FNR, and mitigation ratio.
• Confusion Matrix — per-agent heatmap of predicted vs. true labels.

PDF Export

Click Export PDF to download a formatted report containing all charts, the metrics table, and the experiment configuration.

---

Supported RL Agents

Q-Learning

Tabular off-policy TD method. The observation space is log-bin discretized to keep the state table manageable. Best suited for low-dimensional scenarios. Converges quickly with small episode counts but does not scale well beyond a few hosts.

SARSA

On-policy variant of Q-Learning. Uses the action actually taken for the TD update, making it slightly more conservative in exploration.

DQN - Deep Q-Network

Uses a neural network to approximate the Q-function. Includes an experience replay buffer and a target network. Implemented via Stable-Baselines3.

PPO - Proximal Policy Optimization

Actor-critic on-policy algorithm with a clipped surrogate objective. Robust and sample-efficient; generally the best baseline for continuous or large discrete observation spaces.

A2C - Advantage Actor-Critic

Synchronous on-policy actor-critic. Faster per-step than PPO but may require more episodes to converge. Good for quick experiments.

Supervised Agent

A classification baseline trained with supervised learning on labelled traffic data. Supports incremental learning: the model is updated after each episode with the new observations collected during training.

---

Attack Scenarios

The Attack Generator runs on dedicated Mininet hosts and can produce:

• UDP Flood — high-rate UDP packets toward a victim host.
• TCP Flood — repeated TCP connection attempts.
• ICMP Flood — continuous ping storm.
• SYN Flood — half-open TCP connections to exhaust victim resources.
• Slowloris — slow HTTP attack that holds connections open.

Attacks are injected randomly during training so that agents learn to detect and mitigate them across varying conditions.

---

MARL-PZ — PettingZoo Multi-Agent Mode

The MARL-PZ scenario (marl_pz / marl_pz_from_dataset) is a PettingZoo Parallel-API multi-agent environment — a cleaner, race-condition-free replacement for the legacy threaded MARL scenario. It is modelled as a Decentralized Partially Observable MDP (Dec-POMDP) under the Centralized-Training, Decentralized-Execution (CTDE) paradigm.

• Host Agents (one per host) — observe per-host traffic (4 or 8 features depending on attacks.include_percentage_variations), classify the host as NORMAL / ATTACK_IN / ATTACK_OUT.
• Coordinator (optional, attacks.use_coordinator: true) — aggregates host alerts and relays a signal back to all hosts. When disabled, hosts act as fully independent learners.

Communication Strategies (env_params.communication.strategy)

Strategy	ID	Description	Status
Independent	none	No coordinator, no messages.	Implemented
S0 - Naive Broadcast	naive_broadcast	Coordinator relays any attack alert to all hosts, unfiltered.	Implemented
S1 - Uncertainty-Aware Querying (UAQ)	uaq	Hosts flag alerts as uncertain or confident via Shannon entropy; the coordinator relays only confident alerts.	Implemented
S2 - Federated Policy Sync	federated_sync	Q-tables of all host agents are averaged at fixed intervals. Tabular agents only.	Implemented
S3 - Periodic Policy Exchange	policy_exchange	The best-performing agent's Q-table is copied to lagging peers after each episode. Tabular agents only.	Implemented
S4 - Experience Replay Sharing	experience_sharing	Agents broadcast high-TD-error transitions to peers at fixed intervals. Tabular agents only.	Implemented
S5 - Hierarchical Coordination	hierarchical	Cluster-head aggregation for large IoT topologies.	Future work

Agent Compatibility

A proper multi-agent PPO (MAPPO) implementation is on the roadmap. In the meantime, existing frameworks with validated MAPPO/multi-agent support include:

• MARLlib / RLlib — multi-agent RL examples
• TorchRL — Multi-Agent PPO tutorial
• TorchRL — Multi-Agent DDPG tutorial

---

MARL (Legacy) — Multi-Agent Mode

In the legacy MARL scenario (marl_attacks), two agent types collaborate via an internal message bus:

• Coordinator — observes global network statistics, decides whether to signal an alert to host agents.
• Host Agents (one per host) — observe per-host traffic, take local block / no-block actions.

All agents run in parallel threads. The coordinator's signal is included as an extra feature in each host agent's observation vector.

---

Tips & Troubleshooting

Training does not start

• Make sure the Mininet topology is running (sudo python main.py).
• Check that OpenDayLight is reachable at the configured address and port.
• Verify no previous experiment is still running (use Stop first).
• For _from_dataset variants, confirm that a scenario.json file exists and is selected in the Scenario Source panel.

Charts are not updating

• Confirm the browser has an active WebSocket connection.
• Reload the page - the Socket.IO client will reconnect automatically and resume receiving events.

SDN blocks are never triggered

• Only the Attack-PerHost and MARL scenarios issue SDN drop rules.
• Ensure apply_drop_rules: true in the config; the default is false.
• Verify OpenDayLight credentials and REST API port (8181) are correct.

Low accuracy after many episodes

• Increase episodes or lower learning_rate.
• For tabular agents (Q-Learning / SARSA) try increasing exploration_decay closer to 1.0 (e.g. 0.9995) to explore longer.
• PPO / A2C may need larger n_steps to capture full attack sequences within a single rollout.
• Check that n_bins is not too low.

PPO / A2C assertion error at startup

• Ensure batch_size < n_steps × n_envs. With a single environment, set batch_size = 1 or increase n_steps.

"Incompatible agent(s) for marl_pz" popup / an agent got auto-disabled

• This is expected: PPO and A2C are not supported in the marl_pz scenario.
• Switch to Q-Learning, SARSA, or DQN for marl_pz, or switch the scenario to attacks_ho / marl_attacks if you specifically need PPO/A2C.

PDF export fails

• Ensure WeasyPrint is installed on the server: pip install weasyprint.
• Check server logs (app.log) for the detailed error message.

---

Keyboard & UI Shortcuts

Action	How
Close this manual	Press Esc or click outside the panel
Navigate sections	Click any entry in the left Contents sidebar
Switch panel	Header navigation buttons (Configuration / Training / Results)
Mobile menu	Tap the ☰ button in the top-right corner

---

About

MininetGym is developed at DISMI — University of Modena and Reggio Emilia by Salvo Finistrella, Stefano Mariani, and Franco Zambonelli.

Source code: github.com/dipi-unimore/mininet-gym
Contact: salvo.finistrella@unimore.it
Paper: ScienceDirect — doi.org/10.1016/j.softx.2025.102312