Why I Built a Mikrotik Security Auditor
I’m a mechanical engineer by trade, but I’ve been messing with networks and open source tools at home for years. I’ve written before about self-hosting my NAS with WebDAV and Rclone – I like building things that just work, without paying for a subscription.
MikroTik fits that philosophy perfectly. Their gear is cheap, configurable, and punches way above its weight if you put in the time. I’ve used it to squeeze the most out of my home setup – VLAN segmentation, guest networks, QoS for when the whole family is streaming at once.
So when I got a hAP ax3 and set it up the way I thought was right, I felt pretty good about it. Bridge configured. VLANs isolated. Firewall rules that looked solid.
Then I ran an existing audit tool on my own config and it found things I’d missed. An open DNS resolver. A WPS interface I meant to disable. A default admin account I could swear I’d locked down.
That bothered me. If I missed those, what else was I missing?
I started digging into the audit tool itself. Something felt off about some of the results, so I put the tool under the same scrutiny I’d wanted for my router config. That’s where things got interesting. The audit tool had bugs. Broken regex patterns. Inflated check counts. Hallucinated device model codes in the documentation.
This is where the story takes a turn. I could have fixed those bugs and moved on. Instead, I did something that sounds ridiculous in retrospect: I decided the audit tool needed an audit. Then that audit needed a fact-check. Then that fact-check needed a deep-dive investigation. Five rounds later, I had not just a fixed tool, but a methodology: multiple independent reviewers, each round catching what the previous one missed, with a dedicated agent whose only job was to detect hallucinations.
There’s nothing special about having AI help you write code. What’s unusual is having AI agents catch each other’s mistakes – and yours – before you ship. That’s what made this project different.
The result is a 107-check offline RouterOS security auditor. Zero dependencies, no cloud, no registration. Just Python and your config export.
What the Tool Does – 107 Checks, Zero Dependencies
It’s single-purpose: feed it a RouterOS .rsc export and it tells you what’s misconfigured. 107 checks across 9 domains covering the full surface area of a MikroTik router:
| Domain | Checks | What It Finds |
|---|---|---|
| AUTH | 18 | Default admin, weak passwords, unrestricted services |
| SRV | 17 | Open DNS resolvers, SNMP public, Telnet/FTP enabled |
| FW | 17 | Missing rules, RAW table gaps, IPv6 holes |
| SYS | 10 | Outdated RouterOS (CVE check), missing NTP, no logging |
| NET | 9 | VLAN misconfigs, DHCP leaks, DNS cache poisoning |
| ROUTE | 9 | BGP/OSPF without auth, missing route filters |
| WIFI | 13 | WEP/TKIP, WPS enabled, missing client isolation |
| SCRIPT | 8 | Overly permissive scripts, hardcoded credentials |
| COMP | 6 | Compliance mapping to CIS/NIST/ISO/PCI-DSS |
Every finding gets a CVSS score, maps to whichever compliance framework you care about (CIS, NIST SP 800-53, ISO 27001, PCI-DSS), and includes advice specific to your hardware – hAP ax3, hAP ac2, CCR, RB, whatever. Output is plain text, JSON, or HTML. And since the whole thing runs on Python stdlib, you don’t install anything except Python itself.
How the Audits Stacked Up
I built this using pi, a framework that lets you spin up specialized agents that work together. Here’s what each round looked like.
Round 1: Five Domain Experts Walk Into a Codebase
I created 5 specialized agents, each one an expert in a specific domain of RouterOS security:
- Methodology Expert – audited the audit theory, CVSS scoring logic, and check methodology
- Firewall & Network Security Expert – scrutinized every FW, AUTH, and ROUTE check
- WiFi & Wireless Expert – examined WIFI and CAPsMAN checks, hardware-specific constraints for hAP/CCR/RB devices
- QoS & Systems Expert – audited SYS, NET, and SCRIPT checks for accuracy
- Syntax & Compliance Expert – verified RouterOS syntax correctness and compliance mapping accuracy
Plus a Synthesis Lead to merge everything into one report.
Each agent independently read the source files, analyzed them from their domain perspective, and wrote a detailed report. The synthesis lead compiled everything into a single findings document.
Result: 86 findings, 12 of them critical. That includes 6 fundamentally broken regex patterns that were producing false positives and false negatives simultaneously. The tool would flag a secure config as broken while missing the actual vulnerability.
Round 2: The Fact-Checkers Strike Back
So I ran Round 2 – three independent fact-checkers, plus a Meta-Verifier to synthesize their work:
- Code & Regex Fact-Checker – verified every Python regex claim against the actual code
- RouterOS Fact-Checker – verified every RouterOS syntax claim against MikroTik’s actual documentation
- Docs & Counts Fact-Checker – verified every numerical claim in the report
- Meta-Verifier – synthesized all three reports and specifically looked for hallucinations
This round caught something important. The original report claimed a fictional model code L46UGS-5axD2axD that looked completely plausible – right down to the MikroTik naming convention – but didn’t exist in any MikroTik documentation, anywhere. The check counts were also inflated: the report claimed “21 missing checks” when the actual number was 5.
Grade for the original report: C+ (65-70% accurate). Not terrible, but not good enough for a security tool.
Round 3: Deep-Dive Investigation
The fact-checkers flagged 15 findings as needing more investigation. I spun up 3 deep-dive agents:
- Network Config Deep-Dive – DNS, DHCP, DoH, HW offload edge cases
- Docs & Scripting Deep-Dive – WiFi documentation accuracy, terse mode parsing, remediation script correctness
- Architecture Deep-Dive – the QoS domain gap (the tool wasn’t checking traffic shaping at all), path-context parser blind spots
Result: 10 confirmed, 4 partially correct, 0 refuted. Final grade: B-.
Round 4: The Implementation Workgroup
Now I had a verified set of findings. Time to fix things. I spun up an implementation workgroup:
- Planner – read all 86 findings and mapped exact oldText to newText edits across every file
- Code Worker – applied 25+ fixes to
audit_rsc.py: fixed regex patterns, addedre.DOTALLflags, corrected CVSS scoring vectors, fixed exit codes - Docs Worker – fixed
AUDIT_CHECKS.md,SYNTAX_REFERENCE.md,SECURITY_BASELINE.md,COMPLIANCE_MAPPING.md - Validator – compiled Python, ran against the test config, cross-checked consistency across files
Total changes: 25+ fixes across 6 files.
The Hallucination Guard
The most important architectural decision I made: every workgroup had an independent verification agent – a dedicated “hallucination guard” whose only job was to catch mistakes. When the fact-checkers found the fictional model code, it wasn’t treated as an embarrassment. It simply got corrected. That pattern carried through every phase.
Why This Matters
I built this tool using the same approach I’d use to audit a network: get multiple opinions, verify everything independently, write down what you find, fix things iteratively. It’s not a fancy framework. It’s just good engineering practice applied twice.
Getting Started in 60 Seconds
Installation
pip install mikrotik-rsc-auditor
Or if you prefer isolated environments:
pipx install mikrotik-rsc-auditor
Usage
Export your config from the router, then point the tool at it:
# On your MikroTik
/export file=config_export
# On your workstation
mikrotik-audit config_export.rsc
That’s it. The tool parses the export, runs all 107 checks, and prints a severity-sorted report with remediation commands.
For CI/CD pipelines or sharing with stakeholders:
# JSON output
mikrotik-audit config_export.rsc --json
# HTML report
mikrotik-audit config_export.rsc --html
# With CVE and conflict checking
mikrotik-audit config_export.rsc --cve --conflicts
A Quick Demo – Real Results From a hAP ax3
I ran the tool against an actual hAP ax3 export I had lying around. Here’s what came back:
═══ CRITICAL ═══
AUTH-001: Default admin user enabled
Path: /user admin
CVSS: 9.8
Fix: /user disable admin
FW-001: No default drop rule in forward chain
Path: /ip firewall filter
CVSS: 9.1
Fix: /ip firewall filter add action=drop chain=forward
═══ HIGH ════
SRV-001: DNS allows remote requests
Path: /ip dns
CVSS: 7.5
Fix: /ip dns set allow-remote-requests=no
WIFI-003: WPS enabled on 2.4 GHz
Path: /interface wireless
CVSS: 7.1
Fix: /interface wireless set wlan1 wps-mode=disabled
Two criticals and two highs in under three seconds. Every finding includes the exact command to fix it. This kind of audit would take 20+ minutes manually, if you knew exactly what to look for.
Your 10-Minute RouterOS Security Workflow
If you have a MikroTik router and haven’t looked at the config in a while, here’s the fastest way to check it:
| Step | Action | Command |
|---|---|---|
| 1 | Export your config from the router | /export file=audit_export |
| 2 | Run the audit | mikrotik-audit audit_export.rsc --html |
| 3 | Fix critical and high findings | Copy-paste the remediation commands from the report |
| 4 | Re-audit to verify fixes | mikrotik-audit audit_export_fixed.rsc |
Total time: about 10 minutes. Most configs I’ve tested have at least one critical gap. Many have five or more.
Maintenance and Future Plans
The tool is at v0.1.0 right now. It does its job well for offline audits, but this is the kind of project that evolves as people use it on real hardware. I plan to keep maintaining it, fixing edge cases as they come up, and adding more checks over time. If you run it on your router and find something it missed, that’s useful feedback.
Get the Tool
- GitHub: github.com/donrami/mikrotik-rsc-auditor
- PyPI:
pip install mikrotik-rsc-auditor - License: MIT
Final Thoughts
This tool won’t catch everything. No static analyzer will. But it’ll catch the obvious stuff – the things that keep showing up in incident reports. Default admin accounts. Open DNS resolvers. Missing firewall rules.
Run it today. You’ll probably find at least one critical gap. Most configs I’ve tested had five or more. Fix those, re-audit, move on.
Security is mostly about not making the same obvious mistakes over and over. This helps with that.
Now go audit your router.
— Rami
