Contents
Why local
Most "AI security scanner" tools I'd seen worked the same way: run some recon tools, ship the output to a hosted LLM API, get a report back. That bothered me for two reasons. First, scan data is sensitive — open ports, software versions, headers, sometimes fragments of source. Sending that to a third-party API for every target felt like the wrong default, especially for anyone testing infrastructure they're responsible for. Second, API costs add up fast once you're running deep scans with multiple follow-up questions per session.
So the rule I set for Argos from day one was simple: the LLM has to run on hardware I control, and nothing about the analysis step should require a network call to a model provider. Everything else — the recon tools, the CVE lookups, the report generation — had to work with that constraint, not around it.
The result is a terminal tool, built with Python and Rich for the UI, that does reconnaissance, vulnerability analysis, and CVE enrichment against a target, using a local LLM for the analysis layer. It's meant strictly for systems you own or are explicitly authorized to test, and I say so loudly in the README.
Architecture: recon → analysis → CVE lookup → reporting
The pipeline has four stages, and each one is a separate module so the LLM step can be swapped or skipped entirely without touching the rest.
Recon. A tool runner module wraps around two dozen-plus individual checks — port scanning, WHOIS, HTTP header inspection, DNS, TLS/cipher checks, WAF and CMS fingerprinting, subdomain enumeration, directory enumeration, and a set of lighter-weight probes for things like CORS misconfiguration, clickjacking, and open redirects. These are grouped into three depths (a quick pass, a fuller surface map, and a comprehensive pass) and run concurrently with a thread pool so the wait time stays reasonable even at the deepest level.
Analysis. The combined output of whichever tools ran gets handed to the LLM bridge module, which formats it into a single prompt and sends it to whichever backend is configured. The model's job is strictly interpretive: read the raw tool output and turn it into a structured verdict. It doesn't run anything itself at this stage — it reads text and writes JSON.
CVE lookup. Before the LLM call, a separate pass scans the recon output for recognizable "software name + version" patterns (web servers, databases, CMSs, common services) and queries the NVD CVE API for matches, caching results locally in SQLite for a week so repeat scans don't need another lookup. That enrichment data gets appended to what the model sees, which keeps CVE citations grounded in NVD rather than invented.
Reporting. Everything — the raw scan, the model's structured findings, and the CVE context — gets stored in a local SQLite session database, from which it can be exported to PDF, HTML, JSON, or Markdown, or diffed against a previous scan of the same target to track remediation over time.
Talking to a local model and getting structured output back
The interesting engineering problem wasn't connecting to an LLM — llama.cpp's server exposes an OpenAI-compatible endpoint and Ollama has its own simple chat API, so supporting both backends is mostly a matter of two small HTTP client functions. The hard part was getting a mid-sized local model, often quantized, to reliably return valid JSON instead of a chatty paragraph with the JSON buried somewhere in the middle.
The approach that worked was being extremely explicit in the system prompt about the exact schema expected, and then treating the model's output as untrusted text on the way back, not as drop-in JSON. A small parser strips markdown code fences if the model added them anyway, tries a direct json.loads, and falls back to regex-extracting the first {...} block if that fails. A short, representative slice of the schema looks like this:
{
"risk_level": "LOW | MEDIUM | HIGH | CRITICAL",
"summary": "string",
"vulnerabilities": [
{ "vuln_name": "string", "severity": "string", "description": "string", "fix": "string" }
]
}Keeping the model grounded was the other recurring theme. Local models, especially smaller ones, will confidently invent a CVE number if you let them. The fix wasn't clever prompting tricks — it was removing the opportunity: the prompt explicitly instructs the model not to cite a CVE unless it appears in the scan data or the NVD enrichment block it was given, and to omit a finding entirely rather than guess when the evidence is thin.
Speed on consumer GPUs was the last piece. A live token-per-second counter during streaming responses turned out to matter more for usability than raw throughput — watching "12 tokens/sec" tick up during a 30-second wait feels very different from watching a frozen terminal. Streaming also meant building the client against both llama.cpp's server-sent-events format and Ollama's newline-delimited JSON format, which differ enough that they needed separate parsers.
Limits
A local model, even a capable one, won't match a large hosted model's judgment on ambiguous findings, and the tool is explicit that its exploit suggestions are offered for review, not for blind execution — any payload runner is allowlisted and meant for the operator's own targets. CVE enrichment only covers software the regex patterns recognize, which is a reasonably long but finite list. And ultra-deep scans trade a lot of wall-clock time for coverage, which is a real cost on slower consumer hardware.
To be unambiguous about intent: this tool is built for testing systems you own or have explicit written authorization to assess. It's not meant for, and shouldn't be used against, anything else.
If you want to look at the code: github.com/Hemiize/argos