Skip to main content

Changelog

v2.2.0 — SAST vulnerability engine + structural secret detection​

Added — SAST vulnerability & misconfiguration engine​

  • --misconfig — in addition to secrets, scan source for vulnerabilities and misconfigurations.
  • --misconfig-only — scan for vulnerabilities/misconfigurations only, skipping secret detection.
  • New SAST engine detects (with taint/context gating for low false positives): SQLi, NoSQLi, command injection, code injection, SSTI, XXE, insecure deserialization, LFI/path traversal, LDAP and XPath injection, SSRF, open redirect, CORS misconfig, CSRF-disabled, prototype pollution, XSS across 11 templating engines plus DOM sinks, weak crypto (DES/RC4/ECB/static-IV/weak-RSA), insecure RNG, JWT flaws (alg:none/confusion/hardcoded secret/verify-off), TLS bypasses, timing-unsafe compares, SAML sig-off, hardcoded credentials, sensitive-data logging, and IaC/config misconfig (Terraform, Kubernetes, Dockerfile, GitHub Actions, ASP.NET web.config, WCF/SOAP).
  • 17 languages/formats: python, node/js/ts, react, php, ruby, go, java, kotlin, csharp/.NET, sql, XML/WSDL, JSP, terraform, kubernetes yaml, dockerfile.
  • Each vulnerability finding carries: Vulnerability Name, Severity, Description, Evidence (file:line), Vulnerable Code, Secure Code, Remediation, Technical Impact, Business Impact, CWE, and OWASP mapping.
  • 223 new vulnerability/misconfiguration rules, bringing the total rule count from 179 to 416 (193 secret + 223 vulnerability/misconfiguration).

Added — context-aware / structural secret detection​

  • Whole-file pass catching secrets in nested XML tags, split <key>/<value> pairs, JSON key/value objects, multi-line YAML/properties, and Base64-encoded secrets.
  • Entropy-gate fix: credential-named assignments (e.g. AM_CLIENT_SECRET=…) are now caught on the name signal with no entropy floor, so real low-entropy secrets are no longer dropped.

Added — new secret token types​

  • Slack app/user tokens, Dropbox, PlanetScale, PostHog, Supabase, Figma, GitLab runner/trigger tokens, Stripe test keys, Google OAuth refresh, Twitch, ngrok.

Changed — reporting​

  • HTML report is now collapsible — each finding is an expandable card (summary: severity + name + file:line + CWE; expand: full record incl. vulnerable/secure code, remediation, impacts) with a filter box, severity/file/name sort, and expand/collapse-all. Theme-aware, self-contained.
  • JSON, CSV, SARIF, Excel, and PDF carry all fields, including the vulnerability record.

Examples​

scan4secrets --path ./src --misconfig
scan4secrets --path . --misconfig-only --report html --output report

v2.1.3 — Branded banner + live crawl progress​

Added​

  • Startup banner with byline by m14r41 - github.com/m14r41 printed inside a rounded panel.
  • [+] / [*] status markers for every stage: rules loaded, SAST scan start/end, DAST target, threads/max_urls/timeout, wordlist seeding, deduplication, generic suppression, verification, report writes.
  • Live DAST progress bar (spinner + M/N URLs counter + elapsed time) driven by the existing progress_cb hook in crawl_and_scan. Auto-suppressed under --quiet.

v2.1.2 — Bundling, normalization, and PoC-friendly defaults​

Breaking: Secret values are now shown in full by default in every output (terminal, CSV, JSON, JSONL, HTML, Excel, PDF, SARIF). The --unsafe-show flag is removed and replaced by --mask, which inverts the behaviour for screenshots or shared transcripts. Rationale: this tool is used to produce vendor PoCs where the raw value is the proof; redacted-by-default forced an extra flag on every real run.

Changed​

  • --mask flag replaces --unsafe-show. Default is unmasked across the terminal table and every file reporter.
  • normalize_url accepts every common shape: bare host, host:port, IPv4, IPv4:port, IPv6 [::1], [::1]:port, scheme-less paths, accidental ://prefix typos, whitespace, trailing slashes. :443 stays HTTPS; other ports and raw IPs default to HTTP. Unsupported schemes (ftp://, ws://, file://) are rejected with a clear error.

Fixed​

  • PyInstaller binary bundling: Linux and Windows binaries now ship scan4secrets/config/rules.yaml via explicit --add-data. Previous --collect-all alone did not pick up package-data under --onefile, causing FileNotFoundError: '/tmp/_MEIxxx/scan4secrets/config/rules.yaml' on first run.
  • BeautifulSoup XML warning: XMLParsedAsHTMLWarning is silenced at crawler load (engine/crawler.py). Non-fatal noise on XML/RSS responses no longer pollutes the terminal.

v2.1.0 — DAST defaults + noise reduction​

Changed​

  • Wordlist seeding is now ON by default for any --url run. All 15 bundled wordlists (1279 unique paths) seed every DAST scan.
  • --wordlist FILE [FILE ...] now takes user-supplied wordlist file paths and REPLACES the bundled set for that run (was: a boolean toggle).
  • Use --wordlist-only NAME ... to restrict to specific bundled stems.
  • Use --no-wordlist to disable seeding entirely.
  • Generic-rule suppression runs by default — when a vendor-specific rule fires on a secret value, generic contextual-* / generic-high-entropy-* findings for the same value are dropped. Use --keep-generic to disable.
  • Response-header dedup is now global across the crawl — the same X-Powered-By leak is scanned once, not once per URL.

Added​

  • Inline <script> URL string-literal extraction in HTML responses.
  • Inline <style> and standalone CSS url(...), @import, /*# sourceMappingURL=*.css.map */ extraction.
  • .css.map sourcesContent parsing (matching .js.map).
  • <meta http-equiv="refresh"> URL following.
  • Binary content-type skip (image/*, video/*, audio/*, font/*, application/pdf, etc.).
  • Connection pool sized to 64 (was default 10) — no more "pool is full" warnings on --threads > 10.

v2.0.0 — Rule engine rewrite + DAST overhaul​

Breaking: complete rewrite. Old core/, output/, ui/, config/patterns.json are moved to legacy_* directories and no longer imported. The CLI shape changes (main.py becomes a shim around scan4secrets.cli:main).

Why v2 exists​

Empirical test on a benign Express repo: v1 produced 27 false positives, v2 produces 0. On a deliberately seeded leaky-repo, v2 catches the SSH private keys, PEM blocks, and Docker registry credentials that v1 was structurally incapable of detecting. The defect was the detection model itself (keyword-name match vs vendor value-shape match). v2 fixes the model.

New — detection engine​

  • YAML rules at scan4secrets/config/rules.yaml replacing the keyword-name patterns.json
  • Vendor value-shape regexes for 170+ rule IDs across cloud, CDN/edge, SCM, CI/CD, payments, e-commerce, messaging, carriers, AI/ML, email, monitoring, registries, auth, productivity, mobile/push, data, mapping, Web3, storage, VPN, QA, DB connection URIs, webhooks, and crypto
  • Aho-Corasick keyword pre-filter (pyahocorasick) for O(N) scanning across hundreds of rules
  • Shannon-entropy gate per rule (kills secret = "changeme" style noise)
  • Per-rule allowlists (regex against full line + path globs against file path)
  • Contextual fallback rules for hex tokens, UUIDs, and quoted/unquoted high-entropy values near credential-sounding names
  • Severity model: info / low / medium / high / critical, with a --fail-on CI gate
  • Deduplication keyed on (file, line, sha256(secret), rule_id) across SAST + DAST

New — DAST crawler​

  • Concurrent crawl via ThreadPoolExecutor (--threads, default 16) — replaces v1's recursive single-threaded requests.get
  • Scope control: same-eTLD+1 by default, --strict-host for exact host
  • JS source-map parsing — fetches *.js.map and scans embedded sourcesContent (no other open-source tool does this)
  • JS endpoint extraction — string literals in .js files added to the queue
  • HTTP header scanning — secrets sometimes leak in X-Powered-By, Server, custom headers
  • Authenticated DAST: --cookie, --header K:V (repeatable), --proxy, --insecure
  • Custom UA with sensible default — v1's default python-requests/X UA was blocked by most WAFs
  • Caps: --max-urls, --max-depth, --timeout

New — verification​

  • --verify flag runs one HTTP probe per finding whose rule has a verify: block, sets verified=true|false|null
  • Built-in probes for GitHub (classic + fine-grained PAT), Slack, Stripe, OpenAI
  • Trivial to add new vendors — edit the YAML rule

New — reporters​

  • SARIF 2.1.0 — GitHub Code Scanning, GitLab Security, Sonar, Defect Dojo
  • JSON + JSONL — pipelines, SIEM ingest
  • HTML v2 — severity badges, sortable columns, live filter, dark theme
  • Excel via openpyxl
  • PDF via fpdf2 (UTF-8 safe; v1's fpdf v1 crashed on non-ASCII)
  • CSV with full schema (rule_id, severity, verified, file, line, entropy, redacted, sha256, description, excerpt)
  • Redaction by default: abcd****wxyz. --unsafe-show to include raw values (use carefully)

New — packaging & CI​

  • pyproject.toml — pip install . works, scan4secrets console entry on PATH
  • Dockerfile — multi-stage build, non-root user
  • .pre-commit-hooks.yaml — drop-in for pre-commit framework
  • .github/workflows/release.yml — Windows EXE + Linux EXE binaries, Docker image to GHCR, wheel build
  • .github/workflows/ci.yml — pytest across Python 3.9-3.12

Empirical results​

Repov1 findingsgitleaksv2 findings
Plazmaz/leaky-repo (seeded)35 (mixed TP/FP)2217 (all real, incl. SSH/PEM/Docker keys v1 missed)
expressjs/express (clean)27 (all FP)00

v2 achieves 0% FP rate on benign code (vs v1's per-file FP rate of ~12.7%) and detects high-value secret classes (private keys, Docker registry auth) that v1's regex shape could never match.

v1.x (legacy)​

Original keyword-name detection model. Source available under legacy_* directories for archival reference. Not recommended for production use.