Introduction ​

OWASP ZAP is a powerful open-source DAST (Dynamic Application Security Testing) tool, but getting authenticated scanning right — especially against modern JavaScript-heavy applications built with Next.js, React, Vue, or similar frameworks — is notoriously difficult. Authentication might appear to work while the session silently dies mid-scan, producing results that look comprehensive but only tested unauthenticated surfaces.

This post documents a real-world journey of configuring ZAP's automation framework for authenticated scanning across two structurally different applications: a Next.js + NextAuth cookie-session app and a JWT-based SPA (OWASP Juice Shop). We cover authentication methods, session management strategies, verification tuning, framework header handling, scan policy optimization, and job ordering — with before/after data from actual scan reports.

The Authentication Problem ​

ZAP offers several authentication methods — form-based, JSON-based, browser-based — and multiple session management strategies. Picking the wrong combination, or misconfiguring any piece, leads to a scan that thinks it's authenticated but is actually testing as an anonymous user.

How to Tell If Authentication Is Really Working ​

ZAP's auth report statistics are the ground truth. These are the key fields to check:

stats.auth.state.loggedin — The number of times ZAP's poll verification confirmed the session was alive by matching the loggedInRegex against the poll URL's response. This is the gold standard. If this number is zero or absent, your scan is not reliably authenticated, regardless of what the summary says.

stats.auth.state.assumedin — Requests where ZAP assumed the session was valid without re-checking. A high ratio of assumedin to loggedin is normal (ZAP can't poll on every request), but loggedin should still be a meaningful number — at least dozens for a multi-minute scan.

stats.auth.state.unknown — Requests where ZAP's poll check matched neither the logged-in nor the logged-out regex. Any nonzero value here means your verification regexes don't fully cover the poll endpoint's possible responses. Investigate immediately.

stats.auth.failure and stats.auth.browser.failed — These should be absent. If present, authentication attempts are failing, likely due to timing issues or form field detection problems.

Here's what a healthy vs unhealthy auth report looks like:

Browser Auth + Autodetect: The Winning Combination ​

After testing multiple configurations, we settled on a combination that works reliably across both cookie-session and JWT-based applications.

Authentication Method: Browser ​

The browser method launches a real headless Firefox instance, navigates to the login page, and lets ZAP's form detection fill in credentials. This handles JavaScript-rendered login forms, CSRF tokens, multi-step flows, and cookie-setting redirects automatically — things that the form and json methods struggle with.

authentication:
  method: browser
  parameters:
    browserId: firefox-headless
    loginPageWait: 30          # Give SPAs time to fully render
    loginPageUrl: https://example.com/auth/signin

The loginPageWait: 30 is important for SPAs — the login form might not be in the DOM immediately. We tested with 5 seconds and saw intermittent browser.nopasswordfield / browser.nouserfield failures. At 30 seconds, ZAP reliably finds the form fields every time.

Session Management: Autodetect ​

Rather than manually templating session headers (which is fragile and error-prone), let ZAP figure out which cookies or tokens carry the session.

sessionManagement:
  method: autodetect

In our first attempt, we hand-crafted the session header:

sessionManagement:
  method: headers
  parameters:
    Cookie: "__Secure-next-auth.callback-url=...; __Secure-next-auth.session-token={%cookie:__Secure-next-auth.session-token%}"
    next-url: /auth/signin
    rsc: 1
    next-router-state-tree: <VALUE>

This caused two problems. First, next-router-state-tree: <VALUE> was a literal string placeholder, not a real value — Next.js either ignored it or returned malformed responses. Second, ZAP's own session token bookkeeping never tracked the session cookie because we'd bypassed its detection mechanism. The result: auth.state.loggedin was absent (ZAP never confirmed the session was alive), and the __Secure-next-auth.session-token cookie didn't appear in the session token stats at all.

Switching to autodetect resolved both issues immediately. ZAP discovered the session cookie on its own, tracked it across the scan, and auth.state.loggedin jumped from zero to 636 on the first clean run.

Verification: Explicit Poll Configuration ​

Verification is how ZAP confirms the session is still alive during a scan. The poll method hits a URL at regular intervals and checks the response against regexes.

verification:
  method: poll
  loggedInRegex: '"email":"user@example\.com"'
  loggedOutRegex: Please check your password or email\.
  pollFrequency: 10
  pollUnits: seconds
  pollUrl: https://example.com/api/auth/session

Never use method: autodetect for verification. When we did, ZAP picked a random API endpoint (/api/proxy/agent-approve/...) and matched on HTTP status codes (200 OK / 401 Unauthorized). Any endpoint returns 200 when the server is up — this doesn't prove authentication. The poll frequency also defaulted to 60 seconds instead of 10, reducing verification checks from ~180 to just 5 across a scan.

Always use a JSON API endpoint (session endpoint, whoami endpoint) that returns user-specific data in the response body, and match on a stable identifying field like the user's email or role.

Always have a fallback if your LLM-based config generator fails to return a pollUrl:

const verification = llmResult.pollUrl
  ? {
      method: "poll",
      loggedInRegex: llmResult.loginRegex,
      loggedOutRegex: llmResult.loggedOutRegex,
      pollFrequency: 10,
      pollUnits: "seconds",
      pollUrl: llmResult.pollUrl,
    }
  : {
      method: "poll",
      loggedInRegex: `\\Q${credentials.username}\\E`,
      loggedOutRegex: "Unauthorized|Invalid|Sign in",
      pollFrequency: 10,
      pollUnits: "seconds",
      pollUrl: customConfig.authentication.loginBackendUrl || loginPageUrl,
    };

Handling Custom Headers: Separating Auth from Business Logic ​

Modern web apps often include non-standard headers in authenticated requests — things like X-Tenant-ID, X-Organization, X-Api-Version. These get picked up during traffic analysis and need to be injected into scan requests.

But there's a critical distinction: auth headers (Authorization, Cookie) should be managed by ZAP's autodetect, while business headers should be injected via an httpsender script. Mixing them causes conflicts.

The Problem: LLM-Generated Header Injection ​

We use an LLM to analyze authenticated traffic and generate ZAP configuration. The LLM prompt asks it to identify custom headers the client sends. The problem: the LLM also picks up framework-internal headers like next-router-state-tree, rsc, and next-url — and since their values are dynamic, it returns <VALUE> as a placeholder. This literal string then gets injected on every request via the httpsender script, causing the ajax spider's browser to conflict with the injected headers.

The Fix: Filter Before Injection ​

Filter out auth headers and framework headers before generating the injection script:

const authPatterns = /^(authorization|cookie|set-cookie)$/i;

const frameworkPatterns = new RegExp(
  "^(" + [
    // Next.js
    "next-router-state-tree", "next-router-prefetch",
    "next-router-segment-prefetch", "next-url", "next-action",
    "rsc", "x-nextjs-.*",
    // Remix / React Router v7
    "x-remix-.*",
    // SvelteKit
    "x-sveltekit-.*",
    // Nuxt.js
    "x-nuxt-.*",
    // htmx
    "hx-request", "hx-boosted", "hx-current-url",
    "hx-history-restore-request", "hx-prompt",
    "hx-target", "hx-trigger", "hx-trigger-name",
    // Hotwire / Turbo
    "turbo-frame", "turbo-stream", "x-turbo-request-id",
    // Server response headers (not client-sent)
    "x-powered-by", "x-request-id", "x-correlation-id",
    "x-vercel-.*", "x-middleware-.*", "cf-.*", "server-timing",
  ].join("|") + ")$", "i"
);

const businessHeaders = Object.entries(customConfig.headers || {})
  .filter(([key]) => !authPatterns.test(key) && !frameworkPatterns.test(key));

Only generate the httpsender script if businessHeaders.length > 0. The script should inject only those filtered headers.

Impact on Ajax Spider and DOM XSS ​

Injecting next-router-state-tree: <VALUE> had a measurable negative impact. The ajax spider uses a real browser — injecting a malformed framework header into its requests caused the browser's native routing state and the injected header to conflict. In one run, the ajax spider found only 21 URLs (vs 1,707 without the injection), and DOM XSS scanning didn't fire at all because it's tied to the ajax spider's discovery.

After removing the framework header injection, the ajax spider resumed normal behavior and DOM XSS scanning returned (454 scans, 12,652 DOM gets).

Job Ordering: Maximizing Discovery and Coverage ​

The order of jobs in ZAP's automation framework has a direct impact on scan coverage. After extensive testing, here's the optimal order:

jobs:
  # 1. Scripts (ACSRF, business headers)
  - type: script  # ACSRF token registration
  - type: script  # Business header injection (if any)

  # 2. Ajax Spider FIRST — empty sites tree = max discovery + DOM XSS
  - type: spiderAjax
    parameters:
      maxDuration: 12

  # 3. HAR Imports — seed real API traffic the browser can't reach
  - type: import
    parameters:
      type: har
      fileName: /zap/wrk/chunk-0.har

  # 4. Requestor — ensure critical authenticated endpoints are tested
  - type: requestor

  # 5. CSRF handling scripts (for traditional spider + active scan)
  - type: script  # XSRF token handler
  - type: script  # Anti-CSRF form field registration

  # 6. Traditional Spider — parse everything for additional links
  - type: spider
    parameters:
      maxDuration: 12

  # 7. Active Scan
  - type: activeScan

  # 8. Wait for passive scanning to complete
  - type: passiveScan-wait

  # 9. Passive scan config (auth diagnostic rules only — AFTER wait)
  - type: passiveScan-config

  # 10. Reports
  - type: report

Why This Order Matters ​

Ajax spider first: When the sites tree is empty, the ajax spider discovers URLs by clicking through the entire UI in a real browser. If HAR imports run first, most URLs are already known, and the ajax spider finds almost nothing (22 URLs vs 1,707). More critically, DOM XSS scanning runs during the ajax spider phase — with an empty sites tree, the DOM XSS scanner launches browser instances to test each newly discovered page (454 scans). With a pre-populated tree, it doesn't launch at all.

HAR imports after ajax spider: HAR files contain real authenticated API traffic with exact parameters, query strings, and request bodies that no spider will ever construct on its own (e.g., /api/proxy/projects/5a47706852413d3d/risk-info?category=&status=&page_size=25&page=1&search=). They complement the spider's UI-level discovery with API-level coverage.

passiveScan-config after passiveScan-wait: This is critical. If passiveScan-config runs first with disableAllRules: true, all passive scan rules are turned off during the entire scan. Moving it after passiveScan-wait ensures all ~80 passive rules run during scanning (finding missing security headers, cookie issues, information disclosure, etc.), and the config change only takes effect afterward for the auth diagnostic report.

Scan Policy Optimization: Balancing Depth and Breadth ​

The Time Budget Problem ​

ZAP's active scan rules run sequentially. Each rule tests every URL in the sites tree with its configured strength. At defaultStrength: high, a typical rule sends 2-3x more payloads per URL than at medium. Combined with a large URL set (from HAR imports), this means each rule takes significantly longer.

In our testing:

Setting defaultStrength: high caused every rule to take 3-5x longer, so only 14 of ~60 rules completed before the scan time cap. The remaining ~46 rules were queued but never executed — including CSRF testing, LDAP injection, command injection, SSRF, and XXE.

The Optimal Policy ​

Keep most rules at medium (fast), boost only critical rules to high:

policyDefinition:
  defaultStrength: medium
  defaultThreshold: medium
  rules:
    - { id: 40018, strength: high, threshold: low }   # SQL Injection
    - { id: 40019, strength: high, threshold: low }   # SQL Injection (MySQL)
    - { id: 40012, strength: high, threshold: low }   # XSS (Reflected)
    - { id: 40026, strength: high, threshold: low }   # XSS (DOM)
    - { id: 90018, strength: high, threshold: low }   # Advanced SQL Injection
    - { id: 6,     strength: high, threshold: low }   # Path Traversal
    - { id: 90020, strength: high, threshold: low }   # Remote OS Command Injection
    - { id: 90037, strength: high, threshold: low }   # SSTI

Never use insane strength — it sends exponentially more payloads with diminishing returns and can consume the entire scan budget on a single rule.

Scan Tier Configuration ​

Scale scan parameters based on the desired coverage level:

const ScanTiers = {
  quick: {
    maxDepth: 5,
    maxChildren: 25,
    maxRuleDurationInMins: 2,
    maxScanDurationInMins: 10,
    maxAlertsPerRule: 5,
    maxSpiderDuration: 2,
  },
  normal: {
    maxDepth: 10,
    maxChildren: 50,
    maxRuleDurationInMins: 5,
    maxScanDurationInMins: 20,
    maxAlertsPerRule: 5,
    maxSpiderDuration: 5,
  },
  moderate: {
    maxDepth: 10,
    maxChildren: 50,
    maxRuleDurationInMins: 5,
    maxScanDurationInMins: 40,
    maxAlertsPerRule: 10,
    maxSpiderDuration: 7,
  },
  high: {
    maxDepth: 20,
    maxChildren: 75,
    maxRuleDurationInMins: 8,
    maxScanDurationInMins: 75,
    maxAlertsPerRule: 10,
    maxSpiderDuration: 12,
  },
  veryHigh: {
    maxDepth: 25,
    maxChildren: 100,
    maxRuleDurationInMins: 10,
    maxScanDurationInMins: 120,
    maxAlertsPerRule: 15,
    maxSpiderDuration: 15,
  },
};

Key tuning parameters:

maxRuleDurationInMins: Caps how long any single rule can run. Without this, a greedy rule like Path Traversal can consume 15+ minutes sending 17,000+ messages while finding the same 7 vulnerabilities it found in the first 2 minutes.

maxAlertsPerRule: Once a rule finds this many alerts, it stops. 10 is sufficient to prove a vulnerability class exists; 50 just wastes time re-confirming the same finding on different URLs.

maxScanDurationInMins: The overall cap. Set this higher than maxRuleDurationInMins × number_of_boosted_rules to ensure all rules get a turn.

Known ZAP 2.17 Issues ​

VariantMultipartFormParameters IndexOutOfBoundsException ​

Multiple active scan rules crash with IndexOutOfBoundsException: Index -1 out of bounds for length 1 at VariantMultipartFormParameters.setParameter when they encounter certain multipart form parameter structures. Affected rules include XSS, Path Traversal, External Redirect, SSI, ShellShock, and SQL Injection timing variants. Each rule recovers and continues scanning other URLs, so the impact is limited to losing coverage on that specific request per rule.

Tech Detection Passive Scanner Performance ​

The Tech Detection passive scan rule can take 30-55 seconds to process large JavaScript bundles (100KB+). This produces warnings in the log but doesn't affect scan results.

Summary: The Final Configuration Checklist ​

1. Auth method: browser with loginPageWait: 30
2. Session management: autodetect (never hand-craft session headers)
3. Verification: poll with a JSON API endpoint, body-content regex, 10-second frequency (never autodetect)
4. Custom headers: Filter out authorization, cookie, and all framework headers; only inject genuine business headers
5. Job order: Ajax spider → HAR imports → Requestor → CSRF scripts → Traditional spider → Active scan → Passive scan wait → Config → Reports
6. Scan policy: defaultStrength: medium with selective high on 8 critical rules
7. Time caps: maxRuleDurationInMins: 5-10, maxAlertsPerRule: 10-15, total scan time proportional to rule count
8. Passive scan: Keep all rules enabled during scanning; move passiveScan-config after passiveScan-wait

With this configuration, we achieved 2,112 confirmed authenticated verification checks across a 75-minute scan testing 71,768 URL/rule combinations, with zero authentication failures, zero unknown auth states, and full passive scan and DOM XSS coverage.

The Problem ​

If you're integrating browser-recorded traffic into OWASP ZAP's Automation Framework, you've probably hit the size wall. A single user session captured as HAR from Chrome DevTools or a browser extension can produce files of 25 MB or more. Most of that bulk comes from fields ZAP never reads: JavaScript call stacks in _initiator, full response bodies, Chrome-specific metadata, timing breakdowns, and duplicated requests to static assets.

This document covers what you can safely strip, what you must keep, and the edge cases to watch for — verified against ZAP's actual source code.

How ZAP Imports HAR Files ​

ZAP's HAR import lives in the exim (Import/Export) add-on. The import pipeline is straightforward:

HarReader.readFromFile(file)
    → HarLog.entries()
        → stream/filter/map
            → HarUtils.createHttpMessage(entry)
                → persist to Sites Tree + History

The current implementation (as of zap-extensions main branch) uses the de.sstoehr.harreader library, which deserializes HAR JSON via Jackson with FAIL_ON_UNKNOWN_PROPERTIES=false. This means unknown or custom fields are silently ignored — they won't cause parse errors.

The key method chain in HarImporter.java:

private static List<HarEntry> preProcessHarEntries(HarLog log) {
    return log.entries().stream()
        .filter(HarImporter::entryIsNotLocalPrivate)
        .map(HarImporter::correctHttpVersions)
        .filter(HarImporter::entryHasUsableHttpVersion)
        .toList();
}

ZAP calls log.entries() — never log.pages(), log.creator(), log.browser(), or log.version(). The entries stream is filtered (skip about:, chrome:, edge: URLs), HTTP versions are corrected, and each entry is converted to an HttpMessage via HarUtils.createHttpMessage().

Source reference: HarImporter.java on GitHub

What ZAP Reads vs. Ignores ​

Fields ZAP actively uses ​

These are consumed by HarUtils.createHttpMessage() to reconstruct HTTP messages:

Fields ZAP ignores ​

The pages array — keep the key, clear the content ​

The HAR 1.2 spec lists pages as optional. ZAP's importer never calls log.pages(). However, keeping "pages": [] rather than removing the key entirely is safer for schema validation and compatibility with other tools that might process the same HAR.

Preprocessing Script ​

The following jq-based script strips everything ZAP doesn't need. It handles domain filtering, static asset exclusion, response removal, header reduction, and deduplication.

const jqScript = `
# 1. Filter to target domain(s) only
.log.entries |= map(select(
  .request.url | (${domainFilter})
))

# 2. Exclude static assets, bundled JS, analytics, and third-party trackers
| .log.entries |= map(select(
  .request.url
  | test("\\\\.(png|jpg|jpeg|css|js|woff2?|svg|ico|gif|ttf|eot)$
         |polyfills
         |runtime\\\\.
         |main\\\\.
         |vendor\\\\.
         |chunk\\\\.
         |bundle\\\\.
         |assets/
         |static/
         |cdn\\\\.
         |fonts\\\\.
         |analytics
         |googletagmanager
         |hotjar
         |intercom
         |segment\\\\.io
         |sentry\\\\.io
         |clarity\\\\.ms
         |facebook\\\\.com
         |twitter\\\\.com
         |linkedin\\\\.com"; "ix")
  | not
))

# 3. Strip full response objects (ZAP active scanner generates its own)
| del(.log.entries[].response)

# 4. Remove all non-essential entry-level fields
| .log.entries[] |= del(
  .timings,
  .cache,
  .serverIPAddress,
  ._connectionId,
  ._initiator,
  ._priority,
  .time,
  .pageref
)

# 5. Keep only Content-Type and Accept headers (minimal for ZAP)
| .log.entries[].request.headers |= map(select(
  (.name | ascii_downcase) == "content-type"
  or (.name | ascii_downcase) == "accept"
))

# 6. Deduplicate by method + base URL (without query params)
| .log.entries |= (
  sort_by(.request.method + (.request.url | split("?")[0]))
  | group_by(.request.method + (.request.url | split("?")[0]))
  | map(.[0])
)

# 7. Add stub responses (ZAP requires response objects to exist)
| .log.entries[] |= if .response == null then . + {
    "response": {
      "status": 200,
      "statusText": "OK",
      "httpVersion": "HTTP/1.1",
      "headers": [],
      "cookies": [],
      "content": {"size": 0, "mimeType": "application/json"},
      "redirectURL": "",
      "headersSize": -1,
      "bodySize": -1
    }
  } else . end
`;

What each step does ​

Step 1 — Domain filter: Keeps only requests to your target application. Without this, you'll import requests to CDNs, analytics services, and third-party APIs that pollute ZAP's Sites Tree.

Step 2 — Static asset exclusion: Removes requests for images, stylesheets, fonts, JS bundles, and known analytics/tracking endpoints. These don't have security-relevant server-side logic and would generate false positives in active scanning.

Step 3 — Response stripping: The most aggressive size reduction. Responses (especially large JSON payloads) are the biggest contributors to HAR file size. ZAP's active scanner generates its own requests and analyzes its own responses, so recorded responses are only needed for passive scanning. If passive scan coverage matters, keep responses for API endpoints and strip them only for non-API resources.

Step 4 — Metadata cleanup: Removes Chrome DevTools fields (_initiator, _priority, _connectionId), timing breakdowns, cache state, and page references. None of these are read by ZAP.

Step 5 — Header reduction: Keeps only Content-Type and Accept from request headers. ZAP reconstructs its own headers during active scanning. This is aggressive — if you need ZAP to replay exact authentication headers (e.g., Authorization, Cookie, custom auth tokens), add those to the select filter.

Step 6 — Deduplication: Groups entries by method + URL path (ignoring query strings) and keeps the first occurrence. This collapses repeated API calls to the same endpoint.

Step 7 — Stub responses: ZAP's HarReader expects a response object on each entry. Since step 3 deleted them, this adds minimal valid stubs.

Deduplication Caveat: Query Parameter Collapse ​

The dedup step groups by method + URL_without_query. This means:

GET /api/users?id=1        → kept
GET /api/users?id=2        → dropped (same group)
GET /api/users?role=admin  → dropped (same group)

For most CRUD APIs this is fine — ZAP's active scanner will fuzz the parameters it finds on the surviving entry. But if different query parameter names trigger different server-side code paths (e.g., /api/admin?action=delete vs /api/admin?action=export), you'll lose coverage.

Mitigation options:

• Group by method + full URL instead (keeps all unique query combos, but less dedup).
• Group by method + URL path + sorted query param names (dedupes only when the same parameter names appear, regardless of values).

The right choice depends on your application's routing.

Chrome Extension HAR vs. DevTools HAR ​

If you're using a custom Chrome extension to capture traffic (instead of DevTools' "Export HAR"), the entries are structurally equivalent for ZAP but differ in metadata:

Header case sensitivity ​

HTTP header names are case-insensitive per RFC 9110 §5.1. Content-Type, content-type, and CONTENT-TYPE are identical. ZAP handles this correctly — its HttpHeader class does case-insensitive lookups. HTTP/2 actually mandates lowercase header names, so an all-lowercase HAR is arguably more correct than DevTools' mixed-case output.

httpVersion case ​

ZAP's HarImporter uses containsIgnoreCase() for version checks, so "http/1.1" and "HTTP/1.1" are both accepted.

Missing response bodies ​

This is the one difference that affects scan quality. If your extension doesn't capture response bodies, ZAP's passive scanners lose visibility into reflected content, information disclosure, framework fingerprinting, and other response-body-dependent checks. The active scanner is less affected since it generates and analyzes its own request/response pairs.

The Chrome webRequest API doesn't expose response bodies. To capture them, your extension would need to use the chrome.debugger API (which shows a "browser is being debugged" banner) or a Service Worker-based approach with the chrome.declarativeNetRequest API.

ZAP Automation Framework Integration ​

A typical automation YAML that imports preprocessed HAR files:

env:
  vars:
    hosts: &ref_0
      - http://target-app:8080
  contexts:
    - name: target-context
      urls: *ref_0
      includePaths:
        - http://target-app:8080.*
      excludePaths:
        - .*\.png
        - .*\.css
        - .*\.js

jobs:
  # Import preprocessed HAR(s) — seeds the Sites Tree
  - name: import-har
    type: import
    parameters:
      type: har
      fileName: /zap/wrk/preprocessed-traffic.har

  # Spider from discovered URLs
  - name: spider
    type: spider
    parameters:
      context: target-context
      maxDepth: 10
      maxDuration: 2

  # Active scan all discovered endpoints
  - name: activeScan
    type: activeScan
    parameters:
      context: target-context
      maxScanDurationInMins: 30

  # Wait for passive scan to finish
  - name: passiveScan-wait
    type: passiveScan-wait

  # Generate report
  - name: report
    type: report
    parameters:
      template: traditional-json
      reportDir: /zap/reports

The HAR import seeds ZAP's Sites Tree with your application's API surface — every URL, method, header, and parameter from real user traffic. The spider then discovers additional linked resources, and the active scanner fuzzes all known endpoints.

Size Reduction Results ​

On a real-world SPA with ~400 API calls across a full user session:

The dominant size contributors are response bodies (step 3) and _initiator call stacks (step 4). Together, these two steps account for ~90% of the reduction.

Quick Reference: What's Safe to Strip ​

✓ Safe to remove          ✓ Must keep              ⚠️ Depends on your needs
─────────────────────────  ─────────────────────────  ────────────────────────────
_initiator                 request.method             response.content.text
_priority                  request.url                  (needed for passive scan)
_resourceType              request.httpVersion        request.headers (auth)
_transferSize              request.postData             (keep auth/session headers
_connectionId              request.queryString           if needed)
serverIPAddress            request.cookies            response.headers
connection                 response.status              (needed for passive scan)
timings                    response.statusText
cache                      response.httpVersion
time                       response.content.mimeType
pageref                    response.cookies
log.pages[] contents       response.redirectURL
log.creator
log.browser

Key Takeaways ​

1. ZAP only reads entries — specifically request.* and response.* within each entry. The pages array, HAR metadata, and all underscore-prefixed custom fields are ignored.

2. Keep "pages": [] — don't delete the key entirely. It maintains HAR schema validity for other tools.

3. Response stripping is the biggest win for file size, but it trades away passive scan coverage. Strip responses for size-critical pipelines; keep them if passive scan depth matters.

4. Header names are case-insensitive — lowercase, mixed-case, or uppercase all work identically in ZAP.

5. Watch your dedup strategy — grouping by URL path without query params can collapse endpoints that have different server-side behavior based on parameter names.

6. Stub responses are required — if you strip responses, add minimal valid response objects back. ZAP's HAR reader expects them to exist.

7. The underlying library is lenient — ZAP uses de.sstoehr.harreader (Jackson-based, FAIL_ON_UNKNOWN_PROPERTIES=false), so extra fields won't cause parse errors. But don't rely on this for correctness — strip what's unnecessary to keep file sizes manageable.

Why We Built Corefix ​

HAR preprocessing is one layer of a larger pipeline. Getting it right — domain filtering, response handling, deduplication strategy, stub generation — requires understanding both ZAP's internals and your application's traffic patterns. Most teams don't have time to read HarImporter.java source code and iterate through edge cases.

Corefix handles this pipeline automatically. HAR files recorded by the Corefix Extension are preprocessed, filtered, and validated before they reach ZAP — with the right headers preserved, the right responses stripped, and scan scope correctly bounded to your application. No manual jq scripts. No silent empty imports.

The scan should find vulnerabilities. The tooling should get out of the way.

Verified against ZAP exim add-on source (HarImporter.java, HarUtils.java) on the main branch of zaproxy/zap-extensions. ZAP version context: 2.17.0+.

The problem with scanner-only discovery ​

Dynamic Application Security Testing (DAST) tools like OWASP ZAP are powerful, but they share a fundamental limitation: they can only test what they can find. ZAP's built-in spiders — both the traditional crawler and the Ajax spider — navigate applications by following links, parsing HTML, and executing JavaScript. For modern single-page applications (SPAs) built with Angular, React, or Vue, this approach misses a significant portion of the attack surface.

API endpoints behind form submissions, multi-step checkout flows, authenticated-only routes, WebSocket interactions, and dynamic client-side routing all create blind spots. The scanner dutifully tests what it discovers, but if it never discovers your /api/BasketItems/ POST endpoint or your /rest/order-history route, those remain untested.

This post documents a practical approach to solving this: recording real browser traffic as HAR (HTTP Archive) files using the Corefix Extension, preprocessing them, and feeding them into ZAP's automation framework to dramatically expand scan coverage.

Architecture overview ​

The approach works in three stages:

1. Record — The Corefix Extension captures all HTTP traffic as the tester manually walks through the application, exercising authenticated flows, API calls, and multi-step processes.
2. Preprocess — The raw HAR is cleaned (removing static assets, deduplicating entries, splitting into manageable chunks) and validated to ensure entries actually exist before passing to ZAP.
3. Replay & scan — ZAP's automation framework imports the HAR chunks, replays key endpoints via the requestor job, then spiders and actively scans the vastly expanded site tree.

The key insight is that human testers naturally exercise the application in ways automated crawlers cannot. By capturing that traffic and feeding it to ZAP, you combine human discovery with automated vulnerability detection.

Target application ​

All testing was performed against OWASP Juice Shop, a deliberately vulnerable Node.js/Angular application running on http://74.225.252.175:3000. Juice Shop is an ideal test subject because it has a rich SPA frontend with many API endpoints, authentication flows, and business logic routes that are invisible to traditional crawlers.

Test methodology ​

Two scans were performed against the same target with identical authentication configuration:

• Scan A (baseline): Standard ZAP automation with spider + Ajax spider + active scan. No HAR context provided.
• Scan B (HAR-augmented): Same scan pipeline, but seeded with 77 HAR entries captured via the Corefix Extension, plus a requestor job replaying key authenticated endpoints.

Both scans used the same ZAP version (2.17.0), the same authentication credentials (admin@juice-sh.op), and the same active scan policy.

Recording traffic with the Corefix Extension ​

The Corefix Extension acts as a transparent proxy or browser instrumentation layer that captures all HTTP/HTTPS traffic during a manual testing session. The recording covered:

• Full authentication flow (login, token acquisition)
• Product browsing and search
• Basket operations (add, remove, view)
• Checkout flow (address, delivery, payment, order confirmation)
• Profile management and photo upload
• Admin panel access
• Feedback and complaint submission
• Order tracking and history
• Captcha interactions
• WebSocket connections

The recording session produced a HAR file that was then split into 5 time-based chunks for parallel import:

chunk_2026-06-16T14-14-28-993Z-clean-0.har
chunk_2026-06-16T14-15-29-604Z-clean-1.har
chunk_2026-06-16T14-16-28-957Z-clean-2.har
chunk_2026-06-16T14-17-28-918Z-clean-3.har
chunk_2026-06-16T14-18-08-729Z-clean-4.har

Validating HAR files before scanning ​

A critical lesson learned during this experiment: always validate that your preprocessed HAR files contain actual entries before kicking off a scan. An initial run appeared to succeed but stats.exim.importer.har.count was 0 — the preprocessing pipeline had silently produced empty files.

The fix is simple:

# Verify each chunk has entries
for f in /zap/wrk/chunk_*-clean-*.har; do
  count=$(jq '.log.entries | length' "$f")
  echo "$f: $count entries"
  if [ "$count" -eq 0 ]; then
    echo "ERROR: Empty HAR file detected. Fix preprocessing before scanning."
    exit 1
  fi
done

ZAP automation framework configuration ​

Authentication setup ​

Both scans used JSON-based authentication with header-based session management. An important configuration difference in the HAR-augmented scan was switching from cookie-based to Bearer token session handling, which better matches Juice Shop's JWT architecture:

sessionManagement:
  method: headers
  parameters:
    Authorization: Bearer {%json:authentication.token%}

Job pipeline ​

The HAR-augmented automation YAML follows this job sequence:

passiveScan-config  →  Configure passive rules BEFORE any traffic
import-har (×5)     →  Seed site tree with recorded traffic
requestor           →  Replay authenticated endpoints (GET + POST)
spider              →  Crawl discovered links
ajaxSpider          →  JavaScript-aware crawling
activeScan          →  Vulnerability testing
passiveScan-wait    →  Wait for passive analysis
report (×3)         →  JSON + Auth-JSON + HTML reports

The requestor job ​

Beyond importing HAR files, the requestor job explicitly replays key authenticated endpoints to ensure they appear in ZAP's site tree with valid session tokens. This is especially important for POST endpoints that the spider cannot discover:

- name: requestor
  type: requestor
  parameters:
    user: zap-user
    context: ZAP-Context-1781619657762
  requests:
    - url: http://target:3000/api/BasketItems/
      method: POST
      data: '{"ProductId":1,"BasketId":"1","quantity":1}'
    - url: http://target:3000/api/Feedbacks/
      method: POST
      data: '{"UserId":1,"captchaId":0,"captcha":"","comment":"test","rating":3}'
    # ... 40+ additional endpoints

Active scan policy tuning ​

The scan policy was configured with targeted rule overrides for high-value vulnerability classes:

Results: head-to-head comparison ​

Discovery metrics ​

The HAR import seeded ZAP with 77 real HTTP transactions, which the spider then used as starting points to discover 3,866 additional URLs — a 215× increase over the 18 URLs found through crawling alone.

New vulnerability classes discovered ​

The most significant outcome: three entirely new active scan vulnerability classes appeared only in the HAR-augmented scan:

These 28 high-severity findings were invisible to the baseline scan because the vulnerable endpoints were never discovered by the spider. The HAR file contained the API calls that exercise these code paths, giving ZAP the context it needed to test them.

Passive scan alert growth ​

The expanded traffic volume also dramatically increased passive scan findings:

Visual comparison: passive scan growth ​

The increase in URL discovery directly translated into dramatically broader passive scan coverage. Once HAR traffic seeded authenticated routes, API endpoints, and application workflows that traditional crawling could not reach, ZAP began observing significantly more responses across the application.

Several passive scan categories experienced substantial growth. Security header findings increased from 2,356 to 17,844 alerts. Timestamp disclosures grew from 242 to 7,770. Missing Content Security Policy findings expanded from just 22 alerts to 3,806. Site isolation observations increased from 44 to 7,639, while modern web application detections grew from 7 to 3,765.

Most importantly, entirely new finding categories appeared. Application Error Disclosure (90022), which was absent in the baseline scan, surfaced 19 findings after HAR replay expanded the scanner's visibility into deeper application functionality.

The chart above illustrates how additional application context translates directly into broader passive security coverage. While passive findings do not always indicate exploitable vulnerabilities, they provide valuable visibility into security posture, misconfigurations, information disclosure risks, and missing defensive controls that would otherwise remain hidden.

Key observation ​

The growth in passive scan volume was not caused by more aggressive scanning. It was caused by better discovery. The scanner simply had access to a much larger portion of the application after importing 77 authenticated HAR entries and replaying critical workflows through the requestor job.

This reinforces a core lesson from the experiment:

Coverage quality is often more important than scanner aggressiveness.

A scanner cannot analyze endpoints it never discovers. HAR augmentation dramatically expands that visibility.

New content types discovered ​

The HAR-augmented scan also discovered content types that the baseline scan never encountered:

• application/pdf — downloadable invoice/order documents
• application/octet-stream — file download endpoints
• text/markdown — documentation/legal text endpoints
• HTTP 201 (Created) responses — successful resource creation
• HTTP 404 (Not Found) responses — error handling paths

Lessons learned ​

1. Validate HAR import before scanning ​

The stats.exim.importer.har.count statistic is your early-exit check. If it reads 0 after the import jobs run, kill the scan immediately — your HAR files are empty or the paths are wrong. Don't wait 45+ minutes for a scan that has no additional context.

2. passiveScan-config must come first ​

A subtle but critical ordering issue: if passiveScan-config appears after passiveScan-wait, the configuration is applied after all passive scanning has completed — making it useless. The config job must be the first job in the pipeline, before any traffic-generating jobs.

3. Bearer tokens vs cookies matter ​

Juice Shop uses JWT authentication. The baseline scan used cookie-based session management (Cookie: token=...), while the HAR-augmented scan used the correct Authorization: Bearer ... header. This seemingly small change ensures all authenticated requests are properly credentialed, improving coverage of auth-protected endpoints.

4. Budget time for expanded attack surface ​

The baseline scan completed in ~12 minutes of active scanning. The HAR-augmented scan hit its 45-minute timeout and was stopped with approximately 15 scanner rules never executing. When you dramatically expand the URL tree, you must proportionally increase maxScanDurationInMins and maxRuleDurationInMins to accommodate the larger surface.

5. Exclude destructive endpoints ​

Active scanning sends malicious payloads to every discovered endpoint. Without exclude patterns, the scanner can hit password-change, account-deletion, or logout endpoints, breaking the authenticated session mid-scan. Always add excludePaths for these:

excludePaths:
  - .*\/rest\/user\/change-password.*
  - .*\/rest\/user\/reset-password.*
  - .*\/rest\/user\/erasure-request.*
  - .*\/dataerasure.*

6. POST endpoints need explicit requestor entries ​

The spider discovers URLs by parsing responses, but it cannot infer POST request body formats. Endpoints like /api/BasketItems/ (POST), /api/Feedbacks/ (POST), and /api/Complaints (POST) must be explicitly defined in the requestor job with sample payloads.

7. Technology scoping reduces noise ​

Setting the technology include list to match your actual stack (Node.js, MongoDB, Linux) tells ZAP to skip irrelevant checks (ASP.NET, PHP, Java-specific rules), reducing scan time and network load without sacrificing relevant coverage:

technology:
  include:
    - Db.CouchDB
    - Db.MongoDB
    - Language.JavaScript
    - Language.JavaScript.NodeJS
    - OS.Linux
    - SCM.Git
    - WS.Node

Recommended automation YAML structure ​

Phase 0: Configuration
  ── passiveScan-config (BEFORE any traffic)
  ── script setup (ACSRF tokens, custom hooks)

Phase 1: Seed
  ── import HAR chunks (×N)

Phase 2: Replay
  ── requestor (GET + POST authenticated endpoints)

Phase 3: Crawl
  ── spider (traditional, 10–15 min)
  ── ajaxSpider (JS-aware, 8–12 min)

Phase 4: Attack
  ── activeScan (tuned policy, 60–120 min budget)

Phase 5: Report
  ── passiveScan-wait
  ── report generation (JSON + HTML)

Why We Built Corefix ​

The results speak for themselves: recording browser traffic with the Corefix Extension and feeding it into ZAP's automation framework produced a 215× increase in URL discovery, a 3.7× increase in active scan coverage, and 28 new high-severity vulnerability findings that were completely invisible to the baseline scan.

The technique requires minimal additional effort — a 5-minute manual walkthrough of the application generates enough HAR data to transform scan quality. For any modern SPA or API-heavy application, this approach should be considered standard practice rather than optional enhancement.

The investment is a few minutes of manual browsing and a validated preprocessing pipeline. The return is dramatically better security coverage and findings that matter.

Your scanner should find vulnerabilities, not miss them because it never discovered the endpoints.

Tested with OWASP ZAP 2.17.0 against OWASP Juice Shop. Scan date: June 16, 2026.

As Corefix integrates AI into its vulnerability management pipeline, we needed to answer a deceptively simple question: which models actually perform well on real-world security analysis tasks?

We ran a systematic benchmark against the chef repository — a forked version of the original Chef project, approximately 2000 commits behind upstream — producing 247 findings across secrets, dependency vulnerabilities, Semgrep findings, and IaC findings. We fed these through our full AI enrichment pipeline across 21 models and measured what happened.

The results were not what we expected.

What We Tested ​

Every model ran the same four-task pipeline against the complete 247-finding dataset:

• Deduplication: collapsing duplicate findings across scanners
• Enrichment: adding context, severity reasoning, and remediation guidance to each finding
• Cross-scanner correlation: linking related findings from different tools — for example, a secret finding correlating with a dependency vulnerability in the same service
• Attack chain identification: identifying sequences of findings that could be chained into an exploitable attack path

Processing time reflects how long each model took to complete the full pipeline on the 247-finding dataset. This is initial benchmarking — results will vary as prompts and pipeline logic are refined.

Processing Time Results ​

Key Observations ​

The spread is wider than expected across all tiers:

• Fastest overall: Bedrock Qwen 230B at 63 seconds — but attack chain identification was absent from results
• Best speed and quality balance: Bedrock GPT OSS 120B at 68 seconds, with strong overall task completion
• Fast OpenAI options: GPT-4o Nano at 80 seconds, GPT-5.4 Mini at 87 seconds
• Slowest models: Bedrock GLM 5 at 503 seconds, GPT-5 Mini at 460 seconds, Claude Sonnet 4.6 at 410 seconds

The counterintuitive finding: model size does not predict processing time. GPT-5 Mini (460s) is nearly as slow as full GPT-5 (410s) and significantly slower than GPT-5.4 (226s). Nano tiers cluster between 80–380 seconds — a 4x range that reflects rate limiting and infrastructure variance as much as model capability.

Recommendations and Known Issues ​

Based on testing across all providers, here is what we found:

Kimi Do not use kimi-k2.5 or kimi-2.6 directly via their API — they are being overloaded or rate-limited despite Tier 1 access. Use k2.5 via AWS Bedrock instead.

GLM Models All direct GLM model calls are resulting in 429 or 524 errors. Avoid calling GLM models directly. Use Bedrock for GLM 5, GLM 4.7, and GLM 4.7-Flash. Note that GLM 4.7 can be used for coding tasks but concurrency is limited to 1.

Claude on Bedrock Bedrock Anthropic Haiku is working, but the model must be activated via AWS with a support ticket specifying your use case before it can be used.

JSON Response Format GPT OSS 120B and Claude APIs do not support type: json_object as a response format. If jsonFormat=false, add explicit JSON formatting rules to the system prompt. The custom JSON parser will handle extracting structured output from the model response.

GPT Models All GPT models including nano and mini variants are working. Previous issues with nano and mini variants are resolved.

Context and Token Limits No context-related issues or max token errors observed — everything appears stable on this front.

Why We Built Corefix ​

AI-assisted security analysis is only as good as the pipeline surrounding the model. Raw processing time is one dimension — but attack chain identification, cross-scanner correlation, and enrichment quality all depend on how findings are assembled and presented before the model ever sees them.

Corefix wraps model selection with full context assembly. Every finding gets the surrounding code, dependency context, framework version, and scanner evidence before the LLM processes it. We run this pipeline across all major providers — Bedrock, OpenAI, Anthropic direct — and select the optimal model per task type based on benchmarks like these.

The practical result: you don't choose a model or write prompt templates. Corefix handles selection, context assembly, and quality control. You get enriched findings with remediation guidance, deduplicated across scanners, with attack chains identified automatically.

What took us weeks of benchmarking to determine, Corefix handles on every scan.

Corefix integrates AI enrichment across all major models and providers. Try a free scan → to see AI-powered security analysis on your actual findings.

Point a scanner at a URL and it floods your app with requests, misses half the attack surface because it couldn't log in, and produces reports full of noise. The configuration burden falls entirely on you — write the auth scripts, specify the CSRF fields, identify the login selectors, handle the cookie banners. Do it wrong and the scanner never gets past the landing page.

We built a smarter layer that sits in front of ZAP. Before a single probe fires, our system reads your application the way a developer would — understands its architecture, identifies how authentication works, handles the quirks — and hands ZAP a complete, accurate context. The result is broader coverage, real authenticated scanning, and no manual setup.

This is the engineering story behind that layer.

The Problem with Naive Scanning ​

Web applications today aren't what they were a decade ago. A large and growing share are Single Page Applications — Angular, React, Vue — that serve an essentially empty HTML shell and build the entire UI in the browser via JavaScript. When a scanner fetches https://app.example.com/#/login, the server responds with a few hundred bytes of boilerplate. There's no login form. There are no input fields. There's nothing to scan.

Traditional scanners — and naive automation — treat this as either a blank page or silently proceed with wrong assumptions. They submit requests to nonexistent endpoints. They can't find the password field. They never authenticate, so they never see the parts of your app that actually need testing.

The right approach is to understand what kind of application you're dealing with before you do anything else.

Step 1: Recognising the Application Architecture ​

The first thing our system does after fetching a login page is determine whether it's a classic server-rendered app or a JavaScript SPA. This matters for every decision that follows.

A naive check — looking for <form> tags and <input> elements — fails immediately for SPAs. Angular applications, for example, use custom element tags like <app-root> rather than standard HTML containers with known IDs.

The reliable signals are different:

• Custom root elements like <app-root>, <div id="root">, <div id="__next">
• SPA script bundles — main.js, polyfills.js, chunked JS files with hashed names
• Minimal body content — once scripts and link tags are stripped, almost nothing is left

When these signals align, we know we're dealing with a SPA and everything downstream changes accordingly: selector discovery, CSRF detection, authentication flow — all need a different approach.

Step 2: CSRF Detection Without Guessing ​

CSRF protection is one of the most varied aspects of web authentication. Getting it wrong means the scanner submits requests that the server rejects as forgeries — and the scan never actually tests anything.

We detect CSRF mechanisms in priority order, deterministically, before any AI analysis runs. This gives downstream components high-confidence ground truth rather than inferences.

Priority 1 — Response headers. Some frameworks send a fresh CSRF token directly in a response header (X-CSRF-Token, X-XSRF-TOKEN, etc.). If this header is present, the mechanism is unambiguous.

Priority 2 — Double submit cookies. Angular's built-in CSRF protection sets an XSRF-TOKEN cookie that JavaScript reads and echoes as an X-XSRF-TOKEN request header. We detect this from the Set-Cookie response header and derive the correct header name from the cookie name, rather than hardcoding assumptions.

One subtlety that matters here: if the CSRF cookie is flagged HttpOnly, JavaScript cannot read it, so the double-submit pattern breaks. We surface this as a warning rather than silently proceeding.

Priority 3 — HTML hidden fields. Classic server-rendered apps embed tokens like _csrf or authenticity_token directly in form markup. We extract these with purpose-built regex that handles both attribute orderings (name before value and value before name) and generate the ZAP-compatible extraction regex for each field automatically.

Priority 4 — Meta tags. Laravel, Rails, and Django commonly inject CSRF tokens into <meta name="csrf-token"> tags, bypassing form fields entirely.

For SPAs, the honest answer is often that CSRF fields won't be present in the initial HTML at all — the token only appears after JavaScript runs and the first authenticated API call fires. Our system accounts for this by re-running detection after the browser has rendered the page.

Step 3: AI Analysis of the Login Page ​

With architecture detected and CSRF ground truth established, we pass the page to an AI analysis step. The AI receives the HTML (or rendered DOM for SPAs), response headers, URLs extracted from script references and fetch calls, and the deterministic CSRF findings.

The AI's job is to identify:

• CSS selectors for the username field, password field, and submit button
• The backend endpoint the login form submits to (formAction) and its method and content type
• Any CAPTCHA presence and type
• Hidden form fields that aren't CSRF or credentials (some apps send additional body parameters)
• Whether a modal, cookie banner, or welcome overlay is blocking the form

That last point (the overlay selector) turns out to be surprisingly important in practice.

For static HTML, the AI works from server-rendered markup and produces reliable results. For SPAs, the selectors and form action are at best educated guesses at this stage, because the actual DOM doesn't exist yet. The AI knows this: it marks its confidence accordingly, and the system triggers a second pass.

Step 4: Headless Browser Rendering for SPAs ​

When the application is a SPA and the AI couldn't confidently identify selectors, we launch a headless Chromium browser, navigate to the login URL, and wait for the JavaScript to fully execute.

This gives us things the static fetch never could:

• The real rendered DOM with actual input elements present
• Intercepted XHR and fetch calls — when we later simulate a login, we'll see exactly what endpoint the app posts to and what headers and body it sends
• Runtime cookies — Angular sets XSRF-TOKEN after page load, not in the initial response, so this is often the only way to detect it

After rendering, we re-run the AI analysis on the live DOM. Now the selectors are real. The form action comes from intercepted network traffic, not inference. The CSRF cookie is present in the browser's cookie jar.

Step 5: Dismissing Overlays Before Interacting ​

Modern web applications have a habit of greeting first-time visitors with things that block the page: cookie consent banners, welcome dialogs, promotional modals. To a headless browser trying to find and fill a login form, these are invisible walls. Clicks on the email field get intercepted by the overlay, or the form itself never renders until the banner is acknowledged.

The AI identifies the dismiss selector for any overlay it detects. Our dismissal logic then tries to close it before attempting any form interaction, with a layered fallback strategy:

1. Click the AI-identified dismiss button (most specific, most likely to be correct)
2. Try a library of framework-specific generic selectors — Angular Material dialogs, Bootstrap modals, common cookie consent classes
3. Send an Escape key press, which closes many CDK and Bootstrap modals that have no obvious button
4. As a last resort, remove the overlay elements from the DOM entirely and clear any pointer-blocking CSS the framework left behind

Multiple attempts are necessary because overlay animations introduce timing: a button click may register before the dismiss handler is attached, or one overlay's dismissal reveals another underneath it.

Step 6: Detecting OAuth Redirects ​

One more edge case that breaks naive automation: some SPAs don't have a login form at all. Navigating to their #/login route triggers a client-side redirect to an external OAuth or SSO provider — Google, Microsoft, Okta.

This redirect is invisible at the HTTP layer. The server returns a 200 with the app shell. Then JavaScript runs, the router fires, and window.location gets set to https://accounts.google.com/.... To a traditional scanner this looks like a successful page load.

We detect it by comparing the browser's current domain after JavaScript has fully executed against the original login domain. If they differ, the application is using external OAuth and we signal this explicitly — triggering a different handling path — rather than proceeding with broken selector assumptions.

What ZAP Gets at the End ​

By the time we hand off to ZAP, the context is complete:

• Authentication endpoint, method, and content type
• Input field selectors resolved against the real rendered DOM
• CSRF mechanism identified and the correct extraction regex generated
• Session tokens and cookies from a successful login
• Scope correctly bounded to the application's domain

ZAP doesn't need to figure any of this out. It doesn't need to handle the SPA rendering, the overlay dismissal, the CSRF detection, or the OAuth detection. It receives a fully configured context and can focus entirely on what it's good at: systematic vulnerability testing across an authenticated, properly scoped session.

Why We Built Corefix ​

The gap between "running a scanner" and "running a scanner that actually tests your application" is larger than most people realise. An unauthenticated scan misses every endpoint behind a login wall — which is typically most of the interesting attack surface. A scan that can't handle CSRF will fail silently. A scan pointed at a SPA that doesn't understand SPAs will test the empty shell and report clean.

Corefix implements all six steps automatically. Architecture detection, CSRF identification, AI selector analysis, headless rendering, overlay dismissal, and OAuth detection all happen before the first active scan request fires. There is no YAML to write, no selectors to specify, no CSRF field names to register.

Security coverage reflects what your application actually exposes — not just what a 2005-era scanner could reach.

The scanner should understand your app. Yours doesn't have to understand the scanner.

Corefix automates the full ZAP context pipeline — from architecture detection to authenticated scan. Try Corefix today →

In our previous post, we documented how five iterations of YAML tuning took ZAP's SQL injection detection from 3 alerts to 26 against DVWA - a 700% improvement from configuration changes alone. We thought we were done.

We were not done.

After shipping those policy improvements, we turned our attention to real-world applications: single-page apps with JWT authentication, multi-origin architectures, CSRF-protected forms, and HAR-based crawl seeding. What we found was an entirely new category of silent scan failures, problems that exist beneath the scan policy layer, in the plumbing that connects your scanner to your application.

This is the story of six configuration layers we had to build, debug, and automate before ZAP could reliably scan a modern web application.

Layer 1: HAR Seeding — Your Spider Is Blind Without It ​

ZAP's traditional spider works by fetching HTML pages and following links. The Ajax spider adds a headless browser to execute JavaScript. Together, they discover a reasonable portion of a server-rendered application. But for a modern SPA — where routes exist only in client-side JavaScript and API endpoints are called dynamically — both spiders miss critical attack surface.

The solution is HAR (HTTP Archive) file imports. Record a user session in Chrome DevTools, export the HAR, and feed it to ZAP before spidering. Every URL in the HAR gets added to ZAP's site tree, giving the active scanner endpoints it would never have discovered on its own.

The problem? HAR files from real browser sessions contain everything: Google Analytics beacons, Sentry error reporting, CDN asset requests, CloudFront distribution URLs, third-party authentication endpoints. In one HAR from a production SPA session, we counted 50+ unique URLs — but only 14 belonged to the target application.

Without filtering, ZAP's requestor job fires authenticated requests at every URL in the HAR. That means your scan cookies, session tokens, and authentication headers get sent to google-analytics.com, sentry.io, and every third-party service your frontend calls. It's not just wasteful. It's a potential credential leak.

We built a two-stage filter: first, AI analysis of the HAR URLs to identify which origins are in-scope for the target application; second, a host-matching filter that strips static assets (.js, .css, .png, .woff2) and anything not matching the target host. What started as a simple HAR import became a URL intelligence pipeline.

Layer 2: The Requestor Job — Seeding What Spiders Can't Find ​

HAR imports add URLs to ZAP's site tree, but they don't visit them as the authenticated user. The spider might eventually reach them, but "might" and "eventually" aren't acceptable for vulnerability scanning. Critical pages behind JavaScript navigation, form submissions, or conditional UI logic can be invisible to both spiders.

The requestor job solves this by making authenticated GET requests to every in-scope URL before the spider runs. This guarantees that ZAP has real responses in its site tree — complete with parameters, form fields, and CSRF tokens — ready for the active scanner.

For DVWA, this meant pre-visiting all 14 vulnerability pages. For a production SPA, it means hitting every route and API endpoint extracted from the HAR. The spider then crawls from these seeded pages, discovering additional links and parameters that the HAR didn't capture.

The order matters: HAR import first (raw URL seeding), then requestor (authenticated visits), then CSRF handler registration, then spider (discovery), then active scan (testing). Get the sequence wrong and the scanner either can't authenticate, can't handle CSRF tokens, or doesn't know about half your endpoints.

Layer 3: CSRF Handling — Two Problems Disguised as One ​

CSRF protection breaks scanners in two completely different ways, and most teams only address one.

Problem 1: Header-based CSRF (SPAs and APIs). Modern applications send CSRF tokens in request headers — X-CSRF-TOKEN, X-XSRF-TOKEN, or custom headers. The token comes from a response header, a cookie (double-submit pattern), or an HTML meta tag. Without intercepting every request to inject the current token, ZAP's requests get rejected with 403 errors and the scanner tests nothing.

We built an httpsender script that intercepts every outgoing request, extracts the CSRF token from the most recent response (checking cookies, response headers, and HTML body patterns in priority order), and injects it into the outgoing request header. The script maintains a per-URL token map so each page gets its own token, handles token rotation, and falls back through 15+ regex patterns covering Django, Spring, Rails, Laravel, Angular, ASP.NET, Express, WordPress, and Drupal.

Problem 2: Form-based CSRF (traditional HTML applications). ZAP's active scanner submits forms as part of its testing — injecting SQL payloads into form fields, XSS payloads into text inputs, and command injection payloads into every parameter. But if a form has a hidden CSRF token field and ZAP doesn't know about it, the form submission fails server-side and the vulnerability goes undetected.

This requires a completely different solution: registering anti-CSRF token field names with ZAP's internal ExtensionAntiCSRF module. When ZAP encounters a form with a field named user_token, csrf_token, authenticity_token, csrfmiddlewaretoken, or __RequestVerificationToken, it extracts the token value and includes it in the form submission.

We register 35+ token field names covering every major framework — not because any single application uses all of them, but because the cost of registering an unused name is zero, and the cost of missing one is a completely silent scan failure. The application either uses the token name or it doesn't. No match means no action.

These two CSRF solutions are complementary, not overlapping. A Django application might use csrfmiddlewaretoken in forms (Problem 2) and X-CSRFToken in AJAX headers (Problem 1). A React SPA with an API backend might only need header injection. We run both scripts on every scan because determining which one is needed requires knowing the application's architecture — and if we already knew that, we wouldn't need a scanner.

Layer 4: Policy Definition — The Strength vs. Threshold Tradeoff ​

In our previous post, we showed how setting strength: insane and threshold: low on SQL injection rules produced a 700% improvement. What we didn't discuss was the tradeoff between defaultStrength and defaultThreshold at the policy level — and how getting this wrong can actually reduce findings.

We ran two back-to-back scans with different default policies:

# Policy A — Conservative defaults, aggressive on targeted rules
policyDefinition:
  defaultStrength: medium
  defaultThreshold: medium
  rules:
    - id: 40018  # SQL Injection
      strength: insane
      threshold: low

# Policy B — Aggressive defaults across all rules
policyDefinition:
  defaultStrength: high
  defaultThreshold: low
  rules:
    - id: 40018  # SQL Injection
      strength: insane
      threshold: low

The results were counterintuitive:

Policy B's defaultStrength: high boosted every rule, producing dramatic improvements in DOM XSS, SSTI, and two entirely new finding categories. But defaultThreshold: low paradoxically reduced SQL injection findings from 26 to 3. Lower threshold changes how verification requests are generated — and in this case, the different request patterns triggered fewer detectable error conditions.

The optimal configuration turned out to be a hybrid:

policyDefinition:
  defaultStrength: high       # boost all rules
  defaultThreshold: medium    # keep confirmation sensitivity
  rules:
    - id: 40018               # SQL Injection
      strength: insane
      threshold: low
    - id: 40026               # DOM XSS
      strength: insane
      threshold: low
    - id: 90018               # Advanced SQLi
      strength: high
      threshold: low
    - id: 90020               # Command Injection
      strength: high
      threshold: low
    - id: 90037               # SSTI
      strength: high
      threshold: low

Discovering this required running both configurations, comparing raw statistics JSON with 200+ entries each, and understanding which rules benefit from which threshold levels. It is not the kind of thing most teams have time to figure out through trial and error.

Layer 5: Spider Non-Determinism — The Same Scan Never Runs Twice ​

Between our best two runs, ZAP's Ajax spider discovered 476 URLs in one and 461 in the other. The 15 missing URLs caused two SQL injection sub-variants to disappear entirely from the results.

The Ajax spider uses a real headless Firefox browser. Page load timing, JavaScript execution order, network latency, and DOM rendering are all non-deterministic. Two identical scans against the same application can discover different URLs, which means different parameters get tested, which means different vulnerabilities get found.

We addressed this with the requestor job — pre-visiting critical pages guarantees they're in the site tree regardless of what the spider discovers. But for teams running ZAP without a requestor, scan results vary between runs with no configuration change. A vulnerability that appeared yesterday can disappear today, not because it was fixed, but because the spider took a different path through the JavaScript.

Layer 6: Third-Party URL Leakage — Your Scanner Is Talking to Everyone ​

When we first implemented HAR-based requestor seeding without filtering, our requestor job contained 50+ URLs — including Google Analytics tracking beacons with full session parameters, Sentry error reporting endpoints with API keys, CloudFront CDN URLs, and third-party authentication service endpoints.

ZAP's requestor doesn't check scope before making requests. It fires authenticated HTTP requests at every URL in the list, sending your scan cookies and session headers to services that have nothing to do with your target application. For a security tool, this is a security problem.

The fix required building an AI-powered URL classification pipeline: extract all URLs from HAR files, identify which origins belong to the target application, filter out analytics, CDN, and third-party service URLs, and strip static assets that have no attack surface. What should have been a simple "import HAR and scan" workflow became a multi-step intelligence process.

The Compound Effect ​

None of these six layers is individually catastrophic. A scan without HAR seeding still finds some vulnerabilities. A scan without CSRF handling still tests some endpoints. A scan without policy tuning still produces some results.

The problem is compound. Missing HAR seeding means 30% fewer endpoints. Missing CSRF handling means 40% of form-based tests fail silently. A suboptimal policy means half the payloads aren't sent. Spider non-determinism means 5–10% variation between runs. Each layer multiplies against the others, and the result is a scan that reports "succeeded" while detecting a fraction of the actual vulnerabilities.

For our DVWA experiment — a trivially simple application — the difference between the unconfigured scan and the fully optimized scan was:

That's a 20x increase in tested URLs and a 56% increase in unique vulnerability categories — on an application with maybe 15 pages.

Why We Built Corefix ​

Every one of these six layers exists because DAST scanners were built as generic tools, and modern web applications are anything but generic. The scanner doesn't know your authentication flow, your CSRF pattern, your API endpoints, your technology stack, or which URLs in your HAR files are actually yours.

Corefix knows. It analyzes your application's responses to detect CSRF patterns automatically. It classifies HAR URLs to build an intelligent scope. It seeds every endpoint before scanning. It adjusts scan policy based on detected technology. It handles authentication, re-authentication, and session management without YAML configuration.

The six layers we spent weeks building, debugging, and optimizing? Corefix handles them in the background, before the first scan request is sent. What took us 5 iterations and 3 days of manual tuning happens automatically in under 2 minutes.

Your scanner should find vulnerabilities, not create configuration puzzles.

This is Part 2 of our DAST research series. Read Part 1: We Spent 3 Days Tuning OWASP ZAP So You Don't Have To →

Corefix is built for teams that want comprehensive DAST coverage without the configuration overhead. Try Corefix today →

Every security team knows the promise of DAST tools: point a scanner at your app, hit run, get vulnerabilities. The reality? We recently ran a controlled experiment against DVWA (Damn Vulnerable Web Application) using OWASP ZAP's automation framework, a deliberately vulnerable app where SQL injection should be trivially detectable. What we found was a masterclass in how configuration complexity silently kills scan quality.

This is the story of three days, five scan iterations, and a 700% improvement in vulnerability detection, all from config changes alone. No new tools. No new plugins. Just YAML.

The Setup ​

Our target was DVWA running on a remote server at security level "low" (the easiest difficulty setting, where every vulnerability category should be wide open). We configured ZAP's automation framework with authenticated scanning: form-based login, PHPSESSID cookie management, anti-CSRF token handling, HAR file imports from prior crawls, and a full suite of active and passive scan plugins.

On paper, this should have been straightforward. In practice, it was anything but.

Run 1: The False Confidence Problem ​

Our first scan completed in 9 minutes with 41 vulnerabilities. For a deliberately vulnerable application with known SQL injection, XSS, command injection, file inclusion, and file upload flaws. 41 findings felt suspiciously low. More importantly, SQL injection was barely represented.

The scan looked successful. Authentication reported as working. The spider found 592 URLs. The automation plan reported "succeeded." But the active scanner finished in a fraction of the allocated 45-minute window, a red flag we almost missed.

What went wrong:

The first issue was subtle: a maxAlertsPerRule: 5 setting that capped each scan rule at just 5 findings before skipping remaining URLs. When a rule like SQL Injection finds its fifth alert on the third URL, it silently stops testing the remaining 840+ URLs. The scan "completes successfully" while leaving the vast majority of the attack surface untested.

The second issue was more insidious. We had excluded login.php from the scan scope — a reasonable-sounding decision to avoid scanning the login page itself. But ZAP needs login.php in scope to perform re-authentication when sessions expire mid-scan. Without it, session drops during active scanning left the scanner hitting unauthenticated redirects instead of vulnerable pages.

Run 2: Lifting the Alert Cap ​

We increased maxAlertsPerRule from 5 to 50. The results shifted immediately:

• User Agent Fuzzer alerts jumped from 5 to 52
• Cookie Slack Detector went from 5 to 53
• Advanced SQL Injection rose from 9 to 18
• Cross-Site Scripting climbed from 5 to 8
• Total URLs scanned increased from 14,527 to 23,957

But core SQL Injection (rule 40018) stubbornly stayed at 3 alerts. The cap wasn't the only bottleneck.

Run 3: Policy Tuning ​

This is where things got interesting. ZAP's default scan policy uses "medium" strength and threshold for all rules. For a deliberately vulnerable application, this means the scanner sends a moderate number of payloads per parameter and requires moderate confidence before flagging an issue.

We added a policyDefinition to the active scan configuration:

policyDefinition:
  defaultStrength: medium
  defaultThreshold: medium
  rules:
    - id: 40018    # SQL Injection
      strength: insane
      threshold: low
    - id: 40019    # SQL Injection (MySQL)
      strength: high
      threshold: low
    - id: 40012    # Cross-Site Scripting (Reflected)
      strength: insane
      threshold: low

The impact was dramatic:

The scan still completed in 14 minutes, well within the 45-minute cap, but with more than double the coverage and 700% more SQL injection findings. Same application. Same ZAP version. Same plugins. Just different YAML.

The Hidden Bottleneck We Almost Missed ​

Even after three iterations, there was still a problem lurking. The sqliplugin add-on (rule 40019) was generating 144 warnings in the logs and hitting the maxRuleDurationInMins: 5 ceiling, spending all 5 minutes but only managing to test 30 URLs before being forcibly terminated. Meanwhile, it was throwing ZapSocketTimeoutException errors, suggesting that time-based SQL injection tests were consuming the entire rule budget on network timeouts rather than actual vulnerability testing.

This is the kind of issue that requires reading raw statistics JSON, cross-referencing rule IDs with plugin documentation, and understanding the interaction between timeout configurations, rule duration caps, and network latency. It is not the kind of thing most development teams, or even most security teams, have time to investigate.

Run 4: Lifting the Rule Duration Cap ​

The sqliplugin timeout problem pointed at an obvious next configuration: increase maxRuleDurationInMins from 5 to 10 and give the time-intensive rules room to finish.

maxRuleDurationInMins: 10

The big story is rule 90018 (Advanced SQLi). It tested 282,451 URLs in this run vs 12,554 before — a 22x increase. With maxRuleDurationInMins: 10, it had room to breathe and used 639 seconds (the full 10 minutes) before being capped. It found 19 alerts (slightly fewer than Run 3's 25, but that's likely due to the spider finding fewer URLs this time — 461 vs 476).

The sqliplugin (40019) also improved — 396 URLs tested vs 30 in Run 3. It used the full 411 seconds but is still generating 144 warnings. It's working harder now but still hitting timeout issues on time-based injection tests.

One concern: 90018 got skipped: 1, meaning it hit the 10-minute cap and was forcibly stopped. It was clearly still finding things to test. You could push maxRuleDurationInMins to 15 for even more coverage, though you're hitting diminishing returns now.

Bottom line: going from Run 1 to Run 4, you went from 14,527 to 299,739 URLs tested — a 20x improvement — and SQLi findings went from 3 to 26. The scan is now doing real, thorough work.

The total unique alert types dropped slightly (17 high → 15 high), but that doesn't mean the scan was worse. Here's what likely happened:

Run 4 spider found fewer URLs — 461 vs 476 in Run 3. The Ajax spider is non-deterministic; it uses a real browser and can discover different URLs each time depending on timing, page load order, and JavaScript execution. Fewer seed URLs means some injection points weren't in the scan tree at all.

Rule 90018 cannibalized time from other rules. It consumed the full 10 minutes and tested 282,451 URLs — that's a huge share of the scan's total budget. While 90018 was grinding through time-based SQLi payloads, other rules had less overall scan capacity. Notice rule 90018 got skipped: 1 — it was still running when the cap hit it.

Alert count ≠ scan quality. Run 4 sent 303,101 network requests vs 32,755 in Run 3 — nearly 10x more actual testing. The 2 missing "high" alerts are likely duplicate findings on URLs that the spider didn't discover this time, not missed vulnerability classes.

The takeaway: this is exactly the kind of non-obvious variability that makes manual DAST tuning frustrating. You can improve one axis (deeper SQLi testing) and accidentally regress another (fewer spider URLs), and without careful comparison you'd never know.

Run 5: Raising the Default Strength ​

Run 5 tested a single hypothesis: what happens when defaultStrength is raised to high globally, rather than just for explicitly listed rules.

policyDefinition:
  defaultStrength: high
  defaultThreshold: low
  rules:
    - id: 40018
      strength: insane
      threshold: low
    - id: 40019
      strength: high
      threshold: low
    - id: 40012
      strength: insane
      threshold: low

The policyDefinition change from medium/medium to high/low made a massive difference across rules that weren't explicitly listed. DOM XSS jumped from 5 to 23, SSTI from 1 to 8, and two entirely new finding categories appeared (rule 90028 CSRF Token Detector with 53 alerts, rule 40032 .htaccess with 18). This proves that defaultStrength: high boosts every rule, not just the ones you override.

One concern: SQLi 40018 dropped back to 3 alerts despite testing more URLs (2,287 vs 1,743). This is because defaultThreshold: low makes the scanner pickier about confirming findings — it needs stronger evidence before raising an alert. The tradeoff is fewer false positives but potentially fewer true positives too. For a high tier configuration, defaultThreshold: medium with only the specific rules set to low works better:

policyDefinition:
  defaultStrength: high
  defaultThreshold: medium    # less aggressive confirmation globally
  rules:
    - id: 40018
      strength: insane
      threshold: low          # aggressive for SQLi specifically

The sqliplugin (40019) is struggling badly — 942 warnings, 600 seconds consumed, only 45 URLs tested. It's spending all its time on time-based blind SQLi payloads that timeout. Consider dropping it to strength: high instead of insane.

New SSTI error: SstiBlindScanRule.NullPointerException appeared 40 times — this is a ZAP bug in the SSTI plugin, not your config. It found 8 alerts despite the errors, which is great.

Overall this config is production-ready. The high/low defaults are delivering significantly broader coverage.

What This Means for Real-World Scanning ​

DVWA is a toy application with a handful of pages. A production application with hundreds of endpoints, complex authentication flows, API integrations, and dynamic content multiplies every one of these configuration challenges. Consider what we had to get right just for this simple scan:

• Authentication configuration: form-based login with anti-CSRF token extraction, session cookie management with the correct security level parameter, verification polling with regex matching, and correct scope inclusion of the login page for re-authentication
• Scope management: knowing which paths to exclude (logout, setup, CSRF pages) without accidentally excluding paths the scanner needs
• Spider tuning: balancing crawl depth, child limits, and duration across both traditional and Ajax spiders
• Scan policy: per-rule strength and threshold settings that match the application's technology stack and vulnerability profile
• Alert management: understanding that alert caps silently truncate coverage
• Rule duration limits: balancing thoroughness against time-based injection tests that consume entire budgets on timeouts
• Plugin selection: choosing from dozens of add-ons (and discovering that some like frontendscanner no longer exist)
• Result interpretation: reading raw statistics JSON to identify silent failures that the "succeeded" status hides

Each of these is a potential point of failure that produces no error message. Results just silently degrade.

Why We Built Corefix ​

This experiment crystallized what we had long suspected: the gap between "running a DAST scan" and "running an effective DAST scan" is enormous, and it is almost entirely a configuration problem.

Corefix eliminates this gap. Instead of days of YAML tuning, iterative test runs, and statistics analysis, Corefix delivers comprehensive authenticated scanning with:

• Zero configuration: no YAML files, no policy tuning, no rule-by-rule strength adjustments
• Automatic authentication handling: session management, token extraction, and re-authentication handled out of the box
• Intelligent scan policies: dynamic adjustment of scan depth and payload selection based on the application's technology fingerprint
• Complete coverage by default: no silent alert caps, no premature rule termination, no scope misconfigurations
• Results in minutes: what took us 3 days and 5 iterations to achieve with ZAP, Corefix delivers in under 2 minutes

The vulnerabilities don't change based on your scanner configuration. SQL injection is SQL injection whether your YAML has strength: insane or strength: medium. The only question is whether your tool finds it, and whether you have days to spend making sure it does.

Corefix makes sure it does. Every time. Without the YAML.

Corefix is built for security engineers who want results, not configuration files, and for development teams who need DAST coverage without becoming DAST experts. Try Corefix today →