CLI Reference
Complete flag reference for the webfang binary. The block in the
"Complete flag reference" section below is auto-generated from the
binary itself by scripts/gen-cli-reference.sh (built with the exact release
feature set, ai mcp), so flags, defaults, and WEBFANG_* environment
variables always match reality. CI drift-checks the chapter against the binary
on every doc run; regenerate locally with just cli-doc-regen.
Every flag in this reference is compiled into the binary you are running.
The release artifacts are built with no capability features, so
feature-gated behavior (--js-strategy full, SQLite persistence, adaptive
selectors, tokio-console) is absent or rejected there — the per-flag
consequences are in the
Release Feature Matrix.
Quick start
Scrape a single page by passing the URL positionally:
webfang https://example.com
Crawl a whole site following its sitemap.xml:
webfang https://example.com --use-sitemap --max-pages 50
Extract only the elements matching a CSS selector and export for a RAG pipeline:
webfang https://example.com -s "article.main-content" --export-format jsonl
Resume an interrupted crawl from its last checkpoint:
webfang --batch-file urls.txt --resume
Zero values
Numeric flags follow the Zero Silent Loss policy: a zero is never silently turned into another value.
- Zero disables the feature where an "off" state exists:
--delay-ms 0disables request pacing (no token bucket is allocated),--max-depth 0scrapes only the seed URL. - Zero is rejected with a usage error where zero is meaningless or
destructive:
--concurrency,--download-concurrency,--download-timeout,--max-pages,--rate-limit-burst,--timeout-secs. - Legacy exception: a non-numeric
--rate-limit-burstvalue warns and falls back to the hardware-derived default (numeric zero and out-of-range values are still rejected).
Configuration file
Flags can also come from a TOML file, which is how you set values once
instead of on every invocation. webfang looks for webfang/config.toml
under the platform's user config directory:
| Platform | Resolved from |
|---|---|
| Linux | $XDG_CONFIG_HOME, else ~/.config |
| macOS | ~/Library/Application Support |
| Windows | %APPDATA% |
An absolute XDG_CONFIG_HOME is honored on every platform, not just
Linux — that is what makes hermetic test fixtures possible.
To point at a specific file instead, set WEBFANG_CONFIG to the path of
the config.toml itself. It takes precedence over the table above, and it
differs from XDG_CONFIG_HOME in two ways worth knowing:
- It names the file, while
XDG_CONFIG_HOMEnames a directory that webfang appendswebfang/to. Use it when your file does not live in that subdirectory. - It is honored on Windows, where no environment variable reaches the
platform config lookup: that lookup calls
SHGetKnownFolderPath(FOLDERID_RoamingAppData), a Windows Shell API that reads no env var at all. UntilWEBFANG_CONFIGexisted, a Windows install could not be redirected to a config file by env at all.
A missing file at the platform path is not an error: webfang falls back to its defaults, because a fresh install has no config yet.
That tolerance stops at an explicit WEBFANG_CONFIG. Once you name a file,
webfang holds you to it — the run stops with exit code 78 (configuration
error) and a message naming the path, instead of quietly running with
defaults. A typo in a config path used to be indistinguishable from a
successful run.
WEBFANG_CONFIG | Result |
|---|---|
| unset, or empty | Platform lookup; a missing file means defaults |
| absolute path to a readable, valid TOML file | Used |
| relative path | Rejected, exit 78 |
| path does not exist | Rejected, exit 78 |
| unreadable — wrong permissions, or a directory | Rejected, exit 78 |
| not valid TOML | Rejected, exit 78 |
Two edge cases matter because this variable is usually set from a script or a service unit rather than typed:
- An empty value counts as unset.
WEBFANG_CONFIG= webfang ...behaves exactly like not setting the variable at all, so a script that exports it before it has a value falls back to the platform lookup instead of failing. - A relative path is rejected rather than resolved. It used to be resolved against the current working directory of the process, not against your home directory — predictable in an interactive shell, and arbitrary under a daemon or service manager, whose working directory is whatever the supervisor chose. The same variable could therefore name a different file depending on who started the process, which is why an absolute path is now required.
WEBFANG_CONFIG=/etc/webfang/config.toml webfang https://example.com
Complete flag reference
Everything below is the verbatim output of webfang --help and
webfang completions --help — including every default value and the
matching WEBFANG_* environment variable for each flag.
$ webfang --help
CLI Arguments for the webfang binary.
Parsed using `clap` with derive macros.
# Examples
```no_run use webfang_core::Args; use clap::Parser;
let args = Args::parse_from([ "webfang", "--url", "https://example.com", "--output", "./output", "--export-format", "jsonl", "--resume", ]);
assert_eq!( args.crawler.url.as_ref().map(webfang_core::domain::ValidUrl::as_str), Some("https://example.com") ); ```
Usage: webfang [OPTIONS] [URL]
webfang [OPTIONS] [URL] <COMMAND>
Commands:
completions Generate shell completion scripts
connect Detect and validate a remote embedding endpoint, then write its config
doctor Diagnose the embedding path — config, credential, network, budget
show Dump the effective embeddings provider, the configured budget, and the historical model-profile metadata
recover Recover pages from a stranded batch capture spool into markdown files
help Print this message or the help of the given subcommand(s)
Arguments:
[URL]
URL to scrape (positional shorthand — equivalent to --url). Hardened through the SAME argv-boundary parser as `--url` (#1239): the shorthand cannot bypass the credential strip
Options:
-h, --help
Print help (see a summary with '-h')
-V, --version
Print version
Target:
-u, --url <URL>
URL to scrape (required unless using a subcommand)
[env: WEBFANG_URL=]
-s, --selector <SELECTOR>
Matched elements become the content directly (no main-content cleanup); title and date come from the whole document, excerpt stays empty
[env: WEBFANG_SELECTOR=]
[default: body]
Discovery:
--delay-ms <DELAY_MS>
Delay between requests in milliseconds
[env: WEBFANG_DELAY_MS=]
[default: 1000]
--jitter-ms <JITTER_MS>
Random addition to each delay between requests, in milliseconds (0 = fixed delay; the mean interval becomes delay + jitter/2, so the request rate drops; with 0, a run that fits in the burst is not paced at all)
[env: WEBFANG_JITTER_MS=]
[default: 0]
--max-pages <MAX_PAGES>
Maximum pages to scrape
[env: WEBFANG_MAX_PAGES=]
[default: 10]
--concurrency <CONCURRENCY>
Concurrency level (auto or number, minimum 1)
[env: WEBFANG_CONCURRENCY=]
[default: auto]
--rate-limit-burst <RATE_LIMIT_BURST>
Explicit rate-limiter burst permits (token-bucket capacity).
Overrides the hardware-derived budget-model default (Q1: burst is decoupled from crawl concurrency). Accepts a positive integer, or the keyword `auto` for the derived default (the same spelling `--concurrency` understands). Raw string here ON PURPOSE: validation/conversion happens once in preflight staging via `parse_rate_limit_burst` so CLI, env, and programmatic input all share one semantic. That semantic FAILS CLOSED: `0`, an out-of-range value, and any other typo are rejected with exit 78 rather than degrading to the default — only `auto` and an empty value mean "not set".
[env: WEBFANG_RATE_LIMIT_BURST=]
--use-sitemap
Use sitemap for URL discovery NOTE: HTTP redirects (301/302) are resolved at scrape-time, not parse-time. This avoids redundant HEAD requests during sitemap parsing for better performance
[env: WEBFANG_USE_SITEMAP=]
--sitemap-url <SITEMAP_URL>
Explicit sitemap URL
[env: WEBFANG_SITEMAP_URL=]
Behavior:
--single-page
Scrape only the seed URL without discovery or crawling (batch mode always scrapes one page per URL)
[env: WEBFANG_SINGLE_PAGE=]
--resume
Resume mode - skip URLs already processed
[env: WEBFANG_RESUME=]
--state-dir <STATE_DIR>
Custom state directory for resume mode
[env: WEBFANG_STATE_DIR=]
--download-images
Download images from the page
[env: WEBFANG_DOWNLOAD_IMAGES=]
--download-documents
Download documents from the page
[env: WEBFANG_DOWNLOAD_DOCUMENTS=]
--download-assets
Download all assets (images + documents) from the page
[env: WEBFANG_DOWNLOAD_ASSETS=]
--extraction-fingerprint
Record extraction failure fingerprints in SQLite and attach them to low-quality extraction hints (#792). Repeated low-score extractions on the same site/selector pair accumulate a failure count surfaced in the hint, instead of degrading silently
[env: WEBFANG_EXTRACTION_FINGERPRINT=]
--clean-ai
Use AI-powered semantic cleaning for better RAG output
[env: WEBFANG_CLEAN_AI=]
[alias: --ai]
Display:
-v, --verbose...
Verbosity level: -v (INFO), -vv (DEBUG), -vvv (TRACE)
[env: WEBFANG_VERBOSE=]
-q, --quiet
Quiet mode — suppress info/debug output
[env: WEBFANG_QUIET=]
-n, --dry-run
Dry-run mode — discover URLs and print without scraping
[env: WEBFANG_DRY_RUN=]
--trace-file <TRACE_FILE>
Path to write OTel spans as JSONL for offline debugging
[env: WEBFANG_TRACE_FILE=]
Crawler Settings:
--max-depth <MAX_DEPTH>
Maximum depth to crawl (0 = only seed URL)
[env: WEBFANG_MAX_DEPTH=]
[default: 2]
--timeout-secs <TIMEOUT_SECS>
Request timeout in seconds
[env: WEBFANG_TIMEOUT_SECS=]
[default: 30]
--include-pattern <INCLUDE_PATTERNS>
URL patterns to include (glob-style). Three modes:
* Path: starts with `/` → matched against URL path, e.g. `/pricing`, `/admin/*` * Path glob: starts with `*/` → matched against URL path, e.g. `*/api/*` * Host (default): matched against hostname, e.g. `example.com`, `*.example.com`
Example: to exclude a path, use `--exclude-pattern "/admin/*"`, not `*admin*`
[env: WEBFANG_INCLUDE=]
--exclude-pattern <EXCLUDE_PATTERNS>
URL patterns to exclude (glob-style, same three modes as --include-pattern). Deny takes precedence over allow
[env: WEBFANG_EXCLUDE=]
Download Settings:
--asset-naming <ASSET_NAMING>
Estrategia de nombre de archivo para assets descargados: hash (default), slug, content-disposition
[default: hash]
[possible values: hash, slug, content-disposition]
--download-concurrency <DOWNLOAD_CONCURRENCY>
Máximo de descargas de assets concurrentes por página (mínimo 1)
[env: WEBFANG_DOWNLOAD_CONCURRENCY=]
--max-file-size <MAX_FILE_SIZE>
Maximum file size to download in bytes (default: 50MB)
[env: WEBFANG_MAX_FILE_SIZE=]
[default: 52428800]
--download-timeout <DOWNLOAD_TIMEOUT>
Timeout for individual asset downloads in seconds (minimum 1)
[env: WEBFANG_DOWNLOAD_TIMEOUT=]
[default: 30]
HTTP Client Settings:
--max-retries <MAX_RETRIES>
Maximum number of retry attempts
[env: WEBFANG_MAX_RETRIES=]
[default: 3]
--backoff-base-ms <BACKOFF_BASE_MS>
Base delay for exponential backoff (ms)
[env: WEBFANG_BACKOFF_BASE_MS=]
[default: 1000]
--backoff-max-ms <BACKOFF_MAX_MS>
Maximum delay for exponential backoff (ms)
[env: WEBFANG_BACKOFF_MAX_MS=]
[default: 10000]
--accept-language <ACCEPT_LANGUAGE>
Accept-Language header value
[env: WEBFANG_ACCEPT_LANGUAGE=]
[default: en-US,en;q=0.9]
--user-agent <USER_AGENT>
Custom User-Agent header value (overrides Chrome 145 default)
[env: WEBFANG_USER_AGENT=]
-H, --header <NAME: VALUE>
Inject a custom HTTP header as `Name: Value` (repeatable).
Overrides any default header with the same (case-insensitive) name. Example: `-H "Authorization: Bearer TOKEN"`.
[env: WEBFANG_HEADER=]
--cookie <NAME=VALUE>
Inject a custom cookie as `name=value` (repeatable).
Seeded into the cookie jar before the first request so authenticated crawls work without a prior login round-trip. Example: `--cookie "session=abc123"`.
[env: WEBFANG_COOKIE=]
Sitemap Settings:
--sitemap-depth <SITEMAP_DEPTH>
Maximum recursion depth for sitemap indexes
[env: WEBFANG_SITEMAP_DEPTH=]
[default: 3]
Competitive Features:
--checkpoint-interval <CHECKPOINT_INTERVAL>
Pages between automatic checkpoint saves (0 = disabled) — unified via PersistenceMode with --resume (Checkpoint/Full when enabled)
[env: WEBFANG_CHECKPOINT_INTERVAL=]
[default: 100]
--no-checkpoint
Disable checkpoint persistence entirely — PersistenceMode disables checkpoint (Resume only when combined with --resume)
[env: WEBFANG_NO_CHECKPOINT=]
--ignore-robots
Skip robots.txt enforcement
[env: WEBFANG_IGNORE_ROBOTS=]
--ignore-waf
Skip WAF/CAPTCHA classification (REQ-WAF-07): challenged responses are reported as plain HTTP errors instead of WAF blocks. It does not rescue the fetch — a challenge page is never scraped as content (F-11).
[env: WEBFANG_IGNORE_WAF=]
--autoscale
Enable autoscaled concurrency — dynamically adjusts task concurrency based on RAM usage
[env: WEBFANG_AUTOSCALE=]
--no-session-health
Disable session pool health checks
[env: WEBFANG_NO_SESSION_HEALTH=]
--h2-profile <H2_PROFILE>
TLS/HTTP2 profile name (default: Chrome145)
[env: WEBFANG_H2_PROFILE=]
[default: Chrome145]
JS Rendering:
--js-strategy <JS_STRATEGY>
JavaScript rendering strategy: static (wreq only), hybrid (3-layer), full (Chromiumoxide only)
Possible values:
- static: Static HTTP only (wreq). Fastest, no JS rendering
- hybrid: Hybrid 3-layer: wreq → Obscura → Chromiumoxide
- full: Full JS rendering only (Chromiumoxide). Slowest, handles all SPAs
[env: WEBFANG_JS_STRATEGY=]
[default: static]
--js-wait <JS_WAIT>
Post-load settlement wait for the chromium render path: idle (network-idle, default), <ms> fixed wait (1..=30000), none (capture immediately)
[env: WEBFANG_JS_WAIT=]
[default: idle]
--obscura-binary <OBSCURA_BINARY>
Path to the obscura binary (default: "obscura")
[env: WEBFANG_OBSCURA_BINARY=]
[default: obscura]
Cleanup:
--dom-preprune [<DOM_PREPRUNE>]
Enable DOM pre-pruning before Readability (removes invisible/empty wrappers). Default: enabled (true). Set to false via --dom-preprune=false or WEBFANG_DOM_PREPRUNE=false
[env: WEBFANG_DOM_PREPRUNE=]
[default: true]
[possible values: true, false]
Output:
-o, --output <OUTPUT>
Output directory for scraped content
[env: WEBFANG_OUTPUT=]
[default: output]
-f, --content-format <FORMAT>
Output format for individual files (markdown, text, json) NOTE: For RAG pipeline export, use --pipeline-format instead
Possible values:
- markdown: Markdown format with YAML frontmatter (recommended for RAG)
- json: Structured JSON with metadata
- text: Plain text without formatting
[env: WEBFANG_FORMAT=]
[default: markdown]
--pipeline-format <EXPORT_FORMAT>
Export format for RAG pipeline (jsonl, vector, auto) NOTE: Use --content-format for output file format (markdown, text, json)
Possible values:
- jsonl: JSONL format (JSON Lines - one JSON object per line) Optimal for RAG pipelines and vector database ingestion
- vector: Vector format (JSON with metadata header) Supports embeddings and cosine similarity
- auto: Auto-detect format from existing export files
[env: WEBFANG_EXPORT_FORMAT=]
[default: jsonl]
--retention-days <RETENTION_DAYS>
Prune exports and vault-DB rows older than N days at the end of a successful run (0 = disabled)
[env: WEBFANG_RETENTION_DAYS=]
[default: 0]
Elastic Ingestion:
--cpu-cores <CPU_CORES>
CPU core override for the elastic ingestion Rayon pool (else auto-detect)
[env: WEBFANG_CPU_CORES=]
--ram-budget <RAM_BUDGET>
RAM budget override for the byte-weighted semaphore (`8GB`, `2048MB`, or bytes)
[env: WEBFANG_RAM_BUDGET=]
--db-path <DB_PATH>
SQLite database path override for persisted resources/chunks
[env: WEBFANG_DB_PATH=]
--elastic
Enable elastic ingestion pipeline (streaming, SQLite dedup, Rayon CPU bridge)
[env: WEBFANG_ELASTIC=]
--output-vectors <OUTPUT_VECTORS>
Write extracted vectors to a JSONL file for RAG pipelines. Use `-` for stdout. No SQLite dependency — available in every build (core binary too)
[env: WEBFANG_OUTPUT_VECTORS=]
Batch Processing:
--batch
Enable batch mode — scrape each URL from stdin (one per line), one page per URL, no crawling
[env: WEBFANG_BATCH=]
--batch-file <BATCH_FILE>
Path to a file containing URLs to scrape (one per line), one page per URL, no crawling
[env: WEBFANG_BATCH_FILE=]
--batch-concurrency <BATCH_CONCURRENCY>
Maximum concurrent URLs in batch mode (omit = auto from budget model)
[env: WEBFANG_BATCH_CONCURRENCY=]
--export-checkpoint <EXPORT_CHECKPOINT>
Flush captured pages to markdown every N pages during a batch (0 = only at the clean close)
[env: WEBFANG_EXPORT_CHECKPOINT=]
[default: 0]
Item Pipeline:
--pipeline
Enable item pipeline processing (validate → clean → output)
[env: WEBFANG_PIPELINE=]
--pipeline-output <PIPELINE_OUTPUT>
Pipeline output format: jsonl (default), none
Possible values:
- jsonl: Write items as JSON Lines to a file (default)
- none: No pipeline output — items are processed but not written
[env: WEBFANG_PIPELINE_OUTPUT=]
[default: jsonl]
Obsidian:
--obsidian-wiki-links
Convert same-domain links to Obsidian [[wiki-link]] syntax
[env: WEBFANG_OBSIDIAN_WIKI_LINKS=]
--obsidian-tags <OBSIDIAN_TAGS>
Tags to include in YAML frontmatter (comma-separated)
[env: WEBFANG_OBSIDIAN_TAGS=]
--obsidian-relative-assets
Rewrite downloaded asset paths as relative to the .md file
[env: WEBFANG_OBSIDIAN_RELATIVE_ASSETS=]
--vault <VAULT>
Path to Obsidian vault (auto-detects if not provided).
When provided explicitly, the vault becomes the output base: Markdown,
downloaded assets and the RAG export are written inside it — no need
to duplicate the path in `-o` (which then must stay at its default).
Auto-detected or config-file vaults do NOT redirect output (#762).
[env: WEBFANG_OBSIDIAN_VAULT=]
--quick-save
Quick-save mode: save directly to vault _inbox folder
[env: WEBFANG_OBSIDIAN_QUICK_SAVE=]
--obsidian-rich-metadata
Add rich metadata to frontmatter
[env: WEBFANG_OBSIDIAN_RICH_METADATA=]
AI Settings:
--threshold <THRESHOLD>
Relevance threshold for AI semantic filtering (0.0-1.0)
[env: WEBFANG_THRESHOLD=]
[default: 0.3]
--max-chars <MAX_CHARS>
Maximum characters per chunk before rejection, >= 1 (a chunk-size guard, not a context-window setting; the effective ceiling is the embedding backend's own limit, which a remote endpoint enforces server-side as HTTP 400/413)
[env: WEBFANG_MAX_CHARS=]
[default: 98304]
--offline
Reject every embeddings provider (remote or loopback) with exit 78: no local model since 3.0.0
[env: WEBFANG_OFFLINE=]
LLM Extraction:
--extract-with-llm
Run structured extraction through the configured LLM provider (requires a provider with the `completion` capability; fails at startup otherwise)
[env: WEBFANG_EXTRACT_WITH_LLM=]
--llm-provider <LLM_PROVIDER>
Provider id to use for LLM extraction (default: first provider declaring the `completion` capability)
[env: WEBFANG_LLM_PROVIDER=]
--embedding-provider <EMBEDDING_PROVIDER>
Provider id to use for embeddings (default: first provider declaring the `embedding` capability)
[env: WEBFANG_EMBEDDING_PROVIDER=]
Tier 2 Embeddings:
--tier2-embedding-provider <TIER2_EMBEDDING_PROVIDER>
Provider id to use for Tier 2 semantic inspector embeddings (default: none; the vault-search provider is never adopted — a remote id requires --tier2-consent)
[env: WEBFANG_TIER2_EMBEDDING_PROVIDER=]
--tier2-consent
Explicit consent for Tier 2 remote embeddings egress (required when --tier2-embedding-provider is remote; hash of provider|endpoint logged for audit)
[env: WEBFANG_TIER2_CONSENT=]
EXIT CODES:
0 Success
2 No URLs discovered
3 All scrapers failed
64 Bad CLI arguments (usage error)
65 Data format error (extraction without usable content)
69 Network error, partial success, or SSRF refusal
74 I/O error
76 Protocol error
77 All URLs blocked by robots.txt
78 Configuration error
EXAMPLES:
webfang https://example.com
webfang -u https://example.com
webfang -u https://example.com --ai
webfang -u https://example.com -f jsonl
webfang -u https://example.com -v
webfang -u https://example.com -vv # DEBUG
webfang --batch-file urls.txt --resume
$ webfang completions --help
Generate shell completion scripts
Usage: webfang completions <SHELL>
Arguments:
<SHELL>
Shell to generate completions for
Possible values:
- bash: Bash shell completions
- elvish: Elvish shell completions
- fish: Fish shell completions
- power-shell: PowerShell completions
- zsh: Zsh shell completions
Options:
-h, --help
Print help (see a summary with '-h')
Layer 2 (Obscura): a cheap render, not an anti-WAF guarantee
--js-strategy hybrid fetches in three layers: L1 is wreq (static
HTTP), L2 is the obscura subprocess, L3 is Chromiumoxide.
(--js-strategy full goes straight to L3 and is rejected at preflight by
the release binaries, which carry no chromium feature —
Release Feature Matrix.)
L2 is accepted as a cheap render — webfang does not require
obscura's -stealth build variant, so L2 carries no anti-WAF promise. That
is the decision recorded in #1710; this section is where it lives.
What L2 does
- Runs
obscura fetch --dump htmlonce per page — never markdown — with no cookies and no storage directory (the sessionless contract), and marks the result astext/html. - Costs ~30 MB per subprocess instead of the ~200 MB + CDP that Layer 3 pays: that is the whole reason it stays in the stack.
- Escalates to Layer 3 when the page comes back empty or the subprocess fails.
What L2 does not do
- It is not an anti-WAF guarantee. webfang never passes
-stealth, and a WAF challenge detected at any layer aborts the fetch: the router never escalates against a challenge.
Version floor. The hybrid preflight requires obscura 0.2.5 or newer
(MINIMUM_OBSCURA_VERSION in crates/webfang_core/src/cli/preflight.rs).
An older binary fails the preflight with exit 78 (configuration error),
naming the version it found and the version it requires. A binary whose
--version probe fails or prints something unparseable is not blocked:
the preflight degrades to a warning and the run continues best-effort.
Reading the L2 hit-rate. There are no counters — the hit-rate is derived from the per-layer log events the hybrid router already emits. With a trace file, count the layer events:
webfang https://example.com --js-strategy hybrid --trace-file run.jsonl
# L2 attempts
jq -r 'select(.message | test("Layer 2 \\(Obscura\\): attempting fetch")) | .message' run.jsonl | wc -l
# L2 hits (the render came back with content)
jq -r 'select(.message | test("^Layer 2 returned [0-9]+ bytes$")) | .message' run.jsonl | wc -l
# escalations to L3
jq -r 'select(.message | test("Layer 3 \\(Chromiumoxide\\): attempting fetch")) | .message' run.jsonl | wc -l
The trace file records every level independently of -v (the file layer runs
at TRACE level for the webfang target), so the layer events are in it even
without -vv. The rest of the query cookbook is in
debugging.md.
The four consequences of the decision, recorded so the promise is not re-invented later:
- The L2 hit-rate is derived from the per-layer logs — no new counters, and no strict mode.
- The preflight's
Unknownverdict (an unreadable--version) degrades to a warning instead of failing the run: an unreadable version never blocks the crawl. - The floor in force is 0.2.5. The
0.2.3ask in #1710 is stale: the upstream release moved on, and the floor follows the current stable. - #1710 stays open. It is commented with this status, not closed.
State directory contents
Companion note to --state-dir (documented in the generated block above).
Each --resume run keeps one JSON state file per domain plus a
<...>.json.lock sentinel file next to it. The sentinel is created on
demand and intentionally never removed: its presence does NOT mean a
writer is alive. The real mutual exclusion is the advisory flock held on
the sentinel, which the kernel releases when the process exits — including
after SIGKILL. There is exactly one sentinel per state file, so the leftover
is bounded. Do not delete the .lock file while a run holds it: removing
the path breaks the lock and lets two writers enter the critical section at
once, losing state updates.