JSON Output

The regex command prints JSON when it is given --format=json or --json. It ships in php-regex/regex-cli (composer require --dev php-regex/regex-cli, installed with the CLI guide). These documents are part of the backward compatibility promise: for all of 2.x, a key keeps its name, its type and its meaning. A minor release adds keys and values, and may improve messages; see What a minor release may change. This page lists every key of every document.

Symfony’s and Laravel’s regex:lint print the same lint report, and their regex:transpile the same transpile document, with the same error envelope — the commands come with their bundles (Symfony, Laravel). An argument error their console framework catches before the command runs, such as an unknown option or a missing argument, is reported by the framework in its own form, not as the envelope.

The Rules

One document on stdout

• Every run in JSON mode prints exactly one JSON document on stdout: pretty-printed, slashes and Unicode unescaped, ending with one newline.
• Status lines, progress and notes go to stderr. --quiet silences them; it never silences the document.
• The same holds for every machine format of lint: the JSON, GitHub, Checkstyle and JUnit reports are printed under --quiet, by regex lint and by Symfony’s and Laravel’s regex:lint. --quiet silences the human status lines only.
• A failure prints the error envelope instead, with the exit code the command uses anyway (see Exit Codes).
• PHP’s own diagnostics (warnings, notices) go to stderr in JSON mode, never into the document. A fatal error, such as redos running out of --time-limit, prints the envelope with stage internal and exits with 1. An exception the command does not handle prints the envelope with stage internal and exits with 1 too. A process killed from outside, by a hard limit or a signal, cannot print anything: stdout is then empty, and the exit code is the one the system gives.
• JSON mode is read before anything else: --format=json, --format json and --json count wherever they sit on the command line, so a mistake in another option is reported as JSON too. An unknown command, or --json placed before the command, is reported as JSON as well. "format": "json" in regex.json asks for JSON once lint reads its options: the report and the errors lint finds are JSON, but an error in a global option, read before regex.json, is reported as text unless the command line asks for JSON too.
• --format values are case-insensitive: --format=JSON is --format=json. When the command line holds several --format or --json options, the last one wins, on a failure as on a success.
• An unknown --format value does not ask for JSON: it is a usage error printed as text.

Keys and values

• Every key is snake_case.
• Every key a table lists is present, null when it does not apply. A key whose type says “optional” may be absent.
• A value listed as one of a few words (a severity, a stage, a category) is an open set: a minor release may add a value. Treat a value you do not know as you would the closest one you do, or ignore it.
• Messages, hints and tips are for people. Their wording is not frozen and may improve in any release. Match on issue_id, error_code, stage or severity, never on a message.

Strings are written as they are

A string is written as itself: a pattern holding a newline or a control byte comes out with JSON’s own escape for it (\n, \u0001), which decodes back to the same text. The one exception is a byte that is not valid UTF-8, which JSON cannot carry: each such byte is written as the four characters \xNN (uppercase hex) and the rest of the string is kept. Such a string is a display form, so a document can always be encoded, even for a byte-mode pattern.

The caret of caret_snippet is placed by counting the raw bytes of the pattern. A byte written as \xNN takes four characters in the snippet but counts as one, so the caret of a pattern holding such a byte sits to the left of the place it means, by three columns for each such byte before it.

Positions

Every position counts bytes.

• line, column and file_offset of a lint result point at the start of the PHP expression that holds the pattern: the opening quote of its first string literal. file_offset is null, and column too, when the pattern does not come from a PHP string, as a route requirement read from YAML.
• The pattern body is what sits between the delimiters. In /(a/, offset 0 is the (. This is the coordinate PCRE2’s own error offsets use. An error in the modifiers counts on past the body and its closing delimiter.
• A hotspot’s end is exclusive: the span is start to end - 1.
• Editors count in UTF-16 code units. An editor integration should use the language server, which converts, rather than these byte offsets.

Paths

file is relative to the working directory, with / separators, when the file lies under it, and absolute otherwise. regex lint src and regex lint "$PWD/src" print the same file values. The baseline file uses the same rule, with its . and .. segments resolved and \ read as a separator, so regex lint ./src and regex lint src share a baseline.

Order

• The lint report lists its results by file (compared byte by byte), then line, then column (null first), then file_offset, then source.
• The issues of a result are listed by position (null first), then issue_id.
• The issues of the baseline file are sorted like the results.
• lint.target.range lists the floor first, then the other PHP versions in ascending order.

The order is the same whatever the number of workers (--jobs). Timing fields (duration_ms, wall_ms, avg_ms, cpu_ms) differ between two runs, so two runs may not print the same bytes.

What a minor release may change

• Add a key to any object, and add an optional key to the error envelope.
• Add a value to an open set: a stage, a severity, a category, an issue_id, an error_code.
• Improve a message, a hint, a tip or the escaping of a display string.
• Add an entry to lint.target.range: a PHP version a rule of the library changes at, learned in that release, which the project’s require.php allows.

It never renames or removes a key, changes a key’s type, or turns a success document into one with a top-level error.

Versions

No document carries a version key. The package version is the schema version: regex --version prints it. redos_analysis.analysis_version is something else, the version of the ReDoS model behind a verdict.

The Error Envelope

A run that cannot produce its document prints the envelope instead:

{
    "error": "Unknown option: --bogus",
    "stage": "usage"
}

A success document never has a top-level error key, so testing for it tells the two apart.

Error envelope: error

stage is an open vocabulary. Today it is one of:

Other keys may come next to error and stage in a minor release, as validation did.

Invalid Patterns

An invalid pattern is a finding, not a failure, for the commands that report on patterns:

• lint reports it as an issue of severity error. Its issue_id is the error code, such as regex.group.unclosed, and its validation says why.
• analyze prints its document with parse.ok false, validation saying why, and redos and explain null. The exit code is 1.
• debug prints its document with validation saying why and analysis null. The exit code is 1.

The commands that act on a pattern, transpile and redos, cannot. They print the envelope with stage pattern, carrying the same validation object, and exit with 1. redos judges the pattern given with --safe the same way: an invalid one stops it with the envelope, stage pattern. redos stops so in JSON only: its console output still measures a pattern PHP refuses.

Lint Report

regex lint --format=json prints one report for the whole run. --json is short for --format=json.

{
    "target": {
        "php": "8.2",
        "pcre": "10.40",
        "source": "composer.json require.php",
        "range": [{"php": "8.2", "pcre": "10.40"}, {"php": "8.3", "pcre": "10.42"}, {"php": "8.4", "pcre": "10.44"}, {"php": "8.4.25", "pcre": "10.44"}, {"php": "8.5", "pcre": "10.44"}, {"php": "8.5.10", "pcre": "10.44"}]
    },
    "stats": {"errors": 1, "warnings": 0, "optimizations": 0, "redos_errors": 0, "infos": 0, "lint_errors": 0, "parser_fallbacks": 0},
    "results": [
        {
            "file": "src/Example.php",
            "line": 3,
            "column": 12,
            "file_offset": 18,
            "source": "preg_match()",
            "pattern": "/(a/",
            "location": null,
            "issues": [
                {
                    "severity": "error",
                    "file": "src/Example.php",
                    "line": 3,
                    "column": 12,
                    "file_offset": 18,
                    "position": 2,
                    "issue_id": "regex.group.unclosed",
                    "message": "Expected ) at end of input (found eof)",
                    "hint": null,
                    "tip": null,
                    "source": "preg_match()",
                    "validation": {
                        "is_valid": false,
                        "error": "Expected ) at end of input (found eof)",
                        "complexity_score": 0,
                        "category": "syntax",
                        "offset": 2,
                        "caret_snippet": "Line 1: (a\n          ^",
                        "hint": null,
                        "error_code": "regex.group.unclosed"
                    },
                    "analysis": null,
                    "target": null
                }
            ],
            "optimizations": []
        }
    ]
}

Lint report: lint

Target: lint.target

php, pcre and source name the floor: the lint rules and the ReDoS analysis judge every pattern there. When the PHP comes from require.php, each pattern is also validated (parsed and checked, without the lint rules) at every PHP version above the floor that a rule of the library changes at and that the constraint allows, and at the lowest version of each branch of an OR constraint above the floor (~8.3.0 || >=8.5.3 adds 8.5.3), with the PCRE2 that PHP bundles, or with the release --pcre-version or pcreVersion names. See Several PHP versions.

Target range entry: lint.target.range[]

Stats: lint.stats

Every count is present on every run, 0 when there is none.

Result: lint.results[]

Issue: lint.results[].issues[]

An issue repeats the location of its result, so it can be read alone.

An issue whose target is not null has severity error, and its validation is the verdict of that target. A pattern the floor refuses is reported once, by the floor, and not validated at the other versions. A baseline does not match on target.

Issue target: lint.results[].issues[].target

Optimization entry: lint.results[].optimizations[]

Optimization: lint.results[].optimizations[].optimization

Analyze

regex analyze <pattern> --format=json reports on one pattern. On an invalid pattern, parse.ok is false, validation says why, redos and explain are null, and the exit code is 1.

Analyze: analyze

Parse: analyze.parse

Debug

regex debug <pattern> --format=json reports the ReDoS analysis in depth. It always carries validation; on an invalid pattern analysis is null and the exit code is 1.

Debug: debug

Debug input: debug.input

Redos

regex redos <pattern> --format=json times the pattern against a subject, and a safer pattern given with --safe against the same subject. An invalid pattern, or an invalid --safe pattern, stops it with the envelope (see Invalid Patterns).

Benchmark document: redos

Benchmark input: redos.input

Benchmark settings: redos.settings

Measurements: redos.bench

Benchmark: benchmark

One measurement, under redos.bench.vuln and redos.bench.safe.

Comparison: redos.summary

Transpile

regex transpile <pattern> --format=json rewrites a pattern for another regex dialect. Symfony’s and Laravel’s regex:transpile --format=json print the same document.

The pattern is validated first. An invalid pattern stops the command with the envelope, stage pattern, carrying validation, a semantic error such as /(?<=a+)b/ included. A valid pattern that uses a construct the target has no equivalent for stops it with the envelope too, stage pattern, without a validation key. Both exit with 1:

{
    "error": "Subroutine calls are not supported in JavaScript.",
    "stage": "pattern"
}

Transpile: transpile

Shared Objects

Runtime: runtime

The PCRE a command ran on, in analyze, debug and redos.

Validation: validation

The verdict on a pattern, in analyze, debug, a lint issue and the error envelope. It is the JSON form of ValidationResult: json_encode() of one gives these keys, in this order.

ReDoS analysis: redos_analysis

A ReDoS verdict: analyze.redos, debug.analysis and the analysis of a lint issue. It is the JSON form of RedosAnalysis; the ReDoS guide explains how to read it.

Finding: redos_analysis.findings[]

Hotspot: redos_analysis.hotspots[]

Witness: redos_analysis.witness

The attack is the prefix, then the pump repeated, then the suffix. Each part is written as the body of a PHP double-quoted string, so "<prefix>" . str_repeat("<pump>", $n) . "<suffix>" builds it.

Search cost: redos_analysis.search_cost

One attempt is proven linear, but an unanchored search starts one at each position of a run, and each reads to the end of the run before it fails: on the run repeated n times then the breaker, PCRE2’s interpreter takes a number of steps quadratic in n. preg_match_all(), preg_replace() and preg_split() retry the same way. It is looked for only when one attempt is proven linear: a worse per-attempt verdict already covers the search. The JIT may avoid the cost for some patterns; it is not measured, as the analysis never runs a pattern under the JIT.

Search witness: redos_analysis.search_cost.witness

Each part is written as the body of a PHP double-quoted string, like the witness: "<prefix>" . str_repeat("<run>", $n) . "<breaker>" builds the attack.

Confirmation: redos_analysis.confirmation

Confirmation sample: redos_analysis.confirmation.samples[]

Baseline File

regex lint --generate-baseline=<file> writes the issues of the run to <file>: one JSON document, its issues sorted like the report, ending with a newline. --baseline=<file> then hides the issues it lists. An issue matches an entry by issue_id, file and pattern_hash, so a pattern that moves down its file or a reworded message does not bring it back; line only decides between entries that share all three, and an issue’s target plays no part. One entry hides one issue. A run with --baseline still reports column and file_offset for the issues left.

A 1.x baseline, a plain list of {file, line, message} entries, is still read, matched by file, line and message, with a note on stderr (on stdout with the console report) suggesting to generate it again. A baseline that is missing or holds neither form is a usage error.

{
    "version": 1,
    "issues": [
        {
            "file": "src/Example.php",
            "line": 3,
            "column": 12,
            "issue_id": "regex.group.unclosed",
            "message": "Expected ) at end of input (found eof)",
            "severity": "error",
            "pattern": "/(a/",
            "pattern_hash": "280856e984a6cedd308bdb41caaa4672"
        }
    ]
}

Baseline file: baseline

Baseline entry: baseline.issues[]

Edit on GitHub Copy as Markdown