Skip to content

gh-150817: Speed up Flag bitwise operations#150824

Open
gaborbernat wants to merge 1 commit into
python:mainfrom
gaborbernat:opt/enum-flag-ops
Open

gh-150817: Speed up Flag bitwise operations#150824
gaborbernat wants to merge 1 commit into
python:mainfrom
gaborbernat:opt/enum-flag-ops

Conversation

@gaborbernat
Copy link
Copy Markdown
Contributor

@gaborbernat gaborbernat commented Jun 2, 2026

Combining Flag members with |, & and ^ walks both operands on every call to check that neither carries a None value, raising a clear TypeError when one does. That check only matters in the rare case where a value really is None; for every normal combination of two valid flags it adds work to each operation. Flag arithmetic is hot wherever flags model option sets — the re module's own flags are an IntFlag, so each re.compile(pattern, re.I | re.M) combines them, and permission systems, feature toggles and styling layers do the same in tight loops.

The two values the guard inspects are already in hand: one is the member's own value, the other is the operand value computed just above. This runs the scan over the operands only when one of those is actually None, so a valid combination skips it entirely. The offending-flag walk, the TypeError and its message are unchanged for the invalid cases.

Running flag combinations mined from the top-1000 corpus, including the re flag sets, improves from 72.6 µs to 49.5 µs, 47% faster.

Benchmark base patched
re.RegexFlag combine/and/in 56.5 µs 38.5 µs: 47% faster
IntFlag or/and/xor/in 72.6 µs 49.5 µs: 47% faster
Benchmark (pyperf)

Run base vs patched by swapping Lib/enum.py on the same interpreter. The flag combinations are real re flag sets mined from the top-1000 corpus.

import re, functools, operator, pyperf

combos_str = ["re.DOTALL|re.I", "re.DOTALL|re.M", "re.DOTALL|re.VERBOSE",
    "re.DOTALL|re.VERBOSE|re.M", "re.IGNORECASE|re.VERBOSE", "re.I|re.M",
    "re.I|re.S", "re.I|re.S|re.X", "re.MULTILINE|re.DOTALL",
    "re.MULTILINE|re.DOTALL|re.VERBOSE", "re.M|re.S", "re.S|re.M",
    "re.VERBOSE|re.I", "re.VERBOSE|re.M"]
M = {"I": re.I, "IGNORECASE": re.I, "M": re.M, "MULTILINE": re.M, "S": re.S,
     "DOTALL": re.S, "X": re.X, "VERBOSE": re.X, "A": re.A, "ASCII": re.A}
combos = [[M[p.split(".")[1]] for p in c.split("|")] for c in combos_str]

def run():
    for flags in combos:
        combined = functools.reduce(operator.or_, flags)
        flags[0] in combined

runner = pyperf.Runner()
runner.bench_func("re.RegexFlag combine %d real corpus combos" % len(combos), run)

Resolves #150817.

Flag.__or__, __and__ and __xor__ walked both operands on every call to reject
None values. Run that scan only when one of the operand values is actually
None, so valid combinations skip it. The TypeError and its message are
unchanged for the invalid cases.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Speed up Flag bitwise operations (|, &, ^)

2 participants