-
Notifications
You must be signed in to change notification settings - Fork 1.6k
<regex>
: Properly parse backslashes in character classes of basic regexes
#5523
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
<regex>
: Properly parse backslashes in character classes of basic regexes
#5523
Conversation
…arse flag for forbidden dash at range start
Reviewing now, I'll push changes soon. I updated the PR description from saying " |
Use muellerj2's superior descriptions of `_L_alt_nl` and `_L_no_nl`. Note that `_L_no_nl` is (grep, egrep). Note that `_L_esc_oct` and `_L_esc_ffnx` are (awk). `_L_esc_ffn` confusingly said "(\[fnrtv])" when other comments like `_L_ident_ERE` mean square brackets literally. Spell out "(\f \n \r \t \v)" for clarity and improved searchability. Rephrase `_L_ident_awk`'s comment for clarity. Note that `_L_anch_rstr` is (BRE) only, `_L_paren_bal` is (ERE) only, and `_L_brk_rstr` is (ERE, BRE).
Thanks!! 😻 I pushed some follow-up commits for additional cleanups, please double-check. I really appreciate the well-structured commit history here; ordinarily I would be nervous about mixing a refactoring and a bugfix but this was entirely reasonable. |
I'm mirroring this to the MSVC-internal repo - please notify me if any further changes are pushed. |
Thanks for the infinite bugfixes in infinite combinations, as the Vulcans say! 🖖 🐞 🛠️ |
Fixes #5379.
This renames the parser to add a new member variable describing the lexer mode: Default or inside character class. This allows the lexer to correctly process a backslash when parsing a character class/bracket expression.
I also tried to do this without renaming the parser, but this would mean we would have to pass the lexer mode (in or outside a character class) as an argument to all the functions processing escapes in any way, which is a bit of a pain. By renaming the parser, we need the least changes to the logic itself.
Since the parser is renamed, this PR is also doing a number of minor cleanups to the parser and builder (which is also renamed to do these cleanups).
The PR is split into several commits to simplify reviewing:
_Parser
to_Parser2
._Compile
. Specifically:_L_brk_bal
is assigned its own bit; previously it wasSTL/stl/inc/regex
Line 1879 in cbd091e
_L_grp_esc
flag is added to the awk flags so that the workaround in_ClassAtom
can be removed._Lang_flags
enum and the_L_flags
member variable tounsigned long long
so that we can add more flags more easily in the future. (This already adds_L_dsh_rstr
to signify that the dash-
cannot appear as the starting point of a character range in BREs and EREs, but doesn't perform the parser changes to support it yet.)_Begin
from the parser._Char
is usually achar
orwchar_t
, so it [plus the single-byte_Mode
member variable added in the last commit] can usually fit into the four bytes the compiler must add after_Mchar
._Builder
to_Builder2
._Bmax
and_Tmax
from the builder.<regex>
: Backslashes in character classes are sometimes not matched in basic regular expressions #5379 essentially by making_Is_esc()
always return false when not in default (read: outside-bracketed-character-class) mode. Note that it matters how we change the lexer mode in_Parser2::_Alternative()
:_Next()
and_Expect()
process the first token inside or outside the square brackets, so we must change the mode before calling these functions. The tests check that we didn't get this wrong.