erts: Yield inside memchr() calls done by re:run (PCRE2) #10486
Conversation
CT Test Results 4 files 228 suites 1h 57m 11s ⏱️ Results for commit 886be7a. ♻️ This comment has been updated with latest results. To speed up review, make sure that you have read Contributing to Erlang/OTP and that all checks pass. See the TESTING and DEVELOPMENT HowTo guides for details about how to run test locally. Artifacts// Erlang/OTP Github Action Bot |
There was a problem hiding this comment.
Pull request overview
This PR enhances the cooperative scheduling mechanism in Erlang's PCRE2 regex implementation by introducing yield points inside memchr() calls. When searching very large subject strings with regular expressions, the system can now yield during character searches, preventing long-running operations from blocking the scheduler. This complements the existing yield mechanism that operates inside the main match loop.
Key changes:
- Added
memchr_erlang()wrapper function that can yield during character searches based on a loop budget - Extended yield coverage tracking from
inttouint64_tto handle the increased number of yield points and higher counts - Increased
LOOP_FACTORfrom 10 to 16 to adjust the yielding granularity with the new yield points
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| erts/emulator/pcre/pcre2_match.c | Core implementation of yielding memchr, new MEMCHR_ERLANG macro, extended PcreExecContext with yield state for memchr, changed yield tracking counters to uint64_t, added restore logic for memchr yield points |
| erts/emulator/pcre/pcre.mk | Added generation of pcre2_match_memchr_break_cases.gen.h for memchr yield point restoration, updated yield coverage generation to include MEMCHR_ERLANG calls |
| erts/emulator/pcre/depend.mk | Added dependency on pcre2_match_memchr_break_cases.gen.h for pcre2_match.o |
| lib/stdlib/test/re_SUITE.erl | Enhanced yield coverage checking to accumulate and report total yields, improved error detection for impossible visit/yield ratios |
| lib/stdlib/test/run_pcre_tests.erl | Renamed variable from T to Rest for clarity, reformatted error output messages |
| erts/emulator/beam/erl_bif_re.c | Adjusted LOOP_FACTOR from 10 to 16 to account for additional yield points |
| erts/emulator/beam/erl_bif_info.c | Changed yield tracking from int to uint64_t, added proper heap allocation for large integers, added counter reset, removed redundant is_atom checks |
| erts/emulator/nifs/common/prim_tty_nif.c | Fixed assertion to allow i >= 0 instead of i > 0 (character 0 is valid for isprint) |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Improve cooperative scheduling when searching very large subject strings with regular expressions.