Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 9960dbd

Browse files
masklinnmasklinn
committed
Add benchmarking scripts
useragents.txt sample file kindly provided by @DailyMats out of DailyMotion's data (2023-04-26). The provided scripts allow: - Testing the cache hit rate of various cache configuration (algorithm and size) on sample files, this script uses a dummy parser and is thus extremely fast. - Benchmarking the average entry processing of various parser configurations (base parser + cache algoritm + cache size) on sample files, this is a much slower script but provides a a realistic evaluation, and allows using custom rules (`regexes.yaml` files) to check their impact on the performance of a given base parser. Also added a script for testing threaded parsing, as expected this gets 0 gain over the normal stuff because of the GIL (and re2 seemingly doesn't release the GIL either, though I don't know how beneficial it would be at ~30us per call). May be more useful with 3.13, or possibly with a regex-based extension releasing the GIL, at least the basis for testing things out will be here.
1 parent e719a7e commit 9960dbd

File tree

5 files changed

+75591
-1
lines changed

5 files changed

+75591
-1
lines changed

0 commit comments

Comments
 (0)