I noticed a few dependabot PRs in this repo, such as this one that upgrade dependencies of specific benchmarks. While in general, I think this is good practice, for a benchmark suite, I think we'd want to upgrade these dependencies as infrequently as possible to keep benchmarking results comparable with one another (and not have to always rerun baselines). Occasionally we are forced to upgrade, for example to get compatibility with a new version of CPython, but that should be deliberate.

(It's possible there is a security counterargument to be made, but I'm not a security expert and I don't know specifically whether that matters or not).

Would it make sense to update the dependabot config to only look at the top-level dependencies of pyperformance itself rather than the dependencies of specific benchmarks?