Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Introduce cpp micro-benchmarks #101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Sep 26, 2022
Merged

Introduce cpp micro-benchmarks #101

merged 3 commits into from
Sep 26, 2022

Conversation

@DamianSzwichtenberg
Copy link
Member

@DamianSzwichtenberg DamianSzwichtenberg commented Sep 9, 2022 •
edited
Loading

This PR introduces C++ benchmarking infrastructure (GoogleBenchmark). As pyg-lib can be used as a standalone C++ library it makes sense to have micro-benchmarks that would benchmark not only pieces of code with a Python binding. Additionally, an exemplary benchmark was provided for pyg::sampler::Mapper.

Output for provided benchmark looks like follows:

-----------------------------------------------------------------------------------------------------------------------------------------------------------------
Benchmark                                                                                                       Time             CPU   Iterations UserCounters...
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithFlatHashMap/num_nodes:1024000/num_entries:-1    7394664 ns      7392668 ns           91 Insertion Fail Rate [%]=25.0501
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithFlatHashMap/num_nodes:1152000/num_entries:-1    8760396 ns      8757657 ns           77 Insertion Fail Rate [%]=25.0199
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithFlatHashMap/num_nodes:1280000/num_entries:-1    8948630 ns      8945524 ns           76 Insertion Fail Rate [%]=25.0318
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithFlatHashMap/num_nodes:1408000/num_entries:-1   10103691 ns     10099494 ns           68 Insertion Fail Rate [%]=25.0122
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithFlatHashMap/num_nodes:1536000/num_entries:-1   11302446 ns     11296285 ns           61 Insertion Fail Rate [%]=24.993
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithFlatHashMap/num_nodes:1664000/num_entries:-1   12936696 ns     12929287 ns           54 Insertion Fail Rate [%]=25.0384
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithFlatHashMap/num_nodes:1792000/num_entries:-1   14151064 ns     14144356 ns           49 Insertion Fail Rate [%]=25.0186
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithFlatHashMap/num_nodes:1920000/num_entries:-1   15588397 ns     15576363 ns           45 Insertion Fail Rate [%]=25.0133
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithFlatHashMap/num_nodes:2048000/num_entries:-1   17158246 ns     17144487 ns           41 Insertion Fail Rate [%]=25.007
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithFlatHashMap_BigO                                   7.77 N          7.77 N    
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithFlatHashMap_RMS                                       6 %             6 %    
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithVector/num_nodes:1024000/num_entries:1024000    6776261 ns      6773525 ns          103 Insertion Fail Rate [%]=25.0364
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithVector/num_nodes:1152000/num_entries:1152000    7983127 ns      7979648 ns           89 Insertion Fail Rate [%]=24.9776
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithVector/num_nodes:1280000/num_entries:1280000    8971134 ns      8965624 ns           78 Insertion Fail Rate [%]=25.0282
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithVector/num_nodes:1408000/num_entries:1408000   10156930 ns     10151395 ns           68 Insertion Fail Rate [%]=25.0169
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithVector/num_nodes:1536000/num_entries:1536000   11407066 ns     11399957 ns           61 Insertion Fail Rate [%]=25.004
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithVector/num_nodes:1664000/num_entries:1664000   12593205 ns     12587564 ns           55 Insertion Fail Rate [%]=24.9894
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithVector/num_nodes:1792000/num_entries:1792000   14226021 ns     14215225 ns           49 Insertion Fail Rate [%]=25.0002
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithVector/num_nodes:1920000/num_entries:1920000   15570986 ns     15562529 ns           44 Insertion Fail Rate [%]=24.9714
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithVector/num_nodes:2048000/num_entries:2048000   17869559 ns     17851791 ns           39 Insertion Fail Rate [%]=24.996
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithVector_BigO                                        7.76 N          7.75 N    
BenchmarkMapperCreationAndInsertion<int64_t, int64_t>/WithVector_RMS                                            8 %             8 %

@DamianSzwichtenberg DamianSzwichtenberg self-assigned this Sep 9, 2022
@codecov-commenter
Copy link

codecov-commenter commented Sep 14, 2022 •
edited
Loading

Codecov Report

Merging #101 (296ff53) into master (d0ce8be) will not change coverage.
The diff coverage is n/a.

❗ Current head 296ff53 differs from pull request most recent head fd2e5f2. Consider uploading reports for the commit fd2e5f2 to get more accurate results

@@           Coverage Diff           @@
##           master     #101   +/-   ##
=======================================
  Coverage   89.32%   89.32%           
=======================================
  Files          16       16           
  Lines         412      412           
=======================================
  Hits          368      368           
  Misses         44       44

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@DamianSzwichtenberg
Copy link
Member Author

DamianSzwichtenberg commented Sep 19, 2022 •
edited
Loading

@rusty1s, waiting for some feedback 😉

rusty1s
rusty1s approved these changes Sep 20, 2022
View reviewed changes
Copy link
Member

@rusty1s rusty1s left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is super cool, thanks for adding it!

@DamianSzwichtenberg
Copy link
Member Author

@rusty1s I replied to all comments, can I merge this PR? 😉

rusty1s
rusty1s approved these changes Sep 26, 2022
View reviewed changes
@rusty1s
Copy link
Member

rusty1s commented Sep 26, 2022

@dszwicht Yes, ofc. Please go ahead and merge it if it is ready.

@DamianSzwichtenberg DamianSzwichtenberg merged commit 1dc4a03 into pyg-team:master Sep 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rusty1s rusty1s rusty1s approved these changes

Assignees

@DamianSzwichtenberg DamianSzwichtenberg

Labels

0 - Priority P0 benchmark sampler

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@DamianSzwichtenberg @codecov-commenter @rusty1s