Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[libc++] Make benchmarks forward-compatible with the test suite #114502

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

ldionne
Copy link
Member

@ldionne ldionne commented Nov 1, 2024

This patch fixes warnings and errors that come up when running the benchmarks as part of the test suite. It also adds the necessary Lit annotations to make it pass in various configurations and increases the portability of the benchmarks.

@ldionne ldionne requested a review from a team as a code owner November 1, 2024 02:56
@llvmbot llvmbot added the libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi. label Nov 1, 2024
@llvmbot
Copy link
Member

llvmbot commented Nov 1, 2024

@llvm/pr-subscribers-libcxx

Author: Louis Dionne (ldionne)

Changes

This patch fixes warnings and errors that come up when running the benchmarks as part of the test suite. It also adds the necessary Lit annotations to make it pass in various configurations and increases the portability of the benchmarks.


Patch is 63.56 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/114502.diff

74 Files Affected:

  • (modified) libcxx/test/benchmarks/CartesianBenchmarks.h (+3-3)
  • (modified) libcxx/test/benchmarks/ContainerBenchmarks.h (+5-5)
  • (modified) libcxx/test/benchmarks/VariantBenchmarks.h (+1-1)
  • (modified) libcxx/test/benchmarks/algorithms.partition_point.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/count.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/equal.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/fill.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/find.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/for_each.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/lexicographical_compare.bench.cpp (+6-2)
  • (modified) libcxx/test/benchmarks/algorithms/lower_bound.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/make_heap.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/make_heap_then_sort_heap.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/min.bench.cpp (+5)
  • (modified) libcxx/test/benchmarks/algorithms/min_max_element.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/minmax.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/mismatch.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/pop_heap.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/pstl.stable_sort.bench.cpp (+3)
  • (modified) libcxx/test/benchmarks/algorithms/push_heap.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/ranges_contains.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/ranges_ends_with.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/ranges_make_heap.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/ranges_make_heap_then_sort_heap.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/ranges_pop_heap.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/ranges_push_heap.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/ranges_sort.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/ranges_sort_heap.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/ranges_stable_sort.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/set_intersection.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/sort.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/sort_heap.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/algorithms/stable_sort.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/allocation.bench.cpp (+10-1)
  • (modified) libcxx/test/benchmarks/atomic_wait.bench.cpp (+3)
  • (modified) libcxx/test/benchmarks/atomic_wait_vs_mutex_lock.bench.cpp (+3)
  • (modified) libcxx/test/benchmarks/deque.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/deque_iterator.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/exception_ptr.bench.cpp (+4-1)
  • (modified) libcxx/test/benchmarks/filesystem.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/format.bench.cpp (+5)
  • (modified) libcxx/test/benchmarks/format/write_double_comparison.bench.cpp (+6-1)
  • (modified) libcxx/test/benchmarks/format/write_int_comparison.bench.cpp (+6-1)
  • (modified) libcxx/test/benchmarks/format/write_string_comparison.bench.cpp (+6-13)
  • (modified) libcxx/test/benchmarks/format_to.bench.cpp (+5)
  • (modified) libcxx/test/benchmarks/format_to_n.bench.cpp (+5)
  • (modified) libcxx/test/benchmarks/formatted_size.bench.cpp (+5)
  • (modified) libcxx/test/benchmarks/formatter_float.bench.cpp (+7-3)
  • (modified) libcxx/test/benchmarks/formatter_int.bench.cpp (+5)
  • (modified) libcxx/test/benchmarks/function.bench.cpp (+5-1)
  • (modified) libcxx/test/benchmarks/hash.bench.cpp (-38)
  • (modified) libcxx/test/benchmarks/join_view.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/lexicographical_compare_three_way.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/libcxxabi/dynamic_cast.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/libcxxabi/dynamic_cast_old_stress.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/map.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/monotonic_buffer.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/numeric/gcd.bench.cpp (+4-1)
  • (modified) libcxx/test/benchmarks/ordered_set.bench.cpp (+5-2)
  • (modified) libcxx/test/benchmarks/random.bench.cpp (+3)
  • (modified) libcxx/test/benchmarks/shared_mutex_vs_mutex.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/std_format_spec_string_unicode.bench.cpp (+6-1)
  • (modified) libcxx/test/benchmarks/std_format_spec_string_unicode_escape.bench.cpp (+6-1)
  • (modified) libcxx/test/benchmarks/stop_token.bench.cpp (+3)
  • (modified) libcxx/test/benchmarks/string.bench.cpp (+18-10)
  • (modified) libcxx/test/benchmarks/stringstream.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/system_error.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/to_chars.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/unordered_set_operations.bench.cpp (+15-79)
  • (modified) libcxx/test/benchmarks/util_smartptr.bench.cpp (+6-3)
  • (modified) libcxx/test/benchmarks/variant_visit_1.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/variant_visit_2.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/variant_visit_3.bench.cpp (+2)
  • (modified) libcxx/test/benchmarks/vector_operations.bench.cpp (+2)
diff --git a/libcxx/test/benchmarks/CartesianBenchmarks.h b/libcxx/test/benchmarks/CartesianBenchmarks.h
index eca4e15cd009bc..c712230843ebf4 100644
--- a/libcxx/test/benchmarks/CartesianBenchmarks.h
+++ b/libcxx/test/benchmarks/CartesianBenchmarks.h
@@ -27,11 +27,11 @@ constexpr auto makeEnumValueTuple(std::index_sequence<Idxs...>) {
 }
 
 template <class B>
-static auto skip(const B& Bench, int) -> decltype(Bench.skip()) {
+auto skip(const B& Bench, int) -> decltype(Bench.skip()) {
   return Bench.skip();
 }
 template <class B>
-static auto skip(const B& Bench, char) {
+auto skip(const B&, char) {
   return false;
 }
 
@@ -51,7 +51,7 @@ void makeBenchmarkFromValues(const std::vector<std::tuple<Args...> >& A) {
 }
 
 template <template <class...> class B, class Args, class... U>
-void makeBenchmarkImpl(const Args& A, std::tuple<U...> t) {
+void makeBenchmarkImpl(const Args& A, std::tuple<U...>) {
   makeBenchmarkFromValues<B<U...> >(A);
 }
 
diff --git a/libcxx/test/benchmarks/ContainerBenchmarks.h b/libcxx/test/benchmarks/ContainerBenchmarks.h
index 5404814e7599b3..742c848328604c 100644
--- a/libcxx/test/benchmarks/ContainerBenchmarks.h
+++ b/libcxx/test/benchmarks/ContainerBenchmarks.h
@@ -150,7 +150,7 @@ void BM_EmplaceDuplicate(benchmark::State& st, Container c, GenInputs gen) {
 }
 
 template <class Container, class GenInputs>
-static void BM_Find(benchmark::State& st, Container c, GenInputs gen) {
+void BM_Find(benchmark::State& st, Container c, GenInputs gen) {
   auto in = gen(st.range(0));
   c.insert(in.begin(), in.end());
   benchmark::DoNotOptimize(&(*c.begin()));
@@ -164,7 +164,7 @@ static void BM_Find(benchmark::State& st, Container c, GenInputs gen) {
 }
 
 template <class Container, class GenInputs>
-static void BM_FindRehash(benchmark::State& st, Container c, GenInputs gen) {
+void BM_FindRehash(benchmark::State& st, Container c, GenInputs gen) {
   c.rehash(8);
   auto in = gen(st.range(0));
   c.insert(in.begin(), in.end());
@@ -179,7 +179,7 @@ static void BM_FindRehash(benchmark::State& st, Container c, GenInputs gen) {
 }
 
 template <class Container, class GenInputs>
-static void BM_Rehash(benchmark::State& st, Container c, GenInputs gen) {
+void BM_Rehash(benchmark::State& st, Container c, GenInputs gen) {
   auto in = gen(st.range(0));
   c.max_load_factor(3.0);
   c.insert(in.begin(), in.end());
@@ -193,7 +193,7 @@ static void BM_Rehash(benchmark::State& st, Container c, GenInputs gen) {
 }
 
 template <class Container, class GenInputs>
-static void BM_Compare_same_container(benchmark::State& st, Container, GenInputs gen) {
+void BM_Compare_same_container(benchmark::State& st, Container, GenInputs gen) {
   auto in = gen(st.range(0));
   Container c1(in.begin(), in.end());
   Container c2 = c1;
@@ -208,7 +208,7 @@ static void BM_Compare_same_container(benchmark::State& st, Container, GenInputs
 }
 
 template <class Container, class GenInputs>
-static void BM_Compare_different_containers(benchmark::State& st, Container, GenInputs gen) {
+void BM_Compare_different_containers(benchmark::State& st, Container, GenInputs gen) {
   auto in1 = gen(st.range(0));
   auto in2 = gen(st.range(0));
   Container c1(in1.begin(), in1.end());
diff --git a/libcxx/test/benchmarks/VariantBenchmarks.h b/libcxx/test/benchmarks/VariantBenchmarks.h
index ad36b59d2e879d..bb0c7142791507 100644
--- a/libcxx/test/benchmarks/VariantBenchmarks.h
+++ b/libcxx/test/benchmarks/VariantBenchmarks.h
@@ -35,7 +35,7 @@ static auto genVariants(std::index_sequence<Is...>) {
 
   std::array<V, N> result = {};
   for (auto& v : result) {
-    v = fs[getRandomInteger(0ul, sizeof...(Is) - 1)]();
+    v = fs[getRandomInteger(std::size_t(0), sizeof...(Is) - 1)]();
   }
 
   return result;
diff --git a/libcxx/test/benchmarks/algorithms.partition_point.bench.cpp b/libcxx/test/benchmarks/algorithms.partition_point.bench.cpp
index ed2e337fa6ea31..42ebce8ad2f4ad 100644
--- a/libcxx/test/benchmarks/algorithms.partition_point.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms.partition_point.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14
+
 #include <algorithm>
 #include <array>
 #include <cassert>
diff --git a/libcxx/test/benchmarks/algorithms/count.bench.cpp b/libcxx/test/benchmarks/algorithms/count.bench.cpp
index 7370293fd6efd0..46b85e909efa5d 100644
--- a/libcxx/test/benchmarks/algorithms/count.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/count.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
 #include <algorithm>
 #include <benchmark/benchmark.h>
 #include <cstring>
diff --git a/libcxx/test/benchmarks/algorithms/equal.bench.cpp b/libcxx/test/benchmarks/algorithms/equal.bench.cpp
index 6d63d8c48ce1e0..2dc11585c15c7f 100644
--- a/libcxx/test/benchmarks/algorithms/equal.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/equal.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
 #include <algorithm>
 #include <benchmark/benchmark.h>
 #include <vector>
diff --git a/libcxx/test/benchmarks/algorithms/fill.bench.cpp b/libcxx/test/benchmarks/algorithms/fill.bench.cpp
index 40f37425c394cf..c157b5e5c9862c 100644
--- a/libcxx/test/benchmarks/algorithms/fill.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/fill.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14, c++17, c++20
+
 #include <algorithm>
 #include <benchmark/benchmark.h>
 #include <vector>
diff --git a/libcxx/test/benchmarks/algorithms/find.bench.cpp b/libcxx/test/benchmarks/algorithms/find.bench.cpp
index 6ff2d95ab43533..43d103474ebdf0 100644
--- a/libcxx/test/benchmarks/algorithms/find.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/find.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
 #include <algorithm>
 #include <benchmark/benchmark.h>
 #include <cstring>
diff --git a/libcxx/test/benchmarks/algorithms/for_each.bench.cpp b/libcxx/test/benchmarks/algorithms/for_each.bench.cpp
index 7019dc13ca601a..554c9ec9930434 100644
--- a/libcxx/test/benchmarks/algorithms/for_each.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/for_each.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14
+
 #include <algorithm>
 #include <benchmark/benchmark.h>
 #include <deque>
diff --git a/libcxx/test/benchmarks/algorithms/lexicographical_compare.bench.cpp b/libcxx/test/benchmarks/algorithms/lexicographical_compare.bench.cpp
index 0c545263109d29..29758ea3fa770f 100755
--- a/libcxx/test/benchmarks/algorithms/lexicographical_compare.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/lexicographical_compare.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
 #include <algorithm>
 #include <benchmark/benchmark.h>
 #include <vector>
@@ -19,7 +21,8 @@ static void bm_lexicographical_compare(benchmark::State& state) {
   for (auto _ : state) {
     benchmark::DoNotOptimize(vec1);
     benchmark::DoNotOptimize(vec2);
-    benchmark::DoNotOptimize(std::lexicographical_compare(vec1.begin(), vec1.end(), vec2.begin(), vec2.end()));
+    bool result = std::lexicographical_compare(vec1.begin(), vec1.end(), vec2.begin(), vec2.end());
+    benchmark::DoNotOptimize(result);
   }
 }
 BENCHMARK(bm_lexicographical_compare<unsigned char>)->DenseRange(1, 8)->Range(16, 1 << 20);
@@ -34,7 +37,8 @@ static void bm_ranges_lexicographical_compare(benchmark::State& state) {
   for (auto _ : state) {
     benchmark::DoNotOptimize(vec1);
     benchmark::DoNotOptimize(vec2);
-    benchmark::DoNotOptimize(std::ranges::lexicographical_compare(vec1.begin(), vec1.end(), vec2.begin(), vec2.end()));
+    bool result = std::ranges::lexicographical_compare(vec1.begin(), vec1.end(), vec2.begin(), vec2.end());
+    benchmark::DoNotOptimize(result);
   }
 }
 BENCHMARK(bm_ranges_lexicographical_compare<unsigned char>)->DenseRange(1, 8)->Range(16, 1 << 20);
diff --git a/libcxx/test/benchmarks/algorithms/lower_bound.bench.cpp b/libcxx/test/benchmarks/algorithms/lower_bound.bench.cpp
index 3be5010f5bdf20..d9d57969df67a3 100644
--- a/libcxx/test/benchmarks/algorithms/lower_bound.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/lower_bound.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14
+
 #include <algorithm>
 #include <numeric>
 #include <random>
diff --git a/libcxx/test/benchmarks/algorithms/make_heap.bench.cpp b/libcxx/test/benchmarks/algorithms/make_heap.bench.cpp
index dade7b8273107a..b7320e17c3e500 100644
--- a/libcxx/test/benchmarks/algorithms/make_heap.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/make_heap.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14
+
 #include <algorithm>
 
 #include "common.h"
diff --git a/libcxx/test/benchmarks/algorithms/make_heap_then_sort_heap.bench.cpp b/libcxx/test/benchmarks/algorithms/make_heap_then_sort_heap.bench.cpp
index 48f34f8651b971..5991d2846aee4f 100644
--- a/libcxx/test/benchmarks/algorithms/make_heap_then_sort_heap.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/make_heap_then_sort_heap.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14
+
 #include <algorithm>
 
 #include "common.h"
diff --git a/libcxx/test/benchmarks/algorithms/min.bench.cpp b/libcxx/test/benchmarks/algorithms/min.bench.cpp
index a09bd530d01207..ee426c7e727aa4 100644
--- a/libcxx/test/benchmarks/algorithms/min.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/min.bench.cpp
@@ -6,10 +6,13 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
 #include <algorithm>
 #include <cassert>
 
 #include <benchmark/benchmark.h>
+#include "test_macros.h"
 
 void run_sizes(auto benchmark) {
   benchmark->Arg(1)
@@ -68,7 +71,9 @@ BENCHMARK(BM_std_min<char>)->Apply(run_sizes);
 BENCHMARK(BM_std_min<short>)->Apply(run_sizes);
 BENCHMARK(BM_std_min<int>)->Apply(run_sizes);
 BENCHMARK(BM_std_min<long long>)->Apply(run_sizes);
+#ifndef TEST_HAS_NO_INT128
 BENCHMARK(BM_std_min<__int128>)->Apply(run_sizes);
+#endif
 BENCHMARK(BM_std_min<unsigned char>)->Apply(run_sizes);
 BENCHMARK(BM_std_min<unsigned short>)->Apply(run_sizes);
 BENCHMARK(BM_std_min<unsigned int>)->Apply(run_sizes);
diff --git a/libcxx/test/benchmarks/algorithms/min_max_element.bench.cpp b/libcxx/test/benchmarks/algorithms/min_max_element.bench.cpp
index e2c64236561049..4dbfed66d0c2cf 100644
--- a/libcxx/test/benchmarks/algorithms/min_max_element.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/min_max_element.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14
+
 #include <algorithm>
 
 #include "common.h"
diff --git a/libcxx/test/benchmarks/algorithms/minmax.bench.cpp b/libcxx/test/benchmarks/algorithms/minmax.bench.cpp
index ca1cdb441deef7..7d7c74c9c96aa6 100644
--- a/libcxx/test/benchmarks/algorithms/minmax.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/minmax.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
 #include <algorithm>
 #include <cassert>
 
diff --git a/libcxx/test/benchmarks/algorithms/mismatch.bench.cpp b/libcxx/test/benchmarks/algorithms/mismatch.bench.cpp
index 791782879011e2..348009a230d6e1 100644
--- a/libcxx/test/benchmarks/algorithms/mismatch.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/mismatch.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14
+
 #include <algorithm>
 #include <benchmark/benchmark.h>
 #include <random>
diff --git a/libcxx/test/benchmarks/algorithms/pop_heap.bench.cpp b/libcxx/test/benchmarks/algorithms/pop_heap.bench.cpp
index 26cdd25cce90d0..5fef52284239dd 100644
--- a/libcxx/test/benchmarks/algorithms/pop_heap.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/pop_heap.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14
+
 #include <algorithm>
 
 #include "common.h"
diff --git a/libcxx/test/benchmarks/algorithms/pstl.stable_sort.bench.cpp b/libcxx/test/benchmarks/algorithms/pstl.stable_sort.bench.cpp
index 72541f70640f53..10254ac12cf56f 100644
--- a/libcxx/test/benchmarks/algorithms/pstl.stable_sort.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/pstl.stable_sort.bench.cpp
@@ -6,6 +6,9 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14
+// UNSUPPORTED: libcpp-has-no-incomplete-pstl
+
 #include <algorithm>
 #include <execution>
 
diff --git a/libcxx/test/benchmarks/algorithms/push_heap.bench.cpp b/libcxx/test/benchmarks/algorithms/push_heap.bench.cpp
index ba96fa1469e604..89d8122bd1dbe9 100644
--- a/libcxx/test/benchmarks/algorithms/push_heap.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/push_heap.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14
+
 #include <algorithm>
 
 #include "common.h"
diff --git a/libcxx/test/benchmarks/algorithms/ranges_contains.bench.cpp b/libcxx/test/benchmarks/algorithms/ranges_contains.bench.cpp
index f36ebff9009585..b98e17a00ef830 100644
--- a/libcxx/test/benchmarks/algorithms/ranges_contains.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/ranges_contains.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14, c++17, c++20
+
 #include <algorithm>
 #include <benchmark/benchmark.h>
 #include <iterator>
diff --git a/libcxx/test/benchmarks/algorithms/ranges_ends_with.bench.cpp b/libcxx/test/benchmarks/algorithms/ranges_ends_with.bench.cpp
index 049af7c2e15a35..c975d164c16d45 100644
--- a/libcxx/test/benchmarks/algorithms/ranges_ends_with.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/ranges_ends_with.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14, c++17, c++20
+
 #include <algorithm>
 #include <benchmark/benchmark.h>
 #include <iterator>
diff --git a/libcxx/test/benchmarks/algorithms/ranges_make_heap.bench.cpp b/libcxx/test/benchmarks/algorithms/ranges_make_heap.bench.cpp
index 66a8335a8710c2..c04ea369fea0c9 100644
--- a/libcxx/test/benchmarks/algorithms/ranges_make_heap.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/ranges_make_heap.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
 #include <algorithm>
 
 #include "common.h"
diff --git a/libcxx/test/benchmarks/algorithms/ranges_make_heap_then_sort_heap.bench.cpp b/libcxx/test/benchmarks/algorithms/ranges_make_heap_then_sort_heap.bench.cpp
index 01632c807c178d..b84d3c2cea765e 100644
--- a/libcxx/test/benchmarks/algorithms/ranges_make_heap_then_sort_heap.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/ranges_make_heap_then_sort_heap.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
 #include <algorithm>
 
 #include "common.h"
diff --git a/libcxx/test/benchmarks/algorithms/ranges_pop_heap.bench.cpp b/libcxx/test/benchmarks/algorithms/ranges_pop_heap.bench.cpp
index bcc7a834a655ac..ab3ae6f7c30ae0 100644
--- a/libcxx/test/benchmarks/algorithms/ranges_pop_heap.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/ranges_pop_heap.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
 #include <algorithm>
 
 #include "common.h"
diff --git a/libcxx/test/benchmarks/algorithms/ranges_push_heap.bench.cpp b/libcxx/test/benchmarks/algorithms/ranges_push_heap.bench.cpp
index 902f481228e466..8139ba32cb9747 100644
--- a/libcxx/test/benchmarks/algorithms/ranges_push_heap.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/ranges_push_heap.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
 #include <algorithm>
 
 #include "common.h"
diff --git a/libcxx/test/benchmarks/algorithms/ranges_sort.bench.cpp b/libcxx/test/benchmarks/algorithms/ranges_sort.bench.cpp
index aeb2aedd34af33..d145a159a21fd3 100644
--- a/libcxx/test/benchmarks/algorithms/ranges_sort.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/ranges_sort.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
 #include <algorithm>
 
 #include "common.h"
diff --git a/libcxx/test/benchmarks/algorithms/ranges_sort_heap.bench.cpp b/libcxx/test/benchmarks/algorithms/ranges_sort_heap.bench.cpp
index 62c607cf7a5ed6..90e9b9a6933f5f 100644
--- a/libcxx/test/benchmarks/algorithms/ranges_sort_heap.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/ranges_sort_heap.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
 #include <algorithm>
 
 #include "common.h"
diff --git a/libcxx/test/benchmarks/algorithms/ranges_stable_sort.bench.cpp b/libcxx/test/benchmarks/algorithms/ranges_stable_sort.bench.cpp
index 88327483cf43c3..acc2f3f755fb86 100644
--- a/libcxx/test/benchmarks/algorithms/ranges_stable_sort.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/ranges_stable_sort.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14, c++17
+
 #include <algorithm>
 
 #include "common.h"
diff --git a/libcxx/test/benchmarks/algorithms/set_intersection.bench.cpp b/libcxx/test/benchmarks/algorithms/set_intersection.bench.cpp
index b3fb15fc77b318..9bde4bb29dc22f 100644
--- a/libcxx/test/benchmarks/algorithms/set_intersection.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/set_intersection.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14
+
 #include <algorithm>
 #include <cstdlib>
 #include <iterator>
diff --git a/libcxx/test/benchmarks/algorithms/sort.bench.cpp b/libcxx/test/benchmarks/algorithms/sort.bench.cpp
index f87434b7797124..899272e34795fa 100644
--- a/libcxx/test/benchmarks/algorithms/sort.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/sort.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14
+
 #include <algorithm>
 
 #include "common.h"
diff --git a/libcxx/test/benchmarks/algorithms/sort_heap.bench.cpp b/libcxx/test/benchmarks/algorithms/sort_heap.bench.cpp
index 1372b4d5c9ab7c..ee4b6bfc7387bf 100644
--- a/libcxx/test/benchmarks/algorithms/sort_heap.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/sort_heap.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===----------------------------------------------------------------------===//
 
+// UNSUPPORTED: c++03, c++11, c++14
+
 #include <algorithm>
 
 #include "common.h"
diff --git a/libcxx/test/benchmarks/algorithms/stable_sort.bench.cpp b/libcxx/test/benchmarks/algorithms/stable_sort.bench.cpp
index 024a036e0c2582..c68f73838c3199 100644
--- a/libcxx/test/benchmarks/algorithms/stable_sort.bench.cpp
+++ b/libcxx/test/benchmarks/algorithms/stable_sort.bench.cpp
@@ -6,6 +6,8 @@
 //
 //===------------------------------------------------------------------...
[truncated]

@@ -181,7 +184,7 @@ struct IterateRangeFor : Base {
while (State.KeepRunningBatch(TableSize * NumTables)) {
for (auto& Set : Data.Sets) {
for (auto& V : Set) {
benchmark::DoNotOptimize(V);
benchmark::DoNotOptimize(const_cast<std::set<uint64_t>::reference>(V));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did this change?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here as above: error: 'DoNotOptimize<unsigned long long>' is deprecated: The const-ref version of this method can permit undesired compiler optimizations in benchmarks [-Werror,-Wdeprecated-declarations]

@ldionne ldionne force-pushed the review/benchmarks-make-forward-compatible branch 2 times, most recently from 64583c4 to c5853fa Compare November 1, 2024 13:43
This patch fixes a few warnings and errors when running the benchmarks
as part of the test suite. It also adds most of the necessary Lit
annotations to make it pass on various configurations.
@ldionne ldionne force-pushed the review/benchmarks-make-forward-compatible branch from c5853fa to 710b83a Compare November 4, 2024 15:42
Copy link
Contributor

@philnik777 philnik777 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM as a first step. I think we'll want to do some more cleanup later to make them more portable though. @StephanTLavavej do you have any interest in sharing benchmarks across libc++/the STL?

@StephanTLavavej
Copy link
Member

Sure, that’s interesting! We’ve been building up a set of benchmarks using Google Benchmark.

@ldionne ldionne merged commit b2d2494 into llvm:main Nov 5, 2024
64 checks passed
@ldionne ldionne deleted the review/benchmarks-make-forward-compatible branch November 5, 2024 14:08
PhilippRados pushed a commit to PhilippRados/llvm-project that referenced this pull request Nov 6, 2024
…#114502)

This patch fixes warnings and errors that come up when running the
benchmarks as part of the test suite. It also adds the necessary Lit
annotations to make it pass in various configurations and increases the
portability of the benchmarks.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
libc++ libc++ C++ Standard Library. Not GNU libstdc++. Not libc++abi.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants