[libc++] Remove atomic_wait benchmarks #158289
Open
+0
−609
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The atomic_wait benchmarks are great, but in fact they are a bit too great. Indeed, they are so thorough that they overload the system they're running on, meaning they can't be run on our CI infrastructure. The NxN benchmarks even crash my system on macOS. This patch removes the benchmarks since we're trying to move towards more continuous benchmarking, which requires making a tradeoff between exhaustiveness and ability to run the benchmarks on a regular basis. Instead, it would be better to rework the benchmarks to make them more lightweight, but I think it is reasonable to do that at a later time in order to unblock continuous performance monitoring ASAP.
|
@llvm/pr-subscribers-libcxx Author: Louis Dionne (ldionne) ChangesThe atomic_wait benchmarks are great, but in fact they are a bit too great. Indeed, they are so thorough that they overload the system they're running on, meaning they can't be run on our CI infrastructure. The NxN benchmarks even crash my system on macOS. This patch removes the benchmarks since we're trying to move towards more continuous benchmarking, which requires making a tradeoff between exhaustiveness and ability to run the benchmarks on a regular basis. Instead, it would be better to rework the benchmarks to make them more lightweight, but I think it is reasonable to do that at a later time in order to unblock continuous performance monitoring ASAP. Patch is 22.71 KiB, truncated to 20.00 KiB below, full version: https://github.com/llvm/llvm-project/pull/158289.diff 5 Files Affected:
diff --git a/libcxx/test/benchmarks/atomic_wait_1_waiter_1_notifier.bench.cpp b/libcxx/test/benchmarks/atomic_wait_1_waiter_1_notifier.bench.cpp
deleted file mode 100644
index c3d7e6511925d..0000000000000
--- a/libcxx/test/benchmarks/atomic_wait_1_waiter_1_notifier.bench.cpp
+++ /dev/null
@@ -1,74 +0,0 @@
-//===----------------------------------------------------------------------===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-
-// UNSUPPORTED: c++03, c++11, c++14, c++17
-
-#include "atomic_wait_helper.h"
-
-#include <atomic>
-#include <array>
-#include <chrono>
-#include <cstdint>
-#include <numeric>
-#include <stop_token>
-#include <thread>
-
-#include "benchmark/benchmark.h"
-#include "make_test_thread.h"
-
-using namespace std::chrono_literals;
-
-template <class NotifyPolicy, class NumPrioTasks>
-void BM_1_atomic_1_waiter_1_notifier(benchmark::State& state) {
- [[maybe_unused]] std::array<HighPrioTask, NumPrioTasks::value> tasks{};
- std::atomic<std::uint64_t> a;
- auto thread_func = [&](std::stop_token st) { NotifyPolicy::notify(a, st); };
-
- std::uint64_t total_loop_test_param = state.range(0);
-
- auto thread = support::make_test_jthread(thread_func);
-
- for (auto _ : state) {
- for (std::uint64_t i = 0; i < total_loop_test_param; ++i) {
- auto old = a.load(std::memory_order_relaxed);
- a.wait(old);
- }
- }
-}
-
-BENCHMARK(BM_1_atomic_1_waiter_1_notifier<KeepNotifying, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 16, 1 << 18);
-BENCHMARK(BM_1_atomic_1_waiter_1_notifier<NotifyEveryNus<50>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 10, 1 << 12);
-BENCHMARK(BM_1_atomic_1_waiter_1_notifier<NotifyEveryNus<100>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 10, 1 << 12);
-
-BENCHMARK(BM_1_atomic_1_waiter_1_notifier<KeepNotifying, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 16, 1 << 18);
-BENCHMARK(BM_1_atomic_1_waiter_1_notifier<NotifyEveryNus<50>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 10, 1 << 12);
-BENCHMARK(BM_1_atomic_1_waiter_1_notifier<NotifyEveryNus<100>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 10, 1 << 12);
-
-BENCHMARK(BM_1_atomic_1_waiter_1_notifier<KeepNotifying, NumHighPrioTasks<7>>)
- ->RangeMultiplier(2)
- ->Range(1 << 4, 1 << 6);
-BENCHMARK(BM_1_atomic_1_waiter_1_notifier<NotifyEveryNus<50>, NumHighPrioTasks<7>>)
- ->RangeMultiplier(2)
- ->Range(1 << 3, 1 << 5);
-BENCHMARK(BM_1_atomic_1_waiter_1_notifier<NotifyEveryNus<100>, NumHighPrioTasks<7>>)
- ->RangeMultiplier(2)
- ->Range(1 << 3, 1 << 5);
-
-BENCHMARK_MAIN();
diff --git a/libcxx/test/benchmarks/atomic_wait_N_waiter_N_notifier.bench.cpp b/libcxx/test/benchmarks/atomic_wait_N_waiter_N_notifier.bench.cpp
deleted file mode 100644
index d9b9aa212f602..0000000000000
--- a/libcxx/test/benchmarks/atomic_wait_N_waiter_N_notifier.bench.cpp
+++ /dev/null
@@ -1,167 +0,0 @@
-//===----------------------------------------------------------------------===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-
-// UNSUPPORTED: c++03, c++11, c++14, c++17
-
-#include "atomic_wait_helper.h"
-
-#include <atomic>
-#include <cstdint>
-#include <numeric>
-#include <stop_token>
-#include <pthread.h>
-#include <sched.h>
-#include <thread>
-#include <chrono>
-#include <array>
-
-#include "benchmark/benchmark.h"
-#include "make_test_thread.h"
-
-using namespace std::chrono_literals;
-
-template <class NotifyPolicy, class NumberOfAtomics, class NumPrioTasks>
-void BM_N_atomics_N_waiter_N_notifier(benchmark::State& state) {
- [[maybe_unused]] std::array<HighPrioTask, NumPrioTasks::value> tasks{};
- const std::uint64_t total_loop_test_param = state.range(0);
- constexpr std::uint64_t num_atomics = NumberOfAtomics::value;
- std::vector<std::atomic<std::uint64_t>> atomics(num_atomics);
-
- auto notify_func = [&](std::stop_token st, size_t idx) {
- while (!st.stop_requested()) {
- NotifyPolicy::notify(atomics[idx], st);
- }
- };
-
- std::atomic<std::uint64_t> start_flag = 0;
- std::atomic<std::uint64_t> done_count = 0;
-
- auto wait_func = [&, total_loop_test_param](std::stop_token st, size_t idx) {
- auto old_start = 0;
- while (!st.stop_requested()) {
- start_flag.wait(old_start);
- old_start = start_flag.load();
- for (std::uint64_t i = 0; i < total_loop_test_param; ++i) {
- auto old = atomics[idx].load(std::memory_order_relaxed);
- atomics[idx].wait(old);
- }
- done_count.fetch_add(1);
- }
- };
-
- std::vector<std::jthread> notify_threads;
- notify_threads.reserve(num_atomics);
-
- std::vector<std::jthread> wait_threads;
- wait_threads.reserve(num_atomics);
-
- for (size_t i = 0; i < num_atomics; ++i) {
- notify_threads.emplace_back(support::make_test_jthread(notify_func, i));
- }
-
- for (size_t i = 0; i < num_atomics; ++i) {
- wait_threads.emplace_back(support::make_test_jthread(wait_func, i));
- }
-
- for (auto _ : state) {
- done_count = 0;
- start_flag.fetch_add(1);
- start_flag.notify_all();
- while (done_count < num_atomics) {
- std::this_thread::yield();
- }
- }
- for (auto& t : wait_threads) {
- t.request_stop();
- }
- start_flag.fetch_add(1);
- start_flag.notify_all();
- for (auto& t : wait_threads) {
- t.join();
- }
-}
-
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<KeepNotifying, NumberOfAtomics<2>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 12, 1 << 14);
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<KeepNotifying, NumberOfAtomics<3>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 10, 1 << 12);
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<KeepNotifying, NumberOfAtomics<5>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 10, 1 << 12);
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<KeepNotifying, NumberOfAtomics<7>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 8, 1 << 10);
-
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<NotifyEveryNus<50>, NumberOfAtomics<2>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 10, 1 << 12);
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<NotifyEveryNus<50>, NumberOfAtomics<3>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 8, 1 << 10);
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<NotifyEveryNus<50>, NumberOfAtomics<5>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 8, 1 << 10);
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<NotifyEveryNus<50>, NumberOfAtomics<7>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 6, 1 << 8);
-
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<NotifyEveryNus<100>, NumberOfAtomics<2>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 8, 1 << 10);
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<NotifyEveryNus<100>, NumberOfAtomics<3>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 8, 1 << 10);
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<NotifyEveryNus<100>, NumberOfAtomics<5>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 7, 1 << 9);
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<NotifyEveryNus<100>, NumberOfAtomics<7>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 6, 1 << 8);
-
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<KeepNotifying, NumberOfAtomics<2>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 7, 1 << 9);
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<KeepNotifying, NumberOfAtomics<3>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 7, 1 << 9);
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<KeepNotifying, NumberOfAtomics<5>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 6, 1 << 8);
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<KeepNotifying, NumberOfAtomics<7>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 4, 1 << 6);
-
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<NotifyEveryNus<50>, NumberOfAtomics<2>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 7, 1 << 9);
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<NotifyEveryNus<50>, NumberOfAtomics<3>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 7, 1 << 9);
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<NotifyEveryNus<50>, NumberOfAtomics<5>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 5, 1 << 7);
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<NotifyEveryNus<50>, NumberOfAtomics<7>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 3, 1 << 5);
-
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<NotifyEveryNus<100>, NumberOfAtomics<2>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 6, 1 << 8);
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<NotifyEveryNus<100>, NumberOfAtomics<3>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 6, 1 << 8);
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<NotifyEveryNus<100>, NumberOfAtomics<5>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 5, 1 << 7);
-BENCHMARK(BM_N_atomics_N_waiter_N_notifier<NotifyEveryNus<100>, NumberOfAtomics<7>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 3, 1 << 5);
-
-BENCHMARK_MAIN();
diff --git a/libcxx/test/benchmarks/atomic_wait_helper.h b/libcxx/test/benchmarks/atomic_wait_helper.h
deleted file mode 100644
index cfdacf9e01688..0000000000000
--- a/libcxx/test/benchmarks/atomic_wait_helper.h
+++ /dev/null
@@ -1,92 +0,0 @@
-//===----------------------------------------------------------------------===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-
-#ifndef TEST_BENCHMARK_ATOMIC_WAIT_HELPER_H
-#define TEST_BENCHMARK_ATOMIC_WAIT_HELPER_H
-
-#include <atomic>
-#include <chrono>
-#include <exception>
-#include <stop_token>
-#include <thread>
-
-struct HighPrioTask {
- sched_param param;
- pthread_attr_t attr_t;
- pthread_t thread;
- std::atomic_bool stopped{false};
-
- HighPrioTask(const HighPrioTask&) = delete;
-
- HighPrioTask() {
- pthread_attr_init(&attr_t);
- pthread_attr_setschedpolicy(&attr_t, SCHED_FIFO);
- param.sched_priority = sched_get_priority_max(SCHED_FIFO);
- pthread_attr_setschedparam(&attr_t, ¶m);
- pthread_attr_setinheritsched(&attr_t, PTHREAD_EXPLICIT_SCHED);
-
- auto thread_fun = [](void* arg) -> void* {
- auto* stop = reinterpret_cast<std::atomic_bool*>(arg);
- while (!stop->load(std::memory_order_relaxed)) {
- // spin
- }
- return nullptr;
- };
-
- if (pthread_create(&thread, &attr_t, thread_fun, &stopped) != 0) {
- throw std::runtime_error("failed to create thread");
- }
- }
-
- ~HighPrioTask() {
- stopped = true;
- pthread_attr_destroy(&attr_t);
- pthread_join(thread, nullptr);
- }
-};
-
-template <std::size_t N>
-struct NumHighPrioTasks {
- static constexpr auto value = N;
-};
-
-template <std::size_t N>
-struct NumWaitingThreads {
- static constexpr auto value = N;
-};
-
-template <std::size_t N>
-struct NumberOfAtomics {
- static constexpr auto value = N;
-};
-
-struct KeepNotifying {
- template <class Atomic>
- static void notify(Atomic& a, std::stop_token st) {
- while (!st.stop_requested()) {
- a.fetch_add(1, std::memory_order_relaxed);
- a.notify_all();
- }
- }
-};
-
-template <std::size_t N>
-struct NotifyEveryNus {
- template <class Atomic>
- static void notify(Atomic& a, std::stop_token st) {
- while (!st.stop_requested()) {
- auto start = std::chrono::system_clock::now();
- a.fetch_add(1, std::memory_order_relaxed);
- a.notify_all();
- while (std::chrono::system_clock::now() - start < std::chrono::microseconds{N}) {
- }
- }
- }
-};
-
-#endif // TEST_BENCHMARK_ATOMIC_WAIT_HELPER_H
\ No newline at end of file
diff --git a/libcxx/test/benchmarks/atomic_wait_multi_waiter_1_notifier.bench.cpp b/libcxx/test/benchmarks/atomic_wait_multi_waiter_1_notifier.bench.cpp
deleted file mode 100644
index a14a6a2ad9c98..0000000000000
--- a/libcxx/test/benchmarks/atomic_wait_multi_waiter_1_notifier.bench.cpp
+++ /dev/null
@@ -1,167 +0,0 @@
-//===----------------------------------------------------------------------===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-
-// UNSUPPORTED: c++03, c++11, c++14, c++17
-
-#include "atomic_wait_helper.h"
-
-#include <atomic>
-#include <cstdint>
-#include <numeric>
-#include <stop_token>
-#include <thread>
-#include <chrono>
-#include <array>
-
-#include "benchmark/benchmark.h"
-#include "make_test_thread.h"
-
-using namespace std::chrono_literals;
-
-template <class NotifyPolicy, class NumWaitingThreads, class NumPrioTasks>
-void BM_1_atomic_multi_waiter_1_notifier(benchmark::State& state) {
- [[maybe_unused]] std::array<HighPrioTask, NumPrioTasks::value> tasks{};
-
- std::atomic<std::uint64_t> a;
- auto notify_func = [&](std::stop_token st) { NotifyPolicy::notify(a, st); };
-
- std::uint64_t total_loop_test_param = state.range(0);
- constexpr auto num_waiting_threads = NumWaitingThreads::value;
- std::vector<std::jthread> wait_threads;
- wait_threads.reserve(num_waiting_threads);
-
- auto notify_thread = support::make_test_jthread(notify_func);
-
- std::atomic<std::uint64_t> start_flag = 0;
- std::atomic<std::uint64_t> done_count = 0;
- auto wait_func = [&a, &start_flag, &done_count, total_loop_test_param](std::stop_token st) {
- auto old_start = 0;
- while (!st.stop_requested()) {
- start_flag.wait(old_start);
- old_start = start_flag.load();
- for (std::uint64_t i = 0; i < total_loop_test_param; ++i) {
- auto old = a.load(std::memory_order_relaxed);
- a.wait(old);
- }
- done_count.fetch_add(1);
- }
- };
-
- for (size_t i = 0; i < num_waiting_threads; ++i) {
- wait_threads.emplace_back(support::make_test_jthread(wait_func));
- }
-
- for (auto _ : state) {
- done_count = 0;
- start_flag.fetch_add(1);
- start_flag.notify_all();
- while (done_count < num_waiting_threads) {
- std::this_thread::yield();
- }
- }
- for (auto& t : wait_threads) {
- t.request_stop();
- }
- start_flag.fetch_add(1);
- start_flag.notify_all();
- for (auto& t : wait_threads) {
- t.join();
- }
-}
-
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<KeepNotifying, NumWaitingThreads<3>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 14, 1 << 16);
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<KeepNotifying, NumWaitingThreads<7>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 12, 1 << 14);
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<KeepNotifying, NumWaitingThreads<15>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 10, 1 << 12);
-
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<NotifyEveryNus<50>, NumWaitingThreads<3>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 10, 1 << 12);
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<NotifyEveryNus<50>, NumWaitingThreads<7>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 8, 1 << 10);
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<NotifyEveryNus<50>, NumWaitingThreads<15>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 6, 1 << 8);
-
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<NotifyEveryNus<100>, NumWaitingThreads<3>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 8, 1 << 10);
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<NotifyEveryNus<100>, NumWaitingThreads<7>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 6, 1 << 8);
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<NotifyEveryNus<100>, NumWaitingThreads<15>, NumHighPrioTasks<0>>)
- ->RangeMultiplier(2)
- ->Range(1 << 4, 1 << 6);
-
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<KeepNotifying, NumWaitingThreads<3>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 8, 1 << 10);
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<KeepNotifying, NumWaitingThreads<7>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 6, 1 << 8);
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<KeepNotifying, NumWaitingThreads<15>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 4, 1 << 6);
-
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<NotifyEveryNus<50>, NumWaitingThreads<3>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 8, 1 << 10);
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<NotifyEveryNus<50>, NumWaitingThreads<7>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 6, 1 << 8);
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<NotifyEveryNus<50>, NumWaitingThreads<15>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 4, 1 << 6);
-
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<NotifyEveryNus<100>, NumWaitingThreads<3>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 8, 1 << 10);
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<NotifyEveryNus<100>, NumWaitingThreads<7>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 6, 1 << 8);
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<NotifyEveryNus<100>, NumWaitingThreads<15>, NumHighPrioTasks<4>>)
- ->RangeMultiplier(2)
- ->Range(1 << 4, 1 << 6);
-
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<KeepNotifying, NumWaitingThreads<3>, NumHighPrioTasks<7>>)
- ->RangeMultiplier(2)
- ->Range(1 << 4, 1 << 6);
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<KeepNotifying, NumWaitingThreads<7>, NumHighPrioTasks<7>>)
- ->RangeMultiplier(2)
- ->Range(1 << 3, 1 << 5);
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<KeepNotifying, NumWaitingThreads<15>, NumHighPrioTasks<7>>)
- ->RangeMultiplier(2)
- ->Range(1 << 2, 1 << 4);
-
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<NotifyEveryNus<50>, NumWaitingThreads<3>, NumHighPrioTasks<7>>)
- ->RangeMultiplier(2)
- ->Range(1 << 3, 1 << 5);
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<NotifyEveryNus<50>, NumWaitingThreads<7>, NumHighPrioTasks<7>>)
- ->RangeMultiplier(2)
- ->Range(1 << 2, 1 << 4);
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<NotifyEveryNus<50>, NumWaitingThreads<15>, NumHighPrioTasks<7>>)
- ->RangeMultiplier(2)
- ->Range(1 << 1, 1 << 3);
-
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<NotifyEveryNus<100>, NumWaitingThreads<3>, NumHighPrioTasks<7>>)
- ->RangeMultiplier(2)
- ->Range(1 << 3, 1 << 5);
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<NotifyEveryNus<100>, NumWaitingThreads<7>, NumHighPrioTasks<7>>)
- ->RangeMultiplier(2)
- ->Range(1 << 2, 1 << 4);
-BENCHMARK(BM_1_atomic_multi_waiter_1_notifier<NotifyEveryNus<100>, NumWaitingThreads<15>, NumHighPrioTasks<7>>)
- ->RangeMultiplier(2)
- ->Range(1 << 1, 1 << 3);
-
-BENCHMARK_MAIN();
diff --git a/libcxx/test/benchmarks/atomic_wait_vs_mutex_lock.bench.cpp b/libcxx/test/benchmarks/atomic_wait_vs_mutex_lock.bench.cpp
deleted file mode 100644
index a554c721df017..0000000000000
--- a/libcxx/test/benchmarks/atomic_wait_vs_mutex_lock.bench.cpp
+++ /dev/null
@@ -1,109 +0,0 @@
-//===----------------------------------------------------------------------===//
-//
-// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions.
-// See https://llvm.org/LICENSE.txt for license information.
-// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception
-//
-//===----------------------------------------------------------------------===//
-
-// UNSUPPORTED...
[truncated]
|
|
CC @huixie90 To be clear, removing these benchmarks is just my straw man proposal to unblock more continuous performance benchmarking. If you think you'd be able to tweak the benchmarks to make them lighter while retaining their coverage, we should do that instead. |
|
LGTM. These benchmarks were added because I was actively developing atomic wait. We don’t have to run them in general. I wonder if there are ways to keep them but not running them in usual CI, in case we need to work on it again |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The atomic_wait benchmarks are great, but in fact they are a bit too great. Indeed, they are so thorough that they overload the system they're running on, meaning they can't be run on our CI infrastructure. The NxN benchmarks even crash my system on macOS.
This patch removes the benchmarks since we're trying to move towards more continuous benchmarking, which requires making a tradeoff between exhaustiveness and ability to run the benchmarks on a regular basis.
Instead, it would be better to rework the benchmarks to make them more lightweight, but I think it is reasonable to do that at a later time in order to unblock continuous performance monitoring ASAP.