[close #372] add circuit breaker and smart rawkv client - #358
Conversation
38145a4 to
3b62a5c
Compare
|
/run-all-tests |
| this.listeners = new ArrayList<>(); | ||
| this.currentMetrics = new AtomicReference<>(new SingleWindowMetrics()); | ||
|
|
||
| scheduler = |
There was a problem hiding this comment.
should this background task be managed by the circuit breaker instead of the circuit metrics?
There was a problem hiding this comment.
I think it's a subtask belonging to the circuit breaker. For the metrics, it only needs to record the traffic statistics.
There was a problem hiding this comment.
BTW, we should not start this background thread when the circuit breaker is configured to be disabled.
Signed-off-by: marsishandsome <[email protected]>
f42b506 to
929f3db
Compare
Signed-off-by: marsishandsome <[email protected]>
2eea8ad to
9932dab
Compare
Signed-off-by: marsishandsome <[email protected]>
Signed-off-by: marsishandsome <[email protected]>
Signed-off-by: marsishandsome <[email protected]>
| this.listeners = new ArrayList<>(); | ||
| this.currentMetrics = new AtomicReference<>(new SingleWindowMetrics()); | ||
|
|
||
| scheduler = |
There was a problem hiding this comment.
I think it's a subtask belonging to the circuit breaker. For the metrics, it only needs to record the traffic statistics.
| private final AtomicLong circuitOpened = new AtomicLong(-1); | ||
| private final AtomicReference<Status> status = new AtomicReference<>(Status.CLOSED); | ||
| private final AtomicLong attemptCount = new AtomicLong(0); | ||
| private final AtomicLong attemptSuccessCount = new AtomicLong(0); |
There was a problem hiding this comment.
-
do we really need to set these variables as atomic variables? I can only see they are accessed within the current TiSession.
-
we add the 'final' keyword to these fields but their values are changed during the execution time, seems a little counterintuitive.
There was a problem hiding this comment.
There are two-time windows and their corresponding traffic stats:
- the first time window is to record normal traffic stats when the circuit breaker is closed or disabled in the background and triggers circuit breaker, which is limited to
TiKV_CIRCUIT_BREAK_AVAILABILITY_WINDOW_IN_SECONDSand is called windowOfClosed. - the second time window to record the attempt traffic stats when the circuit breaker is half-opened, which is called windowOfHalfOpened.
these two windows can be posited like this: windowOfClosed -> (circuit breaker opened) -> (after sleep, circuit breaker half-opened) -> windowOfHalfOpened
So can we:
- Define two traffic metrics for these two time windows like the following and remove the atomic variables for attempts:
private final CircuitBreakerMetrics metricsOfClosed;
private final CircuitBreakerMetrics metricsOfHalfOpened- Make
CircuitBreakerImplbe the owner of these two windows, don't start the background task to collectmetricsOfClosedwhen the circuit breaker is disabled.
There was a problem hiding this comment.
do we really need to set these variables as atomic variables?
yes, in order to make CircuitBreakerImpl thread-safe.
There was a problem hiding this comment.
we add the 'final' keyword to these fields but their values are changed during the execution time, seems a little counterintuitive.
yes, we should add final and the object reference does not change. The object content does change.
| this.listeners = new ArrayList<>(); | ||
| this.currentMetrics = new AtomicReference<>(new SingleWindowMetrics()); | ||
|
|
||
| scheduler = |
There was a problem hiding this comment.
BTW, we should not start this background thread when the circuit breaker is configured to be disabled.
Signed-off-by: marsishandsome <[email protected]>
Signed-off-by: ti-srebot <[email protected]>
|
cherry pick to release-3.1 in PR #373 |
Signed-off-by: ti-srebot <[email protected]>
Signed-off-by: ti-srebot <[email protected]> Signed-off-by: marsishandsome <[email protected]>
Signed-off-by: ti-srebot <[email protected]> Signed-off-by: marsishandsome <[email protected]>
Signed-off-by: ti-srebot <[email protected]> Signed-off-by: marsishandsome <[email protected]>
Signed-off-by: ti-srebot <[email protected]> Signed-off-by: marsishandsome <[email protected]>
Signed-off-by: ti-srebot <[email protected]> Signed-off-by: marsishandsome <[email protected]> Co-authored-by: Liangliang Gu <[email protected]>
Signed-off-by: Ankita Wagh <[email protected]>
Signed-off-by: marsishandsome [email protected]
add Circuit Breaker