Hey @Frozen, during the testing I found a bug, more details below.

Result: signPower wasn't updating after consensus crash, last successful value is return
Expected result: signPower should be updated despite the consensus loss

How to reproduce:

1. Start localnet in make debug or make debug-external
2. Your sign power will be at the 68% which is enough for consensus and more than 66.6%
3. Run curl against leader or check the watchdog UI:

curl -X POST -H "Content-Type: application/json" -d '{"jsonrpc": "2.0", "id": 1, "method": "hmy_getLastSigningPower","params": []}' 127.0.0.1:9506
{"jsonrpc":"2.0","id":1,"result":0.68}

4. Kill one of the validators, e.x. on the 9000 port via:

ps -ef | grep harmony | grep 9000
kill PID

5. Consensus will be broken after that, but if you run again curl or check UI in wd you will still get an old 68 signPower, which wasn't updated after crash.

curl -X POST -H "Content-Type: application/json" -d '{"jsonrpc": "2.0", "id": 1, "method": "hmy_getLastSigningPower","params": []}' 127.0.0.1:9506
{"jsonrpc":"2.0","id":1,"result":0.68}

Thanks for the fix, @Frozen

I've tested the following scenario:

1. Spin up localnet
   Result: Sing power == 68%
2. Kill any internal node to loose consensus, e.x. current leader.
3. Check the signing power via hmy_getLastSigningPower
   Result: signing power was 60%

curl -X POST -H "Content-Type: application/json" -d '{"jsonrpc": "2.0", "id": 1, "method": "hmy_getLastSigningPower","params": []}' 127.0.0.1:9610
{"jsonrpc":"2.0","id":1,"result":0.6}

6. Restart killed node, wait until it rejoin consensus.
   Result: Sing power is again == 68%, consensus has been restored

curl -X POST -H "Content-Type: application/json" -d '{"jsonrpc": "2.0", "id": 1, "method": "hmy_getLastSigningPower","params": []}' 127.0.0.1:9600
{"jsonrpc":"2.0","id":1,"result":0.68}

rn, blockchain couldn't spin up in RPC/Rosseta tests due to:

{
    "level": "error",
    "port": "9000",
    "ip": "127.0.0.1",
    "error": "cannot finalize block: Timeout waiting for commit sigs for reward calculation",
    "caller": "/go/src/github.com/harmony-one/harmony/consensus/consensus_block_proposing.go:236",
    "time": "2024-09-06T09:40:39.249311887Z",
    "message": "[ProposeNewBlock] Failed finalizing the new block"
}

@Frozen you latest rebase/conflict fix broke consensus on localnet. Please check

@Frozen you latest rebase/conflict fix broke consensus on localnet. Please check

Rebase on latest dev broke consensus. Fixed

Thanks for the fix, @Frozen

I've tested the following scenario:

1. Spin up localnet
   **Result:** Sing power == 68%

2. Kill any internal node to loose consensus, e.x. current leader.

3. Check the signing power via `hmy_getLastSigningPower`
   **Result:** signing power was 60%

curl -X POST -H "Content-Type: application/json" -d '{"jsonrpc": "2.0", "id": 1, "method": "hmy_getLastSigningPower","params": []}' 127.0.0.1:9610
{"jsonrpc":"2.0","id":1,"result":0.6}

6. Restart killed node, wait until it rejoin consensus.
   **Result:** Sing power is again == 68%, consensus has been restored

curl -X POST -H "Content-Type: application/json" -d '{"jsonrpc": "2.0", "id": 1, "method": "hmy_getLastSigningPower","params": []}' 127.0.0.1:9600
{"jsonrpc":"2.0","id":1,"result":0.68}

Rechecked the same scenario as mentioned above on both shards with external validator:

• signPower dropped bellow 67, e.x. 54
• restored one of the validators
• signPower went to the > 67 and new blocks have started to be created

Result: PASSED ✔️

@Frozen please fix rebase and fix conflict with latest dev

Rebased

just finish testing this one with the latest rebase. All good for me. Good job @Frozen