4|erigon-p | [INFO] [02-21|09:09:12.893] [bor.heimdall] Got new checkpoint start=53764006 end=53764517 rootHash=0xc9b122c51347fa649209555cfb9b3cc72a872b9be9bbe7dd6756fd9fa17f5d90
4|erigon-p | [INFO] [02-21|09:09:13.747] [snapshots:download] Blocks Stat blocks=53549k indices=53549k alloc=9.9GB sys=10.4GB
4|erigon-p | [INFO] [02-21|09:09:13.747] [snapshots:missed-idx] Blocks Stat blocks=0k indices=0k alloc=9.9GB sys=10.4GB
4|erigon-p | [INFO] [02-21|09:09:15.227] [snapshots:missed-idx:reopen] Blocks Stat blocks=53549k indices=53549k alloc=6.6GB sys=11.9GB
4|erigon-p | [INFO] [02-21|09:09:15.227] [2/15 Headers] Waiting for headers... from=53765798
4|erigon-p | [INFO] [02-21|09:09:45.227] [2/15 Headers] No block headers to write in this log period number=53765798 blk/second=0.000 alloc=6.7GB sys=11.9GB invalidHeaders=0 rejectedBadHeaders=0
4|erigon-p | [INFO] [02-21|09:10:15.227] [2/15 Headers] No block headers to write in this log period number=53765798 blk/second=0.000 alloc=6.7GB sys=11.9GB invalidHeaders=0 rejectedBadHeaders=0
4|erigon-p | [INFO] [02-21|09:10:45.227] [2/15 Headers] No block headers to write in this log period number=53765798 blk/second=0.000 alloc=6.8GB sys=11.9GB invalidHeaders=0 rejectedBadHeaders=0
4|erigon-polygon | [INFO] [02-21|09:10:54.913] [bor.heimdall] Got new checkpoint start=53764006 end=53764517 rootHash=0xc9b122c51347fa649209555cfb9b3cc72a872b9be9bbe7dd6756fd9fa17f5d90
4|erigon-polygon | [INFO] [02-21|09:11:15.228] [2/15 Headers] No block headers to write in this log period number=53765798 blk/second=0.000 alloc=5.7GB sys=11.9GB invalidHeaders=0 rejectedBadHeaders=0
4|erigon-polygon | [INFO] [02-21|09:11:45.228] [2/15 Headers] No block headers to write in this log period number=53765798 blk/second=0.000 alloc=5.8GB sys=11.9GB invalidHeaders=0 rejectedBadHeaders=0
4|erigon-polygon | [INFO] [02-21|09:11:45.228] Req/resp stats req=0 reqMin=0 reqMax=0 skel=29 skelMin=53765797 skelMax=53802853 resp=0 respMin=0 respMax=0 dups=0
4|erigon-polygon | [INFO] [02-21|09:11:54.564] [2/15 Headers] Processed highest=53765934 age=23s headers=136 in=159.338 blk/sec=0
4|erigon-polygon | [INFO] [02-21|09:11:54.564] [2/15 Headers] DONE in=2m39.337762614s
4|erigon-polygon | [INFO] [02-21|09:11:54.565] [3/15 BorHeimdall] Processing sync events... from=53765799 to=53765934
4|erigon-polygon | [INFO] [02-21|09:12:00.804] [3/15 BorHeimdall] Sync events processed progress=53765934 lastSpanID=8401 lastStateSyncEventID=2866902 total records=0 fetch time=6.179220466s process time=6.239609367s
4|erigon-polygon | [INFO] [02-21|09:12:00.996] [5/15 Bodies] Processing bodies... from=53765798 to=53765934
4|erigon-polygon | [INFO] [02-21|09:12:09.690] [5/15 Bodies] Processed highest=53765934 blocks=136 in=8.694 blk/sec=15
4|erigon-polygon | [INFO] [02-21|09:12:09.690] [6/15 Senders] Started from=53765798 to=53765934
4|erigon-polygon | [INFO] [02-21|09:12:09.751] [7/15 Execution] Blocks execution from=53765798 to=53765934
4|erigon-polygon | [INFO] [02-21|09:12:11.426] [p2p] GoodPeers eth68=1 eth67=2
4|erigon-polygon | [INFO] [02-21|09:12:12.019] [txpool] stat pending=0 baseFee=0 queued=517 alloc=6.1GB sys=11.9GB
4|erigon-polygon | [INFO] [02-21|09:12:29.848] [7/15 Execution] Completed on block=53765934
4|erigon-polygon | [INFO] [02-21|09:12:29.848] [8/15 HashState] Promoting plain state from=53765798 to=53765934
4|erigon-polygon | [INFO] [02-21|09:12:29.848] [8/15 HashState] Incremental promotion from=53765798 to=53765934 codes=true csbucket=AccountChangeSet
4|erigon-polygon | [INFO] [02-21|09:12:31.595] [8/15 HashState] Incremental promotion from=53765798 to=53765934 codes=false csbucket=AccountChangeSet
4|erigon-polygon | [INFO] [02-21|09:12:33.182] [bor.heimdall] Got new checkpoint start=53764006 end=53764517 rootHash=0xc9b122c51347fa649209555cfb9b3cc72a872b9be9bbe7dd6756fd9fa17f5d90
4|erigon-polygon | [INFO] [02-21|09:12:36.722] [8/15 HashState] Incremental promotion from=53765798 to=53765934 codes=false csbucket=StorageChangeSet
4|erigon-polygon | [INFO] [02-21|09:12:48.586] [9/15 IntermediateHashes] Generating intermediate hashes from=53765798 to=53765934
4|erigon-polygon | [INFO] [02-21|09:13:18.780] [9/15 IntermediateHashes] Calculating Merkle root current key=c788bbd2
4|erigon-polygon | [INFO] [02-21|09:13:33.423] [10/15 CallTraces] Pruned call trace intermediate table from=53675798 to=53675933
4|erigon-polygon | [INFO] [02-21|09:13:57.042] [13/15 LogIndex] processing from=53765799 to=53765934
4|erigon-polygon | [INFO] [02-21|09:14:13.202] [bor.heimdall] Got new checkpoint start=53764006 end=53764517 rootHash=0xc9b122c51347fa649209555cfb9b3cc72a872b9be9bbe7dd6756fd9fa17f5d90
4|erigon-polygon | [INFO] [02-21|09:14:28.432] RPC Daemon notified of new headers from=53765798 to=53765934 hash=0x7189184ab611475f7b85f4cead5b4fb71e587921c4bd2f228348c6dd6042352d header sending=88.019µs log sending=170ns
4|erigon-polygon | [INFO] [02-21|09:14:28.432] Timings (slower than 50ms) Snapshots=3.198s Headers=2m39.337s BorHeimdall=6.239s BlockHashes=191ms Bodies=8.694s Senders=60ms Execution=20.096s HashState=18.737s IntermediateHashes=44.789s CallTraces=7.398s AccountHistoryIndex=4.832s StorageHistoryIndex=11.436s LogIndex=24.609s TxLookup=6.372s
4|erigon-polygon | [INFO] [02-21|09:14:28.432] Tables PlainState=365.9GB AccountChangeSet=335.3GB StorageChangeSet=1.0TB BlockTransaction=28.2GB TransactionLog=4.1TB FreeList=205.3MB ReclaimableSpace=821.2GB
4|erigon-polygon | [INFO] [02-21|09:14:30.252] [snapshots] Prune Blocks to=53549000 limit=100
4|erigon-polygon | [INFO] [02-21|09:14:30.338] [snapshots] Prune Bor Blocks to=53549000 limit=100
4|erigon-polygon | [INFO] [02-21|09:14:39.722] [2/15 Headers] Waiting for headers... from=53765934
4|erigon-polygon | [INFO] [02-21|09:14:41.077] [2/15 Headers] Processed highest=53765994 age=8s headers=60 in=1.356 blk/sec=44
4|erigon-polygon | [INFO] [02-21|09:14:41.083] [3/15 BorHeimdall] Processing sync events... from=53765935 to=53765994
4|erigon-polygon | [INFO] [02-21|09:14:43.084] [3/15 BorHeimdall] Sync events processed progress=53765994 lastSpanID=8401 lastStateSyncEventID=2866905 total records=3 fetch time=1.938431799s process time=2.006490708s
4|erigon-polygon | [INFO] [02-21|09:14:43.194] [5/15 Bodies] Processing bodies... from=53765934 to=53765994
4|erigon-polygon | [INFO] [02-21|09:14:45.785] [5/15 Bodies] Processed highest=53765994 blocks=60 in=2.591 blk/sec=23
4|erigon-polygon | [INFO] [02-21|09:14:45.791] [6/15 Senders] Started from=53765934 to=53765994
4|erigon-polygon | [INFO] [02-21|09:14:45.829] [7/15 Execution] Blocks execution from=53765934 to=53765994
4|erigon-polygon | [INFO] [02-21|09:14:56.089] [snapshots:retire] Blocks Stat blocks=53549k indices=53549k alloc=6.2GB sys=12.3GB
4|erigon-polygon | panic: runtime error: index out of range [136] with length 136
4|erigon-polygon | goroutine 84748 [running]:
4|erigon-polygon | github.com/ledgerwatch/erigon/turbo/snapshotsync/freezeblocks.(*Merger).filesByRange.func1(0x3,
4|erigon-polygon | 0x43f490?)
4|erigon-polygon | github.com/ledgerwatch/erigon/turbo/snapshotsync/freezeblocks/block_snapshots.go:2159 +0x273
4|erigon-polygon | github.com/tidwall/btree.(*Map[...]).nodeScan(0x3180380?, 0x437f25, 0xc0302f5a48?, 0xa)
4|erigon-polygon | github.com/tidwall/[email protected]/map.go:279 +0x95
4|erigon-polygon | github.com/tidwall/btree.(*Map[...]).scan(...)
4|erigon-polygon | github.com/tidwall/[email protected]/map.go:270
4|erigon-polygon | github.com/tidwall/btree.(*Map[...]).Scan(0x3180380?, 0xae?)
4|erigon-polygon | github.com/tidwall/[email protected]/map.go:259 +0x34
4|erigon-polygon | github.com/ledgerwatch/erigon/turbo/snapshotsync/freezeblocks.(*Merger).filesByRange(0x6fc23ac00?, 0xc0012a60a0, 0x330f4a0, 0x3311bb0)
4|erigon-polygon | github.com/ledgerwatch/erigon/turbo/snapshotsync/freezeblocks/block_snapshots.go:2158 +0x239
4|erigon-polygon | github.com/ledgerwatch/erigon/turbo/snapshotsync/freezeblocks.(*Merger).Merge(0xc0302f5e38, {0x312c520, 0xc000a83810}, 0xc000a83810?, {0xc0aa4c2b40, 0x5, 0x0?}, {0xc17f368070, 0x1, 0x1}, ...)
4|erigon-polygon | github.com/ledgerwatch/erigon/turbo/snapshotsync/freezeblocks/block_snapshots.go:2176 +0x1ac
4|erigon-polygon | github.com/ledgerwatch/erigon/turbo/snapshotsync/freezeblocks.(*BlockRetire).retireBlocks(0xc0da7282c0, {0x312c520, 0xc000a83810}, 0x10000051bd4a0?, 0x74d32af5e3a0?, 0x4, 0xc0b2bba9c0, 0xc001290400?)
4|erigon-polygon | github.com/ledgerwatch/erigon/turbo/snapshotsync/freezeblocks/block_snapshots.go:1194 +0x7ce
4|erigon-polygon | github.com/ledgerwatch/erigon/turbo/snapshotsync/freezeblocks.(*BlockRetire).RetireBlocks(0xc0da7282c0, {0x312c520, 0xc000a83810}, 0x0?, 0xc0b7977200?, 0xc0b27fce98?, 0xc000603a00?, 0xc003510f88?)
4|erigon-polygon | github.com/ledgerwatch/erigon/turbo/snapshotsync/freezeblocks/block_snapshots.go:1287 +0x1d4
4|erigon-polygon | github.com/ledgerwatch/erigon/turbo/snapshotsync/freezeblocks.(*BlockRetire).RetireBlocksInBackground.func1()
4|erigon-polygon | github.com/ledgerwatch/erigon/turbo/snapshotsync/freezeblocks/block_snapshots.go:1247 +0xeb
4|erigon-polygon | created by github.com/ledgerwatch/erigon/turbo/snapshotsync/freezeblocks.(*BlockRetire).RetireBlocksInBackground
4|erigon-polygon | github.com/ledgerwatch/erigon/turbo/snapshotsync/freezeblocks/block_snapshots.go:1240 +0x125
4|erigon-polygon | [INFO] [02-21|09:14:57.366] logging to file system log dir=/mnt/backup/polygon/logs file prefix=erigon log level=info json=false
4|erigon-polygon | [INFO] [02-21|09:14:57.366] Build info git_branch=release/2.58 git_tag=v2.58.0 git_commit=9731f7162a5503389eaa65371787d079d9b6f6c4
4|erigon-polygon | [INFO] [02-21|09:14:57.366] Starting Erigon on Bor Mainnet...
4|erigon-polygon | [INFO] [02-21|09:14:57.366] Maximum peer count ETH=100 total=100
4|erigon-polygon | [INFO] [02-21|09:14:57.366] starting HTTP APIs port=28545 APIs=eth,debug,net,trace,web3,erigon,bor,txpool
4|erigon-polygon | [INFO] [02-21|09:14:57.366] torrent verbosity level=WRN
4|erigon-polygon | [INFO] [02-21|09:14:59.469] Set global gas cap cap=50000000
4|erigon-polygon | [INFO] [02-21|09:14:59.470] [Downloader] Running with ipv6-enabled=true ipv4-enabled=true download.rate=16mb upload.rate=4mb
4|erigon-polygon | [INFO] [02-21|09:14:59.475] Opening Database label=chaindata path=/mnt/backup/polygon/chaindata
4|erigon-polygon | [INFO] [02-21|09:14:59.476] [db] open lable=chaindata sizeLimit=10TB pageSize=16384
4|erigon-polygon | [INFO] [02-21|09:14:59.477] Initialised chain configuration config="{ChainID: 137, Homestead: 0, DAO: <nil>, Tangerine Whistle: 0, Spurious Dragon: 0, Byzantium: 0, Constantinople: 0, Petersburg: 0, Istanbul: 3395000, Muir Glacier: 3395000, Berlin: 14750000, London: 23850000, Arrow Glacier: <nil>, Gray Glacier: <nil>, Terminal Total Difficulty: <nil>, Merge Netsplit: <nil>, Shanghai: <nil>, Cancun: <nil>, Prague: <nil>, Engine: bor}" genesis=0xa9c28ce2141b56c474f1dc504bee9b01eb1bd7d1a507580d5519d4437a97de1b
But the problem still occurs, after running for a while it encounters a panic that causes the program to restart
System information
Erigon version:
2.58.0OS & Version: AlmaLinux 9.2
Erigon Command (with flags/config):
/root/erigon-stable/build/bin/erigon --datadir=/mnt/backup/polygon
--ethash.dagdir=/mnt/backup/polygon/ethash
--chain=bor-mainnet
--port=20303
--http.addr=0.0.0.0
--http.port=28545
--authrpc.port=28551
--torrent.port=22069
--p2p.allowed-ports=20303,20304,20305,20306,20307
--private.api.addr=127.0.0.1:29090
--bor.heimdall=https://heimdall-api.polygon.technology
--http
--http.api=eth,debug,net,trace,web3,erigon,bor,txpool
--db.size.limit=10TB
--db.pagesize=16KB
--bodies.cache=4294967296
Expected behaviour
normal synchronization
Actual behaviour
panic: runtime error: index out of range [136] with length 136
Backtrace
I've tried
integration state_stages --unwind=100 --chain=bor-mainnet --datadir=/mnt/backup/polygon --bor.heimdall=https://heimdall-api.polygon.technologyBut the problem still occurs, after running for a while it encounters a panic that causes the program to restart
How do I fix this?