Stuck at starting bootstrapper L1 (avalanchego-v1.12.1)

[thanhxeon2470]

I run a Fuji L1 node(is olso validator) with id: NodeID-LNFnxhbyESsC4f5J173YVtw4MbdrAe4Zc (validation_id: 2Pjx6J1Tj69KiHjz2osZvXacJtmdk8P9Xy5yz4kSmBZAkvgFBE)
The node has been running for about 1 month, there was an error nonce too low can not validate new block. I tried to restart but it got stuck at this step.

[02-06|08:28:40.003] INFO <VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8 Chain> core/blockchain.go:382
[02-06|08:28:40.004] INFO <VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8 Chain> core/blockchain.go:715 Loaded most recent local header          number=100,657 hash=ecf0b4..755ed4 age=2d21h52m
[02-06|08:28:40.004] INFO <VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8 Chain> core/blockchain.go:716 Loaded most recent local full block      number=100,657 hash=ecf0b4..755ed4 age=2d21h52m
[02-06|08:28:40.004] INFO <VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8 Chain> core/blockchain.go:1803 Loaded Acceptor tip                      hash=ecf0b4..755ed4
[02-06|08:28:40.004] INFO <VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8 Chain> core/blockchain.go:1811 Skipping state reprocessing              root=e3e97f..794dff
[02-06|08:28:40.005] INFO <VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8 Chain> core/blockchain.go:1779 Initializing snapshots                   async=true rebuild=true headHash=ecf0b4..755ed4 headRoot=e3e97f..794dff
[02-06|08:28:40.008] INFO <VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8 Chain> core/blockchain.go:563 Warmed accepted caches                   start=100,626 end=100,657 t=2.640256ms
[02-06|08:28:40.008] INFO <VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8 Chain> core/blockchain.go:569 Starting Acceptor                        "queue length"=64
[02-06|08:28:40.008] WARN <VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8 Chain> internal/shutdowncheck/shutdown_tracker.go:67 Unclean shutdown detected                booted=2025-01-07T06:42:43+0000 age=1mo1h45m
[02-06|08:28:40.008] INFO <VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8 Chain> core/txpool/legacypool/legacypool.go:486 Legacy pool tip threshold updated        tip=0
[02-06|08:28:40.011] DEBUG <VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8 Chain> plugin/evm/block.go:173 Block verification does not require proposerVM context block=2oMKc53kSfaFL337NhmYHGkJhUhhHuLjcfq3VNKdcnRujwsqb8 height=100,657
[02-06|08:28:40.012] INFO <VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8 Chain> proposervm/vm.go:205 initialized proposervm {"state": "after fork", "forkHeight": 1, "lastAcceptedHeight": 100657}
[02-06|08:28:40.012] INFO <VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8 Chain> snowman/engine.go:96 initializing consensus engine
[02-06|08:28:40.013] INFO health/worker.go:115 registered new check and initialized its state to failing {"name": "health", "name": "VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8", "tags": ["2ESaotneW9feByABEHUoka3gjZPAgnjmbiWCYxYw7hAuT3ectL"]}
[02-06|08:28:40.014] INFO <VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8 Chain> plugin/evm/vm.go:1104 Enabled APIs: eth, eth-filter, net, admin, web3, internal-eth, internal-blockchain, internal-transaction, internal-debug, internal-account, internal-personal, debug, debug-tracer, debug-file-tracer, debug-handler, validators, warp
[02-06|08:28:40.014] DEBUG <VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8 Chain> rpc/websocket.go:104 Allowed origin(s) for WS RPC interface [*]
[02-06|08:28:40.016] INFO server/server.go:210 adding route {"url": "/ext/bc/VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8", "endpoint": "/rpc"}
[02-06|08:28:40.016] INFO server/server.go:210 adding route {"url": "/ext/bc/VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8", "endpoint": "/ws"}
[02-06|08:28:40.016] INFO server/server.go:210 adding route {"url": "/ext/bc/VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8", "endpoint": "/validators"}
[02-06|08:28:40.018] INFO <VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8 Chain> bootstrap/bootstrapper.go:183 starting bootstrapper {"lastAcceptedID": "2rKidXc7Eb8bm1yypT9AdXWX4q1XN8KQkmgGiXBMfu9HCFD64v", "lastAcceptedHeight": 100657}
[02-06|08:28:40.019] DEBUG <VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8 Chain> peer/network.go:360 adding new peer                          nodeID=NodeID-LNFnxhbyESsC4f5J173YVtw4MbdrAe4Zc

curl http://127.0.0.1:9652/ext/bc/VhdtacxdFewFKCnCJkyavYWeohf1DjgaQnQAg7H6LKoaCbyp8/rpc \
  -X POST \
  -H "Content-Type: application/json" \
  --data '{"method":"eth_blockNumber","params":[],"id":1,"jsonrpc":"2.0"}'

API call rejected because chain is not done bootstrapping

[thanhxeon2470]

Solved problem.
chains/manager.go:1301

tracker.NewStartup(connectedBeacons, (3*bootstrapWeight+3)/4)

Because I have 3 validators L1, but 2/3 run out of validator balance

Before

avalanche validator list l1
✔ Fuji Testnet
+----------------------------------------------------------------------------------------------------------------------------+
|                                                     L1 VALIDATORS                                                    |
+------------------------------------------+----------------------------------------------------+--------+-------------------+
| NODE ID                                  | VALIDATION ID                                      | WEIGHT | REMAINING BALANCE |
+------------------------------------------+----------------------------------------------------+--------+-------------------+
| NodeID-6yxwjZnS7H5tMpZQpnCtbkFY9ZVcqv22F | 2gzk1SeZY14ZtnXbaDoEMh7g8vps6gDCwyCB2QbewNVZx6uwAj |     20 |       0.099864832 |
+------------------------------------------+----------------------------------------------------+--------+-------------------+
| NodeID-EopocixQp8yC9PZyUScan4WmGkXoYfD8m | Y45Q1fpAS6pBHzCTkhDs7RegyaSzKRyybzogDuaVrCB1Vg7KH  |     20 |               0.1 |
+------------------------------------------+----------------------------------------------------+--------+-------------------+
| NodeID-LNFnxhbyESsC4f5J173YVtw4MbdrAe4Zc | 2Pjx6J1Tj69KiHjz2osZvXacJtmdk8P9Xy5yz4kSmBZAkvgFBE |    100 |         2.7934272 |
+------------------------------------------+----------------------------------------------------+--------+-------------------+

After

avalanche validator list l1
✔️ Fuji Testnet
✗ could not get balance for node NodeID-111111111111111111116DBWJs due to failed to decode client response: fetching L1 validator "11111111111111111111111111111111LpoYY" failed: not found
+----------------------------------------------------------------------------------------------------------------------------+
|                                                     L1 VALIDATORS                                                    |
+------------------------------------------+----------------------------------------------------+--------+-------------------+
| NODE ID                                  | VALIDATION ID                                      | WEIGHT | REMAINING BALANCE |
+------------------------------------------+----------------------------------------------------+--------+-------------------+
| NodeID-111111111111111111116DBWJs        | 11111111111111111111111111111111LpoYY              |     40 |                 0 |
+------------------------------------------+----------------------------------------------------+--------+-------------------+
| NodeID-LNFnxhbyESsC4f5J173YVtw4MbdrAe4Zc | 2Pjx6J1Tj69KiHjz2osZvXacJtmdk8P9Xy5yz4kSmBZAkvgFBE |    100 |       2.655095252 |
+------------------------------------------+----------------------------------------------------+--------+-------------------+

But NewStartup formula is incorrect in this case, so I can't start any nodes.