Skip to content

Commit

Permalink
Merge branch 'main' into fix/next-project
Browse files Browse the repository at this point in the history
  • Loading branch information
shimkiv authored Jun 5, 2024
2 parents 2b516e8 + ae4e2bd commit 55f29ba
Show file tree
Hide file tree
Showing 75 changed files with 964 additions and 754 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -143,19 +143,19 @@ where:

:warning: Running a replayer from scratch on a Devnet/Mainnet database can take up to a couple of days. The recommended best practice is to break the replayer into smaller parts by using the checkpoint capabilities of the replayer.

:warning: You must run the replayer using the Mainnet version. You can run it from the Docker image at `gcr.io/o1labs-192920/mina-archive:2.0.0-039296a-bullseye`.
:warning: You must run the replayer using the Mainnet version. You can run it from the Docker image at `minaprotocol/mina-archive:3.0.0-93e0279-bullseye`.

## Missing blocks

The daemon node unavailability can cause the archive node to miss some of the blocks. This recurring missing blocks issue consistently poses challenges. To address this issue, you can reapply missing blocks.

If you uploaded the missing blocks to Google Cloud, the missing blocks can be reapplied from precomputed blocks to preserve chain continuity.

1. To automatically verify and patch missing blocks, use the [download_missing_blocks.sh](https://raw.githubusercontent.com/MinaProtocol/mina/2.0.0berkeley_rc1/src/app/rosetta/download-missing-blocks.sh) script.
1. To automatically verify and patch missing blocks, use the [download-missing-blocks.sh](https://github.com/MinaProtocol/mina/blob/berkeley/scripts/archive/download-missing-blocks.sh) script.

The `download-missing-blocks` script uses `localhost` as the database host so the script assumes that psql is running on localhost on port 5432. Modify `PG_CONN` in `download_missing_block.sh` for your environment.

1. Install the required `mina-archive-blocks` and `mina-missing-blocks-auditor` scripts that are packed in the `gcr.io/o1labs-192920/mina-archive:2.0.0-039296a-bullseye` Docker image.
1. Install the required `mina-archive-blocks` and `mina-missing-blocks-auditor` scripts that are packed in the `minaprotocol/mina-archive:3.0.0-93e0279-bullseye` Docker image.

1. Export the `BLOCKS_BUCKET`:

Expand Down Expand Up @@ -183,7 +183,7 @@ O1labs maintains a Google bucket containing precomputed blocks from Devnet and M
Note: It's important to highlight that precomputed blocks for **Devnet** between heights `2` and `1582` have missing fields or incorrect transaction data. Utilizing these blocks to patch your Devnet archive database will result in failure. For those who rely on precomputed blocks from this bucket, please follow the outlined steps:

1. Download additional blocks from `gs://mina_network_block_data/devnet-extensional-bundle.tar.gz`.
2. Install the necessary `mina-archive-blocks` script contained within the `gcr.io/o1labs-192920/mina-archive:2.0.0-039296a-bullseye` Docker image.
2. Install the necessary `mina-archive-blocks` script contained within the `minaprotocol/mina-archive:3.0.0-93e0279-bullseye` Docker image.
3. Execute mina-archive-blocks to import the extracted blocks from step 1 using the provided command:

```sh
Expand Down
24 changes: 12 additions & 12 deletions docs/berkeley-upgrade/flags-configs.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -12,25 +12,25 @@ keywords:

# Post-Upgrade Flags and Configurations for Mainnet

Please refer to the Berkeley node release notes [here](https://github.com/MinaProtocol/mina/releases/tag/2.0.0).
Please refer to the Berkeley node release notes [here](https://github.com/MinaProtocol/mina/releases/tag/3.0.0).

### Network details

```
Chain ID
5f704cc0c82e0ed70e873f0893d7e06f148524e3f0bdae2afb02e7819a0c24d1
a7351abc7ddf2ea92d1b38cc8e636c271c1dfd2c081c637f62ebc2af34eb7cc1
Git SHA-1
039296a260080ed02d0d0750d185921f030b6c9c
93e02797f72abe2cbf8dfca6f2328e9c8cd76546
Seed List
https://bootnodes.minaprotocol.com/networks/mainnet.txt
https://storage.googleapis.com/mina-seed-lists/mainnet_seeds.txt
Node build
https://github.com/MinaProtocol/mina/releases/tag/2.0.0
https://github.com/MinaProtocol/mina/releases/tag/3.0.0
```

### Block Producers
### Block Producer's

Start your node post-upgrade in Mainnet with the flags and environment variables listed below.

Expand All @@ -42,7 +42,7 @@ mina daemon
--generate-genesis-proof true
--libp2p-keypair <keyfile path>
--log-json
--peer-list-url https://bootnodes.minaprotocol.com/networks/mainnet.txt
--peer-list-url https://storage.googleapis.com/mina-seed-lists/mainnet_seeds.txt
ENVIRONMENT VARIABLES
RAYON_NUM_THREADS=6
Expand All @@ -60,7 +60,7 @@ mina daemon
--file-log-rotations 500
--libp2p-keypair <keyfile path>
--log-json
--peer-list-url https://bootnodes.minaprotocol.com/networks/mainnet.txt
--peer-list-url https://storage.googleapis.com/mina-seed-lists/mainnet_seeds.txt
--run-snark-coordinator <public key>
--snark-worker-fee 0.001
--work-selection [seq|rand]
Expand All @@ -87,8 +87,8 @@ Running an Archive Node involves setting up a non-block-producing node and a Pos
For more information about running archive nodes, see [Archive Node](/node-operators/archive-node).

The PostgreSQL database requires two schemas:
1. The PostgreSQL schema used by the Mina archive database: in the [release notes](https://github.com/MinaProtocol/mina/releases/tag/2.0.0)
2. The PostgreSQL schema extensions to support zkApp commands: in the [release notes](https://github.com/MinaProtocol/mina/releases/tag/2.0.0)
1. The PostgreSQL schema used by the Mina archive database: in the [release notes](https://github.com/MinaProtocol/mina/releases/tag/3.0.0)
2. The PostgreSQL schema extensions to support zkApp commands: in the [release notes](https://github.com/MinaProtocol/mina/releases/tag/3.0.0)

The non-block-producing node must be configured with the following flags:
```
Expand All @@ -100,7 +100,7 @@ mina daemon
--generate-genesis-proof true
--libp2p-keypair <keyfile path>
--log-json
--peer-list-url https://bootnodes.minaprotocol.com/networks/mainnet.txt
--peer-list-url https://storage.googleapis.com/mina-seed-lists/mainnet_seeds.txt
ENVIRONMENT VARIABLES
MINA_LIBP2P_PASS
Expand All @@ -127,7 +127,7 @@ docker run
--name rosetta --rm \
-p 3088:3088 \
--entrypoint '' \
gcr.io/o1labs-192920/mina-rosetta:<POST UPGRADE RELEASE> \
minaprotocol/mina-rosetta:3.0.0-93e0279-bullseye \
/usr/local/bin/mina-rosetta \
--archive-uri "${PG_CONNECTION_STRING}" \
--graphql-uri "${GRAPHQL_URL}" \
Expand Down
30 changes: 22 additions & 8 deletions docs/berkeley-upgrade/requirements.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,16 @@ Please note the following are the hardware requirements for each node type after

## Mina Daemon Requirements

### IP and Port configuration​
### Installation

:::caution

If you have `mina-generate-keypair` installed, you will need to first `sudo apt remove mina-generate-keypair` before installing `mina-mainnet=3.0.0-93e0279`.
The `mina-generate-keypair` binary is now installed as part of the mina-mainnet package.

:::

### IP and Port configuration

**IP:**

Expand All @@ -39,15 +48,20 @@ Mina uses by default the port `8302` which is the default libp2p port.

You can use a different port by setting the `--external-port` flag.

### Generation of libp2p keypair​
### Node Auto-restart

Ensure your nodes are set to restart automatically after a crash. For guidance, refer to the [auto-restart instructions](/node-operators/block-producer-node/connecting-to-the-network#start-a-mina-node-with-auto-restart-flows-using-systemd)

## Seed Peer Requirements

### Generation of libp2p keypair

Each node within the network must possess its own distinct libp2p key pair. Although the same libp2p keys can be reused from before the upgrade, if you need to manually generate new libp2p keys, use the following command:
To ensure connectivity across the network, it is essential that all seed nodes start with the **same** `libp2p` keypair.
This consistency allows other nodes in the network to reliably connect.
Although the same libp2p keys can be reused from before the upgrade, if you need to manually generate new libp2p keys, use the following command:

```
mina libp2p generate-keypair -privkey-path <path-to-the-key-file>
mina libp2p generate-keypair --privkey-path <path-to-the-key-file>
```
Further information on [generating key pairs](/node-operators/generating-a-keypair) on Mina Protocol.

### Node Auto-restart

Ensure your nodes are set to restart automatically after a crash. For guidance, refer to the [auto-restart instructions](/node-operators/block-producer-node/connecting-to-the-network#start-a-mina-node-with-auto-restart-flows-using-systemd)
Further information on [generating key pairs](/node-operators/generating-a-keypair) on Mina Protocol.
12 changes: 6 additions & 6 deletions docs/berkeley-upgrade/upgrade-steps.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,15 @@ Below it's the description in detail of all the upgrade steps and what which nod
- Review the [upgrade readiness checklist](https://docs.google.com/document/d/1rTmJvyaK33dWjJXMOSiUIGgf8z7turxolGHUpVHNxEU/edit#heading=h.2hqz0ixwjk3f) to confirm they have covered the required steps.
- Upgrade their nodes to the 1.4.1 stable version
- Ensure servers are provisioned to run Berkeley nodes, meeting the new hardware requirements
- Upgrade their nodes to the node version [2.0.0](https://github.com/MinaProtocol/mina/releases/tag/2.0.0), with stop-slots, when this version becomes available
- Upgrade their nodes to the node version [3.0.0](https://github.com/MinaProtocol/mina/releases/tag/3.0.0), with stop-slots, when this version becomes available
- Start the archive node initial migration if they run archive nodes and wish to perform the migration in a decentralized manner

**Please note:** a simplified Node Status service will be part of the upgrade tooling and enabled by default in Pre-Upgrade release with the stop-slots ([2.0.0](https://github.com/MinaProtocol/mina/releases/tag/2.0.0)). This feature will allow for a safe upgrade by monitoring the amount of upgraded active stake. Only non-sensitive data will be reported. If operators are not comfortable sharing their node version, they will have the option to disable the node version reports by using the appropriate node flag `--node-stats-type none`
**Please note:** a simplified Node Status service will be part of the upgrade tooling and enabled by default in Pre-Upgrade release with the stop-slots ([3.0.0](https://github.com/MinaProtocol/mina/releases/tag/3.0.0)). This feature will allow for a safe upgrade by monitoring the amount of upgraded active stake. Only non-sensitive data will be reported. If operators are not comfortable sharing their node version, they will have the option to disable the node version reports by using the appropriate node flag `--node-stats-type none`

### Block Producers and SNARK Workers
1. Review the [upgrade readiness checklist](https://docs.google.com/document/d/1rTmJvyaK33dWjJXMOSiUIGgf8z7turxolGHUpVHNxEU).
1. Provision servers that meet the minimum hardware requirements, including the new 32GB RAM requirement and support for _AVX_ and _BMI2_ CPU instructions.
1. Upgrade nodes to node version [2.0.0](https://github.com/MinaProtocol/mina/releases/tag/2.0.0) ([2.0.0](https://github.com/MinaProtocol/mina/releases/tag/2.0.0) has built-in stop slots).
1. Upgrade nodes to node version [3.0.0](https://github.com/MinaProtocol/mina/releases/tag/3.0.0) ([3.0.0](https://github.com/MinaProtocol/mina/releases/tag/3.0.0) has built-in stop slots).

### Archive Node Operators and Rosetta Operators
- Two migration processes will be available to archive node operators: _trustless_ and _trustful_. If the archive node operator wants to perform the _trustless_ migration, they should follow these steps; otherwise, proceed to the Upgrade phase. The _trustful_ migration will rely on o1Labs database exports and Docker images to migrate the archive node database and doesn’t require any actions at this stage.
Expand All @@ -42,15 +42,15 @@ Below it's the description in detail of all the upgrade steps and what which nod
- Perform the initial archive node migration. Since Mainnet is a long-lived network, the initial migration process can take up to 48 hours, depending on your server specification and infrastructure.
- If your Mina Daemon, archive node, or PostgreSQL database runs on different machines, the migration performance will be greatly impacted.
- For more information on the archive node migration process, please refer to the [Archive Migration](/berkeley-upgrade/archive-migration) section.
2. Upgrade all nodes to the latest stable version [2.0.0](https://github.com/MinaProtocol/mina/releases/tag/2.0.0).
2. Upgrade all nodes to the latest stable version [3.0.0](https://github.com/MinaProtocol/mina/releases/tag/3.0.0).
3. Provision servers that meet the minimum hardware requirements, primarily the new 32GB RAM requirement.
4. Upgrade their nodes to the version that includes built-in stop slots before the pre-defined _stop-transaction-slot_.

### Exchanges
1. Make sure to test your system integration with Berkeley's new features. Pay special attention to:
- If you use the **o1labs/client-sdk** library to sign transactions, you should switch to **mina-signer** https://www.npmjs.com/package/mina-signer. **o1labs/client-sdk was deprecated some time ago and will be unusable** once the network has been upgraded. Please review the migration instructions in [Appendix](/berkeley-upgrade/appendix)
- If you use the **o1labs/client-sdk** library to sign transactions, you should switch to **[mina-signer](https://www.npmjs.com/package/mina-signer)**. o1labs/client-sdk was **deprecated** some time ago and will be **unusable** once the network has been upgraded. Please review the migration instructions in [Appendix](/berkeley-upgrade/appendix).
- If you rely on the archive node SQL database tables, please review the schema changes in Appendix 1 of this document.
2. Upgrade all nodes to the latest stable version [2.0.0](https://github.com/MinaProtocol/mina/releases/tag/2.0.0).
2. Upgrade all nodes to the latest stable version [3.0.0](https://github.com/MinaProtocol/mina/releases/tag/3.0.0).
3. Provision servers that meet the minimum hardware requirements, particularly the new 32GB RAM requirement.
4. Upgrade your nodes to the version that includes built-in stop slots before the pre-defined _stop-transaction-slot_.

Expand Down
12 changes: 9 additions & 3 deletions docs/exchange-operators/rosetta/run-with-docker.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ To run Mina Rosetta on Apple silicon, you can use the steps in [Building and run

:::note

If you want to build your own docker image, you can find more details in [Minas Rosetta repository](https://github.com/MinaProtocol/mina/blob/develop/src/app/rosetta/README.md). However, for most users, its not necessary to build your own image in order to interact with the API.
If you want to build your own docker image, you can find more details in [Mina's Rosetta repository](https://github.com/MinaProtocol/mina/blob/develop/src/app/rosetta/README.md). However, for most users, it's not necessary to build your own image in order to interact with the API.

:::

Expand All @@ -43,26 +43,32 @@ The container in `/rosetta` includes three entrypoints, which each run a differe

Run the container with following command (replace the image tag with one from dockerhub that's compatible with the network you are trying to connect to, also replace `--entrypoint` and any environment variable if needed):

```
docker run -it --rm --name rosetta \
--entrypoint=./docker-start.sh \
-p 10101:10101 -p 3085:3085 -p 3086:3086 -p 3087:3087 \
minaprotocol/mina-rosetta:2.0.0-039296a-focal
minaprotocol/mina-rosetta:3.0.0-93e0279-focal
```

You can also create a file with the environment variables and pass it to the docker run command with `--env-file` flag. For example, create a file named `mainnet.env` with the following content:

```
MINA_NETWORK=mainnet
PEERS_LIST_URL=https://bootnodes.minaprotocol.com/networks/mainnet.txt
PEERS_LIST_URL=https://storage.googleapis.com/mina-seed-lists/mainnet_seeds.txt
MINA_ARCHIVE_DUMP_URL=https://storage.googleapis.com/mina-archive-dumps
MINA_GENESIS_LEDGER_URL=http://673156464838-mina-genesis-ledgers.s3-website-us-west-2.amazonaws.com/mainnet/genesis_ledger.json
BLOCKS_BUCKET=https://storage.googleapis.com/mina_network_block_data
```

Then run the container with the following command:

```
docker run -it --rm --name rosetta \
--entrypoint=./docker-start.sh \
-p 8302:8302 -p 3085:3085 -p 3086:3086 -p 3087:3087 \
--env-file mainnet.env \
minaprotocol/mina-rosetta:<tag>
```

Example environment files for public networks can be found in [this](https://github.com/MinaProtocol/mina/blob/develop/src/app/rosetta/scripts) directory of Mina's Rosetta repository.

Expand Down
8 changes: 4 additions & 4 deletions docs/mina-protocol/sending-a-payment.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -167,19 +167,19 @@ Once you feel comfortable with the basics of creating an address, and sending &

Sometimes you may wish to send many transactions: for example, to payout rewards to those delegating to you if you're running a staking pool.

All information here is relevant as of the 2.0.0 build:
All information here is relevant as of the 3.0.0 build:

### Rate limiting

Currently, nodes on the network will rate limit receiving messages from a given node. As of the 2.0.0 build, your node will also follow this rate limit when sending transactions. Specifically, the limit is currently set at 10 transactions every 15 seconds computed over a 5 minute window. If you attempt to send transactions faster than this rate, your node will queue them up and flush them as older transactions expire from the window upon which the rate limit is computed. You do not need to throttle sending these transactions yourself.
Currently, nodes on the network will rate limit receiving messages from a given node. As of the 3.0.0 build, your node will also follow this rate limit when sending transactions. Specifically, the limit is currently set at 10 transactions every 15 seconds computed over a 5 minute window. If you attempt to send transactions faster than this rate, your node will queue them up and flush them as older transactions expire from the window upon which the rate limit is computed. You do not need to throttle sending these transactions yourself.

Note that older releases of the mina daemon do not perform this rate limiting; if you are running an older version, you should manually limit the number of transactions. Due to overheads from rebroadcasting transactions, we do not recommend sending more than 50 transactions every 5 minutes if you need to manually rate limit.

### My node crashed or disconnected before I could finish sending transactions

The Mina daemon does _not_ currently persist the transaction pool. This means that the transactions that your node will be unaware of any transactions that you've sent so far if your node crashes in the middle of this process. As of the 2.0.0 build, you can resend all transactions (exactly in the same manner as before) and they will be rebroadcasted on the network.
The Mina daemon does _not_ currently persist the transaction pool. This means that the transactions that your node will be unaware of any transactions that you've sent so far if your node crashes in the middle of this process. As of the 3.0.0 build, you can resend all transactions (exactly in the same manner as before) and they will be rebroadcasted on the network.

If you believe you were temporarily disconnected from the network, but your node stayed online (i.e. the gossip network may have missed one or more of your transactions), as of the 2.0.0 build, you can resend any of the transactions locally and they will be broadcasted again to the network even if your node thinks they've already been shared.
If you believe you were temporarily disconnected from the network, but your node stayed online (i.e. the gossip network may have missed one or more of your transactions), as of the 3.0.0 build, you can resend any of the transactions locally and they will be broadcasted again to the network even if your node thinks they've already been shared.

### Cancelling a transaction and setting a new fee

Expand Down
2 changes: 1 addition & 1 deletion docs/node-developers/sandbox-node.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ docker run \
--name mina \
-e RUN_DEMO=true \
-e MINA_PRIVKEY_PASS='' \
minaprotocol/mina-daemon:2.0.0-039296a-bullseye-mainnet
minaprotocol/mina-daemon:3.0.0-93e0279-bullseye-mainnet
```

Expand Down
Loading

0 comments on commit 55f29ba

Please sign in to comment.