imp(docs): introductory doc and configuration guidance (#8)

README.md

@@ -1,5 +1,112 @@
 # Ethereum Private Network with Kurtosis
 
-This repository contains the necessary files to deploy a private Ethereum network using Kurtosis with the Ethereum Package.
-
 [![Tests](https://github.com/gustavovalverde/eth-private-network/actions/workflows/tests-k8s-run.yaml/badge.svg)](https://github.com/gustavovalverde/eth-private-network/actions/workflows/tests-k8s-run.yaml)
+
+This repository contains the necessary files to deploy a private Ethereum network using [Kurtosis](https://www.kurtosis.com/) with the [Ethereum Package](https://github.com/ethpandaops/ethereum-package).
+
+## TLDR to test this
+
+### Prerequisites
+
+- [Kurtosis CLI](https://docs.kurtosis.com/install/)
+- [Docker](https://docs.docker.com/install/)
+- [Minikube](https://kubernetes.io/docs/setup/) (optional)
+
+With Docker and Kurtosis CLI installed, you can run the following command to deploy the network:
+
+```shell
+kurtosis run --enclave private-testnet --args-file network_params.yaml .
+```
+
+If you want to run the network in a local Kubernetes cluster, that's outside the scope of the TLDR, but you can find the instructions in the [Minikube section](#minikube).
+
+## Reasoning behind this tool selection
+
+As this repository is scoped to deploy a private Ethereum network, the tools selected were chosen to make the deployment as easy as possible. Kurtosis is a tool that allows you to deploy a network of services in a containerized environment, and it provides a simple way to define the network parameters and the services that will be deployed, allowing to integrate existing logging, monitoring, and testing tools used in the Ethereum ecosystem, while keeping an ample amount of flexibility that allows to use different combinations of Execution Layers (ELs) with Consensus Layers (CLs), using custom images, alternate validators, and different networks, like Goerli, Sepolia, Holesky, and others.
+
+Additionally, the Ethereum Package being extended here is supported by [multiple DevOps](https://ethpandaops.io/team/) that are part of Ethereum, the Ethereum Foundation, and other organizations, giving a sense of trust and reliability to the package.
+
+### Why not use Docker Compose?
+
+While Docker Compose is a great tool to deploy a network of services, it lacks some features that Kurtosis provides, such as the ability to define an API to modify the network, services and the ability to run tests on the network, giving the ability to create a package that can be shared with others—just as the Ethereum Package was built and it's being extended here—.
+
+### Why not use Helm?
+
+When talking about packages the next tool that comes to mind is Helm—the package manager for Kubernetes—, but Helm is more focused on deploying applications in a Kubernetes cluster, and you might not be familiar with Kubernetes, and you might not want to learn it just to deploy a private Ethereum network for testing purposes.
+
+> [!INFO]
+> If you are looking to deploy a private Ethereum network in a production environment, you should consider using Helm or other tools that are more focused on deploying applications for production environments.
+
+### Caveats
+
+Kurtosis is a tool that is still in development, and it might have some rough edges, but it's a great tool to deploy a network of services in a containerized environment, and it provides a simple way to define the network parameters and the services that will be deployed when paired with the [Ethereum Package](https://github.com/ethpandaops/ethereum-package).
+
+Unfortunately, Kurtosis doesn't have a way to easily migrate or deploy using other tools, such as Helm, Argo CD, or others, causing friction when trying to transition from a development environment to a production environment.
+
+> [!WARNING]
+> Right now, Kurtosis is best suited for loca/dev/testing use cases and simplifying onboarding to a distributed app. It's missing some features fto use it as a production deployment or infrastructure management tool.
+
+And
+
+> [!NOTE]
+> If you are looking to deploy a private Ethereum network in a production environment (or public Dev/Testnet networks), you should consider using Helm or other tools that are more focused for this purpose. Consider using https://github.com/ethpandaops/template-devnets or https://github.com/ethpandaops/ethereum-helm-charts
+
+## Understanding Kurtosis and the Ethereum Package
+
+The best places to start are:
+
+- [Setting up a local Ethereum testnet with Kurtosis](https://docs.kurtosis.com/how-to-local-eth-testnet)
+- The Ethereum Package [Architecture](https://github.com/ethpandaops/ethereum-package/blob/main/docs/architecture.md)
+- The Ethereum Package [Usage](https://ethpandaops.io/posts/kurtosis-deep-dive/)
+
+### Advanced topics
+
+- [Running an L2 devnet](https://ethpandaops.io/posts/kurtosis-l2/) on top of the Ethereum Package
+- [Testing](https://ethpandaops.io/posts/assertoor-introduction/) with the Ethereum Package
+
+## Minikube
+
+If you want to run the network in a local Kubernetes cluster, you can use Minikube. To do so, you need to have Minikube installed and running, to do so, you can follow this instructions:
+
+- [Install Minikube](https://github.com/kubernetes/minikube)
+- Run `minikube start`
+- Copy these contents to `kurtosis-config.yaml` file in `"$(kurtosis config path)"`:
+
+```yaml
+config-version: 2
+should-send-metrics: true
+kurtosis-clusters:
+  docker:
+    type: "docker"
+  minikube:
+    type: "kubernetes"
+    config:
+      kubernetes-cluster-name: "minikube"
+      storage-class: "standard"
+      enclave-size-in-megabytes: 10
+```
+
+- Run `kurtosis cluster set minikube`
+- In another terminal, run `kurtosis gateway`. This will act as a middle man between your computer's ports and your services deployed on Kubernetes ports and has to stay running as a separate process. More info: https://docs.kurtosis.com/k8s/#iv-configure-kurtosis
+- Run `kurtosis engine restart --enclave-pool-size 3` (this step is optional, but it's recommended taking it as it improves the user experience during the enclave creation, specifically regarding speed)
+- Run `kurtosis run --enclave private-testnet --args-file network_params.yaml .`
+
+### Troubleshooting
+
+If your minikube gets exhausted, you can run it with maximum resources available on your machine. This might be a bit overkill, but it's a good way to ensure that you have enough resources to run your Ethereum Network.
+
+```shell
+minikube start --memory=max --cpus=max
+```
+
+If you're having this error, it might be because your minikube is running out of resources:
+
+```shell
+ERRO[2024-09-30T19:57:16+01:00] An error occurred connecting to engine '35f9b86b4c674c95a30e075f3104f0e7':
+Expected to be able to get a port forwarded connection to engine '35f9b86b4c674c95a30e075f3104f0e7', instead a non-nil error was returned
+ --- at /root/project/cli/cli/kurtosis_gateway/live_engine_client_supplier/reconnecting_engine_client_supplier.go:160 (LiveEngineClientSupplier.replaceCurrentEngineInfo) ---
+Caused by: Expected to be able to find an api endpoint for Kubernetes portforward to engine '35f9b86b4c674c95a30e075f3104f0e7', instead a non-nil error was returned
+ --- at /root/project/cli/cli/kurtosis_gateway/connection/provider.go:69 (GatewayConnectionProvider.ForEngine) ---
+Caused by: Expected to find exactly 1 running Kurtosis Engine pod, instead found '0'
+ --- at /root/project/cli/cli/kurtosis_gateway/connection/provider.go:137 (GatewayConnectionProvider.getEnginePodPortforwardEndpoint) --- 
+```

network_params.yaml

@@ -1,18 +1,36 @@
+# More parameters can be found in https://github.com/ethpandaops/ethereum-package?tab=readme-ov-file#configuration
+# Specification of the participants in the network
 participants:
+  # EL(Execution Layer) Specific flags
+    # The type of EL client that should be started
+    # Valid values are geth, nethermind, erigon, besu, ethereumjs, reth, nimbus-eth1
   - el_type: geth
+    # The Docker image that should be used for the EL client
     el_image: "ethereum/client-go:v1.14.8"
+  # CL(Consensus Layer) Specific flags
+    # The type of CL client that should be started
+    # Valid values are nimbus, lighthouse, lodestar, teku, prysm, and grandine
     cl_type: lighthouse
+    # The Docker image that should be used for the CL client;
     cl_image: "sigp/lighthouse:v5.3.0"
+    # Count of nodes to spin up for this participant
     count: 3
+    # Prometheus additional configuration for a given participant prometheus target.
+    # Execution, beacon and validator client targets on prometheus will include this
+    # configuration.
     prometheus_config:
       scrape_interval: 15s
       labels: {}
+# Configuration parameters for the network
 network_params:
-  seconds_per_slot: 6
-  genesis_delay: 120
+# Number of seconds per slot on the Beacon chain
+  seconds_per_slot: 12
+# How long you want the network to wait before starting up
+  genesis_delay: 20
+# Additional services available in the Ethereum Package that can be enabled 
 additional_services:
   - prometheus_grafana # Spins up a grafana with prometheus to monitor the performance of the devnet
-  # - blockscout # TODO: No image available for Arm64. Wait for https://github.com/blockscout/blockscout-rs/pull/747. Use Dora instead.
+  # - blockscout # TODO: No image available for Arm64. Wait for https://github.com/blockscout/blockscout-rs/pull/747. Using Dora instead.
   - dora # A ligthweight beaconchain explorer
   - el_forkmon # A tool to keep track on a set of nodes, and see if they keep in step or if they go out of consensus.
   - beacon_metrics_gazer # A validator metrics tool to monitor the performance of the validators on the network
@@ -21,34 +39,69 @@ additional_services:
   - dugtrio # A consensus layer load balancer
   - blutgang # An execution layer load balancer
   - tracoor # A beacon data and execution trace explorer
+# Configuration place for prometheus
 prometheus_params:
   storage_tsdb_retention_time: "1d"
   storage_tsdb_retention_size: "512MB"
+  # Resource management for prometheus container
+  # CPU is milicores
+  # RAM is in MB
   min_cpu: 10
   max_cpu: 1000
   min_mem: 128
   max_mem: 2048
+# Configuration place for grafana
 grafana_params:
+  # A list of locators for grafana dashboards to be loaded be the grafana service
   additional_dashboards: []
     # TODO: add extra custom dashboards and alerts to the list once it's supported
     # - src/grafana/provisioning/alerting/alert_rules.yaml
     # - src/grafana/provisioning/dashboards/custom_darshboard.yaml
+  # Resource management for grafana container
+  # CPU is milicores
+  # RAM is in MB
   min_cpu: 10
   max_cpu: 1000
   min_mem: 128
   max_mem: 2048
+# Configuration place for the assertoor testing tool - https://github.com/ethpandaops/assertoor
 assertoor_params:
+  # Check chain stability
+  # This check monitors the chain and succeeds if:
+  # - all clients are synced
+  # - chain is finalizing for min. 2 epochs
+  # - >= 98% correct target votes
+  # - >= 80% correct head votes
+  # - no reorgs with distance > 2 blocks
+  # - no more than 2 reorgs per epoch
   run_stability_check: true
-  run_transaction_test: true
+  # Check block proposals
+  # This check monitors the chain and succeeds if:
+  # - all client pairs have proposed a block
+  run_transaction_test: false # Being done in the block-proposal-check.yaml
+  # Run additional tests from external test definitions
+  # Entries may be simple strings (link to the test file) or dictionaries with more flexibility
+  # eg:
+  #   - https://raw.githubusercontent.com/ethpandaops/assertoor/master/example/tests/block-proposal-check.yaml
+  #   - file: "https://raw.githubusercontent.com/ethpandaops/assertoor/master/example/tests/block-proposal-check.yaml"
+  #     config:
+  #       someCustomTestConfig: "some value"
   tests:
     - ./.github/workflows/tests/assertoor/all-opcodes-test.yaml
     - ./.github/workflows/tests/assertoor/block-proposal-check.yaml
 # global_log_level: "info"
+# EngineAPI Snooper global flags for all participants
+# Default to false
 snooper_enabled: false
+# Enables Ethereum Metrics Exporter for all participants
 ethereum_metrics_exporter_enabled: true
 # disable_peer_scoring: false
-# persistent: true
-xatu_sentry_enabled: false # TODO: Enable this once the Xatu Server/Clickhouse is deployed outside of Kurtosis
+# Whether the environment should be persistent; this is WIP and is slowly being rolled out accross services
+# Note this requires Kurtosis greater than 0.85.49 to work
+# Note Erigon, Besu, Teku persistence is not currently supported with docker.
+persistent: false
+# Enables Xatu Sentry for all participants - https://github.com/ethpandaops/xatu
+xatu_sentry_enabled: false # TODO: Enable this once a Xatu Server/Clickhouse deployment has been added to this package
 xatu_sentry_params:
   xatu_sentry_image: ethpandaops/xatu:latest
   xatu_server_addr: localhost:8080