Skip to content
This repository has been archived by the owner on Nov 13, 2024. It is now read-only.

Commit

Permalink
[RTC-405] Update clustering documentation (#72)
Browse files Browse the repository at this point in the history
  • Loading branch information
mickel8 authored Jan 3, 2024
1 parent 2e8f62f commit f09474c
Show file tree
Hide file tree
Showing 2 changed files with 22 additions and 42 deletions.
46 changes: 11 additions & 35 deletions docs/cluster.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,13 +82,13 @@ To form a cluster using `NODES_LIST` strategy:
Run the first Jellyfish:

```sh
JF_DIST_ENABLED=true JF_DIST_NODE_NAME=j1@127.0.0.1 mix phx.server
JF_DIST_ENABLED=true JF_DIST_NODE_NAME=j1@localhost mix phx.server
```

Run the second Jellyfish

```sh
JF_DIST_ENABLED=true JF_DIST_NODE_NAME=j2@127.0.0.1 JF_DIST_NODES="j1@127.0.0.1" JF_PORT=4002 JF_METRICS_PORT=9468 mix phx.server
JF_DIST_ENABLED=true JF_DIST_NODE_NAME=j2@localhost JF_DIST_NODES="j1@localhost" JF_PORT=4002 JF_METRICS_PORT=9468 mix phx.server
```

:::info
Expand All @@ -110,7 +110,6 @@ x-jellyfish-template: &jellyfish-template
environment: &jellyfish-environment
JF_SERVER_API_TOKEN: "development"
JF_DIST_ENABLED: "true"
JF_DIST_MODE: "sname"
JF_DIST_NODES: "j1@jellyfish1 j2@jellyfish2"
restart: on-failure

Expand Down Expand Up @@ -166,86 +165,63 @@ To form a cluster using `DNS` strategy:

1. Enable distribution mode with `JF_DIST_ENABLED=true`
1. Chose `DNS` strategy with `JF_DIST_STRATEGY_NAME`.
1. Set `JF_DIST_MODE` to `name`.
1. Give your node a name with `JF_DIST_NODE_NAME`.<br />
**Important** It has to be in the form of `<nodename>@<ip_address>`
**Important** It has to be in the form of `<nodename>@<hostname>`
where all Jellyfishes MUST have the same `<nodename>`.
1. Specify a query under which Jellyfishes are register in DNS with `JF_DIST_QUERY`. <br />
**Important** Jellyfish does not register itself in DNS.
It is user responsibility to enusre that your Jellyfish is registered in DNS under `JF_DIST_QUERY`.

### Running with Docker

This simple docker compose file sets a cluster of two Jellyfishes and starts a DNS with use of dnsmasq.
This simple docker compose file sets a cluster of two Jellyfishes using internal Docker DNS.

```yml
version: "3"
x-jellyfish-template: &jellyfish-template
build: .
environment: &jellyfish-environment
JF_SERVER_API_TOKEN: "development"
JF_DIST_ENABLED: "true"
JF_DIST_STRATEGY_NAME: "DNS"
JF_DIST_MODE: "name"
restart: on-failure
services:
app1:
<<: *jellyfish-template
environment:
<<: *jellyfish-environment
JF_HOST: "localhost:4001"
JF_PORT: 4001
JF_DIST_NODE_NAME: app@172.28.1.2
JF_DIST_QUERY: app.dns-network
ports:
- 4001:4001
networks:
dns-network:
ipv4_address: 172.28.1.2
default:
aliases:
- app.dns-network
dns:
- 172.28.1.4:5353
app2:
container_name: name
<<: *jellyfish-template
environment:
<<: *jellyfish-environment
JF_HOST: "localhost:4002"
JF_PORT: 4002
JF_DIST_NODE_NAME: app@172.28.1.3
JF_DIST_QUERY: app.dns-network
ports:
- 4002:4002
networks:
dns-network:
ipv4_address: 172.28.1.3
default:
aliases:
- app.dns-network
dns:
- 172.28.1.4:5353
dnsmasq:
image: andyshinn/dnsmasq:2.78
volumes:
- /etc/resolv.conf:/etc/resolv.dnsmasq.conf
command: -d -q --no-hosts --no-resolv --no-poll --server 127.0.0.11 --listen-address 0.0.0.0 --port 5353 --log-queries --log-facility=- --address=/./127.0.0.11
networks:
dns-network:
ipv4_address: 172.28.1.4
aliases:
- dnsmasq
networks:
dns-network:
ipam:
config:
- subnet: 172.28.1.0/24
```

Because we run Jellyfishes in the same Docker network we don't need to export EPMD (`4369`) or distribution (`9000`)
ports.
We also didn't have to explicitly set `JF_DIST_NODE_NAME`.
The default value (`jellyfish@(hostname)`) is automatically resolved to `jellyfish@<ip_address>`
and is routable from other nodes in the docker network.

## Verifying that a cluster has been created

Expand Down
18 changes: 11 additions & 7 deletions docs/getting_started/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -181,20 +181,24 @@ Defaults to `false`.
Possible values are `DNS` or `NODES_LIST`.
Defaults to `NODES_LIST`.
* `JF_DIST_NODE_NAME` - Node name used in a cluster.
Defaults to `jellyfish@(hostname)`. <br/>
It consists of two parts - nodename@hostname.
The first part identifies a node on a single machine and can
be any string.
The second part identifies the host machine and has to be an
ip address or FQDN of a machine Jellyfish runs on.
If you run a cluster using `NODES_LIST` strategy on a single machine
or in the same docker network and you don't want to use IP addresses or FQDN as hostnames,
you can use short names (see `JF_DIST_MODE`).
The second part identifies the host machine jellyfish runs on and has to be resolvable.
Its format depends on `JF_DIST_MODE`.
If `JF_DIST_MODE` is set to `name`, the `hostname` has to be an IP address or FQDN.
If `JF_DIST_MODE` is set to `sname`, the `hostname` can be any string.
If you run a cluster using `DNS` strategy, every Jellyfish instance must have `nodename` set to the same value.
If `hostname` is not an IP address, it will be automatically resolved as DNS strategy requires
node names to use IP addresses.
This also means that `DNS` strategy has to be run with `JF_DIST_MODE` set to `name`.
See our [docker-compose-epmd.yaml](https://github.com/jellyfish-dev/jellyfish/blob/main/docker-compose-epmd.yaml) or [docker-compose-dns.yaml](https://github.com/jellyfish-dev/jellyfish/blob/main/docker-compose-dns.yaml), which we use in our integration tests or refer to [Cluster](../cluster.md) section for examples.
* `JF_DIST_MODE` - distribution mode - can be `name` or `sname`.<br/>
Defaults to `name`.<br/>
Defaults to `sname`.<br/>
When using `name`, your hostname has to be an IP address or FQDN of a machine Jellyfish runs on.
When using `sname`, your hostname can be any string.
See our [docker-compose.yaml](https://github.com/jellyfish-dev/jellyfish/blob/main/docker-compose.yaml), which we use in our integration tests for an example.
See `JF_DIST_NODE_NAME` for more information.
* `JF_DIST_COOKIE` - used to group Jellyfishes in a cluster.<br/>
Defaults to `jellyfish_cookie`.<br/>
Use different cookies to create multiple clusters on the same machine.<br/>
Expand Down

0 comments on commit f09474c

Please sign in to comment.