OpenWRT 24.10-rc2

[bohdan-s]

I am trying to get started with ctrld on the new OpenWRT 24.10-rc2

I install curl via opkg then run the standard install command
sh -c 'sh -c "$(curl -sSL https://api.controld.com/dl)" -s xxxxxx forced'

On install all outbound traffic on WAN stops, no ping, no web traffic nothing.
Only way to get traffic back is to stop the service and reboot the router.

Every 2-3 times I run the service it will start without breaking internet access, but it does not intercept DNS traffic at all.
When it doesn't break the internet it shows the below logs:

Log below:

root@OpenWrt:~# ctrld run --cd xxxxxxxx --iface=auto --homedir=/etc/controld --config=/etc/controld/ctrld.toml -vv
Dec 9 03:16:48.000 INF loading config file from: /etc/controld/ctrld.toml
Dec 9 03:16:48.053 INF starting ctrld v1.3.11
Dec 9 03:16:48.055 INF os: linux 6.6.63
Dec 9 03:16:48.056 INF fetching Controld D configuration from API: xxxxxxx mode=cd
Dec 9 03:16:48.059 DBG resolving "api.controld.com" using bootstrap DNS ["76.76.2.22:53" "202.128.112.1:53" "[fe80::2293:39ff:fef6:75e7]:53"]
Dec 9 03:16:50.061 DBG got answer from nameserver: 76.76.2.22
Dec 9 03:16:52.062 DBG got answer from nameserver: 76.76.2.22
Dec 9 03:16:52.062 DBG API IPs: [2606:1a40:3::1 147.185.34.1]
Dec 9 03:16:53.431 INF generating ctrld config from Control-D configuration mode=cd
Dec 9 03:16:53.432 INF listener.0 could not listen on address: 0.0.0.0:53, trying: 0.0.0.0:53
Dec 9 03:16:53.433 INF listener.0 could not listen on address: 0.0.0.0:53, trying localhost: 127.0.0.1:53
Dec 9 03:16:53.434 INF listener.0 could not listen on address: 127.0.0.1:53, trying current ip with port 5354
Dec 9 03:16:53.438 INF writing config file to: /etc/controld/ctrld.toml
Dec 9 03:16:53.443 DBG control server started: /var/run/ctrld_control.sock
Dec 9 03:16:53.444 INF using bootstrap IP for upstream.0 bootstrap_ip=76.76.2.22
Dec 9 03:16:53.453 DBG checking default route interface iface=eth0
Dec 9 03:16:53.459 WRN no default route IP found
Dec 9 03:16:53.460 DBG start hosts file discovery
Dec 9 03:16:53.460 INF starting DNS server on listener.0: 0.0.0.0:5354
Dec 9 03:16:53.461 DBG start dhcp discovery
Dec 9 03:16:53.462 DBG router setup on start
Dec 9 03:16:53.466 DBG start arp discovery
Dec 9 03:16:53.467 ERR could not configure router error="open /tmp/dnsmasq.d/ctrld.conf: no such file or directory"
Dec 9 03:16:53.468 DBG starting custom config reload timer mode=api-reload
Dec 9 03:16:53.468 DBG start checking DNS loop
Dec 9 03:16:53.468 DBG skipping external: upstream.0
Dec 9 03:16:53.468 DBG end checking DNS loop
Dec 9 03:16:53.469 DBG listening ndp on: ::%br-lan.66
Dec 9 03:16:53.470 DBG listening ndp on: ::%eth0
Dec 9 03:16:53.471 DBG listening ndp on: ::%br-lan
Dec 9 03:16:53.473 DBG start ptr discovery
Dec 9 03:16:53.474 DBG start mdns discovery
Dec 9 03:16:53.573 DBG sending doh request to: 76.76.2.22:443
Dec 9 03:16:53.645 DBG could not find avahi-browse binary, skipping. error="exec: "avahi-browse": executable file not found in $PATH"
Dec 9 03:16:53.645 WRN stop probing "[ff02::fb]:5353": write udp6 [::]:5353->[ff02::fb]:5353: sendto: network is unreachable
Dec 9 03:16:53.645 WRN stop probing "224.0.0.251:5353": write udp4 0.0.0.0:5353->224.0.0.251:5353: sendto: operation not permitted
Dec 9 03:16:53.665 DBG sending doh request to: 76.76.2.22:443
Dec 9 03:17:53.469 DBG start checking DNS loop
Dec 9 03:17:53.470 DBG skipping external: upstream.0
Dec 9 03:17:53.471 DBG end checking DNS loop
Dec 9 03:18:53.470 DBG start checking DNS loop
Dec 9 03:18:53.471 DBG skipping external: upstream.0
Dec 9 03:18:53.471 DBG end checking DNS loop

Stopping service here...

Dec 9 03:19:26.404 DBG dns watchers stopped
Dec 9 03:19:26.404 INF Service stopped
Dec 9 03:19:26.405 DBG router cleanup on stop
Dec 9 03:19:26.405 ERR could not cleanup router error="remove /tmp/dnsmasq.d/ctrld.conf: no such file or directory"
Dec 9 03:19:26.407 DBG Restoring DNS for interface iface=eth0
Dec 9 03:19:26.555 DBG dns: [rc=unknown ret=direct]
Dec 9 03:19:26.555 DBG dns: using "direct" mode
Dec 9 03:19:26.631 DBG Restoring DNS successfully iface=eth0

[bohdan-s]

And logs for when the internet traffic stops

The key error seems to be "no default route IP found"

GNU nano 8.2 /tmp/ctrld.log
{"level":"info","time":"2024-12-09T03:37:00Z.360","message":"starting ctrld v1.3.11"}
{"level":"info","time":"2024-12-09T03:37:00Z.362","message":"os: linux 6.6.63"}
{"level":"info","mode":"cd","time":"2024-12-09T03:37:06Z.133","message":"fetching Controld D configuration from API: xxxxx"}
{"level":"debug","time":"2024-12-09T03:37:06Z.134","message":"resolving "api.controld.com" using bootstrap DNS ["76.76.2.22:53" "202.128.112.1:53"]"}
{"level":"debug","time":"2024-12-09T03:37:08Z.135","message":"got answer from nameserver: 76.76.2.22"}
{"level":"debug","time":"2024-12-09T03:37:11Z.167","message":"got answer from nameserver: 76.76.2.22"}
{"level":"debug","time":"2024-12-09T03:37:11Z.167","message":"API IPs: [2606:1a40:3::1 147.185.34.1]"}
{"level":"info","mode":"cd","time":"2024-12-09T03:37:12Z.543","message":"generating ctrld config from Control-D configuration"}
{"level":"info","time":"2024-12-09T03:37:12Z.544","message":"listener.0 could not listen on address: 0.0.0.0:53, trying: 0.0.0.0:53"}
{"level":"info","time":"2024-12-09T03:37:12Z.545","message":"listener.0 could not listen on address: 0.0.0.0:53, trying localhost: 127.0.0.1:53"}
{"level":"info","time":"2024-12-09T03:37:12Z.545","message":"listener.0 could not listen on address: 127.0.0.1:53, trying current ip with port 5354"}
{"level":"info","time":"2024-12-09T03:37:12Z.548","message":"writing config file to: /etc/controld/ctrld.toml"}
{"level":"debug","time":"2024-12-09T03:37:12Z.550","message":"control server started: /var/run/ctrld_control.sock"}
{"level":"info","bootstrap_ip":"76.76.2.22","time":"2024-12-09T03:37:12Z.551","message":"using bootstrap IP for upstream.0"}
{"level":"debug","iface":"eth0","time":"2024-12-09T03:37:12Z.554","message":"checking default route interface"}
{"level":"warn","time":"2024-12-09T03:37:12Z.557","message":"no default route IP found"}
{"level":"info","time":"2024-12-09T03:37:12Z.557","message":"starting DNS server on listener.0: 0.0.0.0:5354"}
{"level":"debug","time":"2024-12-09T03:37:12Z.558","message":"start hosts file discovery"}
{"level":"debug","time":"2024-12-09T03:37:12Z.559","message":"router setup on start"}
{"level":"debug","time":"2024-12-09T03:37:12Z.559","message":"start dhcp discovery"}
{"level":"debug","time":"2024-12-09T03:37:12Z.568","message":"start arp discovery"}
{"level":"error","error":"open /tmp/dnsmasq.d/ctrld.conf: no such file or directory","time":"2024-12-09T03:37:12Z.570","message":"could not configure router"}
{"level":"debug","time":"2024-12-09T03:37:12Z.570","message":"start checking DNS loop"}

[bohdan-s]

Install logs (attempted dev, but logs are the same for release as well)

As soon as the installer hits "Generating controld config: /etc/controld/ctrld.toml" that's when all traffic stops.

root@OpenWrt:~# sh -c 'sh -c "$(curl -sSL https://api.controld.dev/dl)" -s xxxxxx'

        __         .__       .___
  _____/  |________|  |    __| _/
_/ ___\   __\_  __ \  |   / __ |
\  \___|  |  |  | \/  |__/ /_/ |
 \___  >__|  |__|  |____/\____ |
     \/       installer       \/

---------------------
|    System Info    |
---------------------
OS Type      : linux
OS Vendor    : openwrt
OS Version   : 24.10.0-rc2
Arch         : aarch64
CPU          : aarch64_generic
Free RAM     : 3692 MB / 3935 MB
---------------------
|  Install Details  |
---------------------
Resolver ID  : xxxxx
Binary URL   : https://dl.controld.dev/linux-arm64/ctrld
Install Path : /usr/sbin
---------------------
Install binary and run it? (y/n): y
 - Starting download
 - Making binary executable
 - Launching /usr/sbin/ctrld
---------------------
Dec  9 06:12:39.000 NTC Reading config: /etc/controld/ctrld.toml
Dec  9 06:12:39.274 NTC Starting service
Dec  9 06:12:39.665 NTC Generating controld config: /etc/controld/ctrld.toml


Dec  9 06:13:38.204 ERR failed to connect to upstream.0, endpoint: https://dns.controld.com/13sfsk6qy4l error="context deadline exceeded"
Dec  9 06:13:38.204 ??? ================================
Dec  9 06:13:38.205 ??? An error occurred while performing test query: no answer from ctrld listener
Dec  9 06:13:38.205 ??? ================================
Dec  9 06:13:40.401 NTC Service uninstalled

[verymucho]

It's because dnsmasq now creates a unique dir for each instance rather than the old /tmp/dnsmasq.d.

For a quick/dirty fix you can change/comment the line in the /etc/init.d/dnsmasq file (make a backup if desired) from:

config_get dnsmasqconfdir "$cfg" confdir "/tmp/dnsmasq${cfg:+.$cfg}.d"

to:

config_get dnsmasqconfdir "$cfg" confdir "/tmp/dnsmasq.d"

Which is the old behavior and do a service dnsmasq restart. After that, ctrld should work again.

Once a fix is provided for ctrld you can undo your changes.

[bohdan-s]

Thank you. That fixes the DNS issue and now it runs and intercepts traffic.
I still have the issue that it is somehow stopping all traffic on some start ups.
Are there any more detailed logs it provides to maybe get an idea of what it’s doing at that stop that breaks traffic ?

[verymucho]

Thank you. That fixes the DNS issue and now it runs and intercepts traffic. I still have the issue that it is somehow stopping all traffic on some start ups. Are there any more detailed logs it provides to maybe get an idea of what it’s doing at that stop that breaks traffic ?

Wish I knew the answer to that one. I've been seeing it too whenever the ctrld service starts. I have to go into Luci/SSH login and restart the WAN interface to get my internet working again. Then it stays working as long as I don't restart the router/ctrld service.

Not sure what's causing it yet.

[NoSync]

Thank you. That fixes the DNS issue and now it runs and intercepts traffic. I still have the issue that it is somehow stopping all traffic on some start ups. Are there any more detailed logs it provides to maybe get an idea of what it’s doing at that stop that breaks traffic ?

Wish I knew the answer to that one. I've been seeing it too whenever the ctrld service starts. I have to go into Luci/SSH login and restart the WAN interface to get my internet working again. Then it stays working as long as I don't restart the router/ctrld service.

Not sure what's causing it yet.

Same. I installed watchcat to patch that issue, so I don't have to do anything manually, but it is indeed still there.

[timlu85]

It's because dnsmasq now creates a unique dir for each instance rather than the old /tmp/dnsmasq.d.

For a quick/dirty fix you can change/comment the line in the /etc/init.d/dnsmasq file (make a backup if desired) from:

config_get dnsmasqconfdir "$cfg" confdir "/tmp/dnsmasq${cfg:+.$cfg}.d"

to:

config_get dnsmasqconfdir "$cfg" confdir "/tmp/dnsmasq.d"

Which is the old behavior and do a service dnsmasq restart. After that, ctrld should work again.

Once a fix is provided for ctrld you can undo your changes.

Thx modify config work for me on new release OpenWrt 24.10.0 !!