Metrics are not working

[Blarc]

I'm running the operator on Talos v1.9.1.
When I k port-forward svc/wireguard-metrics-svc 9586:9586 and open http://localhost:9586/metrics I get only:

# HELP wireguard_sent_bytes_total Bytes sent to the peer
# TYPE wireguard_sent_bytes_total counter

# HELP wireguard_received_bytes_total Bytes received from the peer
# TYPE wireguard_received_bytes_total counter

# HELP wireguard_latest_handshake_seconds UNIX timestamp seconds of the last handshake
# TYPE wireguard_latest_handshake_seconds gauge

So I set --verbose "true" arg on metric container and got:

[2025-01-09T19:31:46Z TRACE prometheus_exporter_base] serve_function:: options == Options { verbose: true, prepend_sudo: false, separate_allowed_ips: false, extract_names_config_files: None, interfaces: None, export_remote_ip_and_port: false, export_latest_handshake_delay: false }
[2025-01-09T19:31:46Z TRACE prometheus_wireguard_exporter] interfaces_to_handle == ["all"]
[2025-01-09T19:31:46Z TRACE prometheus_wireguard_exporter] peer_entry_hashmap == None
[2025-01-09T19:31:46Z TRACE prometheus_wireguard_exporter] wg show all dump stdout ==
[2025-01-09T19:31:46Z TRACE prometheus_wireguard_exporter] wg show all dump stderr == Unable to access interface wg0: Operation not permitted

Then I tried to add --prepend-sudo "true" arg to metrics container and got:

[2025-01-09T19:37:22Z TRACE prometheus_exporter_base] serve_function:: options == Options { verbose: true, prepend_sudo: true, separate_allowed_ips: false, extract_names_config_files: None, interfaces: None, export_remote_ip_and_port: false, export_latest_handshake_delay: false }
[2025-01-09T19:37:22Z TRACE prometheus_wireguard_exporter] interfaces_to_handle == ["all"]
[2025-01-09T19:37:22Z TRACE prometheus_wireguard_exporter] peer_entry_hashmap == None
[2025-01-09T19:37:22Z WARN prometheus_exporter_base] internal server error == Os { code: 2, kind: NotFound, message: "No such file or directory" }

Any help would be appreciated 🙏 .

[Blarc]

So I believe I the problem is that the agent and metrics containers have their security context configured different.

Metrics container is run as non-root:

securityContext:
  allowPrivilegeEscalation: false
  capabilities:
    add:
    - NET_ADMIN
    drop:
    - ALL
  readOnlyRootFilesystem: true
  runAsGroup: 65534
  runAsNonRoot: true
  runAsUser: 65534

While agent is run as root:

securityContext:
  allowPrivilegeEscalation: false
  capabilities:
    add:
    - NET_ADMIN
  readOnlyRootFilesystem: true

And this is the reason why metrics container can't read the metrics from /etc/wireguard or wherever it reads them from.
After manually editing the deployment and removing the following part from metrics container configuration, the metrics start working:

readOnlyRootFilesystem: true
runAsGroup: 65534
runAsNonRoot: true
runAsUser: 65534

I'll open a new issue (#223), since this seems as a bug to me.