Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add App Net based CSM Observability Test #72

Closed
wants to merge 41 commits into from

Conversation

stanley-cheung
Copy link
Contributor

  • Added app_net_csm_observability_test.py that's largely a copy of gamma/csm_observability_test.py with some adjustments
  • Refactor some flags to be passed to the k8s_xds_server_runner

This PR was based on top of #33.

Able to run both tests successfully on gamma-enabled cluster.

- Ping GMP endpoint in client and server pod curl localhost:9464/metrics during test
- This will provide insight into what GMP gets from the OTel plugin before sending metrics to Cloud Monitoring
- But this doesn't work yet. Can't quite figure out what's the exact request URL to use. Tried quite a few combinations and none of them works.
- `enable_csm_observability` is now a constructor parameter for the client runner and the server runner, instead of a `run()` parameter.
- If `enable_csm_observability`, we want to do a port forwarding for the `pod_monitoring_port = 9464`, such that we can issue a GET request against it in the CSM test
- Added `monitoring_port` and `monitoring_host` as class variable to the client runner and server runner.
- Override `_xds_test_server_for_pod` in the `GammaServerRunner` class
- Minimize the places where we need to hardcode the port `9464`
- Added a PrometheusLogger class to write the prometheus endpoint log to a separate log file related to logs_subdir
- monitoring_port is now stored in XdsTestClient and XdsTestServer rather than the runner
- use rpc_host instead of a separate monitoring_host
- put the constant DEFAULT_MONITORING_PORT in the KubernetesBaseRunner
- fixed requirements.txt and requirements.lock
- removed a previous hack is_legit_time_series()
- renamed function to ping_prometheus_endpoint instead of referring it as GMP
- Added app_net_csm_observability_test.py that's largely a copy of gamma/csm_observability_test.py
- Refactor some flags to be passed to the k8s_xds_server_runner
@stanley-cheung stanley-cheung requested a review from sergiitk April 18, 2024 21:47
@@ -103,13 +100,14 @@ def __init__(
namespace_template=namespace_template,
debug_use_port_forwarding=debug_use_port_forwarding,
enable_workload_identity=enable_workload_identity,
enable_csm_observability=enable_csm_observability,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead, let's put all this into deployment_args, see

class ServerDeploymentArgs:
pre_stop_hook: bool = False
termination_grace_period: dt.timedelta = dt.timedelta()

sergiitk pushed a commit that referenced this pull request Jun 6, 2024
- Created a new test `app_net_csm_observability_test.py` that's largely
a copy of `gamma/csm_observability_test.py` with some adjustments.
- Refactor some flags to be passed to the `k8s_xds_server_runner`.
- Use `ServerDeploymentArgs` everywhere

This PR was based on top of #33, and a replacement of #72.
@stanley-cheung stanley-cheung deleted the app-net-csm-o11y branch June 6, 2024 17:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants