Skip to content

Commit

Permalink
CPU optimizations and recommendations
Browse files Browse the repository at this point in the history
Signed-off-by: Richard Wall <richard.wall@venafi.com>
  • Loading branch information
wallrj committed Apr 9, 2024
1 parent 131a9ad commit e70bd00
Show file tree
Hide file tree
Showing 2 changed files with 36 additions and 0 deletions.
36 changes: 36 additions & 0 deletions content/docs/devops-tips/large-clusters.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,42 @@ with thousands of Certificate and Secret resources.
The defaults in the Helm chart or YAML manifests are intended for general use.
You will need to modify the configuration if your Kubernetes cluster has thousands of Certificate resources and TLS Secrets.

## CPU



### Recommendations

#### Disable client-side rate limiting for Kubernetes API requests

By default cert-manager throttles the rate of requests to the Kubernetes API server.
Historically this was intended to prevent cert-manager from overwhelming the Kubernetes API server,
but modern versions of Kubernetes implement [API Priority and Fairness](https://kubernetes.io/docs/concepts/cluster-administration/flow-control/),
which obviates the need for client side throttling.
Disable the client-side rate limiter as follows:

```yaml
config:
apiVersion: controller.config.cert-manager.io/v1alpha1
kind: ControllerConfiguration
kubernetesAPIQPS: -1
kubernetesAPIBurst: -1
```
> 📖 Read [API documentation for ControllerConfiguration](https://cert-manager.io/docs/reference/api-docs/#controller.config.cert-manager.io%2fv1alpha1).
>
> 📖 Read [kubernetes#111880: Disable client-side rate-limiting when AP&F is enabled](https://github.com/kubernetes/kubernetes/issues/111880).
>
> 📖 Read client-go source code to [understand why negative QPS disables the rate limiter](https://github.com/kubernetes/kubernetes/blob/6813625b7cd706db5bc7388921be03071e1a492d/staging/src/k8s.io/client-go/rest/config.go#L351-L364).
>
> 🔗 Examples of other projects that disable client-side rate limiting are: [FluxCD](https://github.com/fluxcd/pkg/issues/269).
### Rationale
### Evidence
<img src="/docs/devops-tips/large-clusters/default-cpu-1.png" alt="Scatter chart showing cert-manager CPU usage and cluster resource counts over time" />
## Memory
### Recommendations
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit e70bd00

Please sign in to comment.