Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

POC: support for k8s secrets #330

Merged
merged 8 commits into from
Jul 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ GINKGO ?= $(LOCALBIN)/ginkgo

## Tool Versions
KUSTOMIZE_VERSION ?= v5.0.1
CONTROLLER_TOOLS_VERSION ?= v0.12.0
CONTROLLER_TOOLS_VERSION ?= v0.14.0
GINKGO_VERSION ?= v2.15.0

.PHONY: kustomize
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -157,11 +157,11 @@ anager.secretAccessor` permission
```
kubectl create secret generic gcp-sa-private-key --from-file=sa-private-key.json
```
1. Install mysql-operator with `--set cloudSecretManagerType=gcp --set gcpProjectId=$PROJECT_ID`
1. Install mysql-operator with `--set adminUserSecretType=gcp --set gcpProjectId=$PROJECT_ID`
```
helm repo add nakamasato https://nakamasato.github.io/helm-charts
helm repo update
helm install mysql-operator nakamasato/mysql-operator --set cloudSecretManagerType=gcp --set gcpProjectId=$PROJECT_ID
helm install mysql-operator nakamasato/mysql-operator --set adminUserSecretType=gcp --set gcpProjectId=$PROJECT_ID
```
1. You can specify `type: gcp` for `adminUser` and `adminPassword`.

Expand Down
4 changes: 2 additions & 2 deletions api/v1alpha1/mysql_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -94,9 +94,9 @@ type Secret struct {
// Secret Name
Name string `json:"name"`

// +kubebuilder:validation:Enum=raw;gcp
// +kubebuilder:validation:Enum=raw;gcp;k8s

// Secret Type (e.g. gcp, raw)
// Secret Type (e.g. gcp, raw, k8s)
Type string `json:"type"`
}

Expand Down
1 change: 0 additions & 1 deletion api/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

27 changes: 22 additions & 5 deletions cmd/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ import (
metricsserver "sigs.k8s.io/controller-runtime/pkg/metrics/server"

mysqlv1alpha1 "github.com/nakamasato/mysql-operator/api/v1alpha1"
"github.com/nakamasato/mysql-operator/internal/controller"
controllers "github.com/nakamasato/mysql-operator/internal/controller"
"github.com/nakamasato/mysql-operator/internal/mysql"
"github.com/nakamasato/mysql-operator/internal/secret"
//+kubebuilder:scaffold:imports
Expand All @@ -59,20 +59,25 @@ func main() {
var metricsAddr string
var enableLeaderElection bool
var probeAddr string
var cloudSecretManagerType string
var adminUserSecretType string
var projectId string
var secretNamespace string
flag.StringVar(&metricsAddr, "metrics-bind-address", ":8080", "The address the metric endpoint binds to.")
flag.StringVar(&probeAddr, "health-probe-bind-address", ":8081", "The address the probe endpoint binds to.")
flag.BoolVar(&enableLeaderElection, "leader-elect", false,
"Enable leader election for controller manager. "+
"Enabling this will ensure there is only one active controller manager.")
flag.StringVar(&cloudSecretManagerType, "cloud-secret-manager", "",
flag.StringVar(&adminUserSecretType, "cloud-secret-manager", "",
"The cloud secret manager to get credentials from. "+
"Currently, only support gcp")
flag.StringVar(&projectId, "gcp-project-id", "",
"GCP project id. Set this value to use cloudSecretManagerType=gcp. "+
"GCP project id. Set this value to use adminUserSecretType=gcp. "+
"Also can be set by environment variable PROJECT_ID."+
"If both are set, the flag is used.")
flag.StringVar(&secretNamespace, "k8s-secret-namespace", "",
"Kubernetes namespace where MYSQL credentials secrets is located. Set this value to use adminUserSecretType=k8s. "+
"Also can be set by environment variable SECRET_NAMESPACE."+
"If both are set, the flag is used.")
opts := zap.Options{
Development: true,
TimeEncoder: zapcore.ISO8601TimeEncoder,
Expand Down Expand Up @@ -109,7 +114,8 @@ func main() {
secretManagers := map[string]secret.SecretManager{
"raw": secret.RawSecretManager{},
}
if cloudSecretManagerType == "gcp" {
switch adminUserSecretType {
case "gcp":
if projectId == "" {
projectId = os.Getenv("PROJECT_ID")
}
Expand All @@ -121,6 +127,17 @@ func main() {
defer gcpSecretManager.Close()
setupLog.Info("Initialized gcpSecretManager", "projectId", projectId)
secretManagers["gcp"] = gcpSecretManager
case "k8s":
if secretNamespace == "" {
secretNamespace = os.Getenv("SECRET_NAMESPACE")
}
k8sSecretManager, err := secret.Newk8sSecretManager(ctx, secretNamespace, mgr.GetClient())
if err != nil {
setupLog.Error(err, "failed to initialize k8sSecretManager")
os.Exit(1)
}
setupLog.Info("Initialized k8sSecretManager", "namespace", secretNamespace)
secretManagers["k8s"] = k8sSecretManager
}
if err = (&controllers.MySQLReconciler{
Client: mgr.GetClient(),
Expand Down
19 changes: 12 additions & 7 deletions config/crd/bases/mysql.nakamasato.com_mysqldbs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.12.0
controller-gen.kubebuilder.io/version: v0.14.0
name: mysqldbs.mysql.nakamasato.com
spec:
group: mysql.nakamasato.com
Expand Down Expand Up @@ -33,14 +33,19 @@ spec:
description: MySQLDB is the Schema for the mysqldbs API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
Expand Down
25 changes: 16 additions & 9 deletions config/crd/bases/mysql.nakamasato.com_mysqls.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.12.0
controller-gen.kubebuilder.io/version: v0.14.0
name: mysqls.mysql.nakamasato.com
spec:
group: mysql.nakamasato.com
Expand Down Expand Up @@ -41,14 +41,19 @@ spec:
description: MySQL is the Schema for the mysqls API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
Expand All @@ -64,10 +69,11 @@ spec:
description: Secret Name
type: string
type:
description: Secret Type (e.g. gcp, raw)
description: Secret Type (e.g. gcp, raw, k8s)
enum:
- raw
- gcp
- k8s
type: string
required:
- name
Expand All @@ -80,10 +86,11 @@ spec:
description: Secret Name
type: string
type:
description: Secret Type (e.g. gcp, raw)
description: Secret Type (e.g. gcp, raw, k8s)
enum:
- raw
- gcp
- k8s
type: string
required:
- name
Expand Down
78 changes: 42 additions & 36 deletions config/crd/bases/mysql.nakamasato.com_mysqlusers.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
annotations:
controller-gen.kubebuilder.io/version: v0.12.0
controller-gen.kubebuilder.io/version: v0.14.0
name: mysqlusers.mysql.nakamasato.com
spec:
group: mysql.nakamasato.com
Expand Down Expand Up @@ -37,14 +37,19 @@ spec:
description: MySQLUser is the Schema for the mysqlusers API
properties:
apiVersion:
description: 'APIVersion defines the versioned schema of this representation
of an object. Servers should convert recognized schemas to the latest
internal value, and may reject unrecognized values. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources'
description: |-
APIVersion defines the versioned schema of this representation of an object.
Servers should convert recognized schemas to the latest internal value, and
may reject unrecognized values.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources
type: string
kind:
description: 'Kind is a string value representing the REST resource this
object represents. Servers may infer this from the endpoint the client
submits requests to. Cannot be updated. In CamelCase. More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds'
description: |-
Kind is a string value representing the REST resource this object represents.
Servers may infer this from the endpoint the client submits requests to.
Cannot be updated.
In CamelCase.
More info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds
type: string
metadata:
type: object
Expand All @@ -68,42 +73,42 @@ spec:
conditions:
items:
description: "Condition contains details for one aspect of the current
state of this API Resource. --- This struct is intended for direct
use as an array at the field path .status.conditions. For example,
\n type FooStatus struct{ // Represents the observations of a
foo's current state. // Known .status.conditions.type are: \"Available\",
\"Progressing\", and \"Degraded\" // +patchMergeKey=type // +patchStrategy=merge
// +listType=map // +listMapKey=type Conditions []metav1.Condition
`json:\"conditions,omitempty\" patchStrategy:\"merge\" patchMergeKey:\"type\"
protobuf:\"bytes,1,rep,name=conditions\"` \n // other fields }"
state of this API Resource.\n---\nThis struct is intended for
direct use as an array at the field path .status.conditions. For
example,\n\n\n\ttype FooStatus struct{\n\t // Represents the
observations of a foo's current state.\n\t // Known .status.conditions.type
are: \"Available\", \"Progressing\", and \"Degraded\"\n\t //
+patchMergeKey=type\n\t // +patchStrategy=merge\n\t // +listType=map\n\t
\ // +listMapKey=type\n\t Conditions []metav1.Condition `json:\"conditions,omitempty\"
patchStrategy:\"merge\" patchMergeKey:\"type\" protobuf:\"bytes,1,rep,name=conditions\"`\n\n\n\t
\ // other fields\n\t}"
properties:
lastTransitionTime:
description: lastTransitionTime is the last time the condition
transitioned from one status to another. This should be when
the underlying condition changed. If that is not known, then
using the time when the API field changed is acceptable.
description: |-
lastTransitionTime is the last time the condition transitioned from one status to another.
This should be when the underlying condition changed. If that is not known, then using the time when the API field changed is acceptable.
format: date-time
type: string
message:
description: message is a human readable message indicating
details about the transition. This may be an empty string.
description: |-
message is a human readable message indicating details about the transition.
This may be an empty string.
maxLength: 32768
type: string
observedGeneration:
description: observedGeneration represents the .metadata.generation
that the condition was set based upon. For instance, if .metadata.generation
is currently 12, but the .status.conditions[x].observedGeneration
is 9, the condition is out of date with respect to the current
state of the instance.
description: |-
observedGeneration represents the .metadata.generation that the condition was set based upon.
For instance, if .metadata.generation is currently 12, but the .status.conditions[x].observedGeneration is 9, the condition is out of date
with respect to the current state of the instance.
format: int64
minimum: 0
type: integer
reason:
description: reason contains a programmatic identifier indicating
the reason for the condition's last transition. Producers
of specific condition types may define expected values and
meanings for this field, and whether the values are considered
a guaranteed API. The value should be a CamelCase string.
description: |-
reason contains a programmatic identifier indicating the reason for the condition's last transition.
Producers of specific condition types may define expected values and meanings for this field,
and whether the values are considered a guaranteed API.
The value should be a CamelCase string.
This field may not be empty.
maxLength: 1024
minLength: 1
Expand All @@ -117,11 +122,12 @@ spec:
- Unknown
type: string
type:
description: type of condition in CamelCase or in foo.example.com/CamelCase.
--- Many .condition.type values are consistent across resources
like Available, but because arbitrary conditions can be useful
(see .node.status.conditions), the ability to deconflict is
important. The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)
description: |-
type of condition in CamelCase or in foo.example.com/CamelCase.
---
Many .condition.type values are consistent across resources like Available, but because arbitrary conditions can be
useful (see .node.status.conditions), the ability to deconflict is important.
The regex it matches is (dns1123SubdomainFmt/)?(qualifiedNameFmt)
maxLength: 316
pattern: ^([a-z0-9]([-a-z0-9]*[a-z0-9])?(\.[a-z0-9]([-a-z0-9]*[a-z0-9])?)*/)?(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])$
type: string
Expand Down
3 changes: 0 additions & 3 deletions config/rbac/role.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,11 +10,8 @@ rules:
- secrets
verbs:
- create
- delete
- get
- list
- patch
- update
- watch
- apiGroups:
- ""
Expand Down
4 changes: 2 additions & 2 deletions docs/developer-guide/helm.md
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ We use GitHub Actions to update the repo.
helm install mysql-operator-0.1.0.tgz --generate-name
```

Optionally, you can add `--set cloudSecretManagerType=gcp --set gcpProjectId=$PROJECT_ID` to use GCP SecretManager to get AdminUser and/or AdminPassword.
Optionally, you can add `--set adminUserSecretType=gcp --set gcpProjectId=$PROJECT_ID` to use GCP SecretManager to get AdminUser and/or AdminPassword.

<details>

Expand Down Expand Up @@ -99,7 +99,7 @@ We use GitHub Actions to update the repo.
```
1. (Optional) upgrade an existing release
```
helm upgrade mysql-operator-0-1680913123 $HELM_PATH --set cloudSecretManagerType=gcp --set gcpProjectId=$PROJECT_ID
helm upgrade mysql-operator-0-1680913123 $HELM_PATH --set adminUserSecretType=gcp --set gcpProjectId=$PROJECT_ID
```
1. Uninstall
```
Expand Down
2 changes: 1 addition & 1 deletion docs/usage/gcp-secretmanager.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ In this example, we'll use Cloud SQL for MySQL, and run mysql-operator on GKE.

```
helm install mysql-operator nakamasato/mysql-operator \
--set cloudSecretManagerType=gcp \
--set adminUserSecretType=gcp \
--set gcpServiceAccount=${SA_NAME}@${PROJECT}.iam.gserviceaccount.com \
--set gcpProjectId=$PROJECT \
--set cloudSQL.instanceConnectionName=$PROJECT:$REGION:$INSTANCE_NAME \
Expand Down
Loading
Loading