Skip to content

Commit

Permalink
add UpdatePolicy to config
Browse files Browse the repository at this point in the history
  • Loading branch information
@mattes
mattes committed Oct 2, 2020
1 parent 656f52c commit e80fd42
Show file tree
Hide file tree
Showing 7 changed files with 165 additions and 27 deletions.
2 changes: 1 addition & 1 deletion Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ example:
go build -mod vendor
(cd example && \
INPUT_CREDS=.google_application_credentials.json \
GITHUB_RUN_NUMBER=140 \
GITHUB_RUN_NUMBER=170 \
GITHUB_SHA=13e82dd30df4e87118faa98712a5aebb0ab05c45 \
../gce-deploy-action)

Expand Down
42 changes: 25 additions & 17 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -35,29 +35,37 @@ deploys:
github-sha: $GITHUB_SHA
tags:
- my-tag123
update_policy:
min_ready_sec: 30

delete_instance_templates_after: false
```
### Config Reference
| Variable | Description |
|--------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `deploys.*.name` | ***Required*** Name of the deploy |
| `deploys.*.project` | Name of the Google Cloud project |
| `deploys.*.creds` | Either a path or the contents of a Service Account JSON Key. Required, if not specified in Github action. |
| `deploys.*.region` | ***Required*** Region of the instance group. |
| `deploys.*.instance_group` | ***Required*** Name of the instance group. |
| `deploys.*.instance_template_base` | ***Required*** Instance template to be used as base. |
| `deploys.*.instance_template` | ***Required*** Name of the newly created instance template. |
| `deploys.*.startup_script` | Path to script to run when VM boots. [Read more](https://cloud.google.com/compute/docs/startupscript) |
| `deploys.*.shutdown_script` | Path to script to run when VM shuts down. [Read more](https://cloud.google.com/compute/docs/shutdownscript) |
| `deploys.*.cloud_init` | Path to cloud-init file. [Read more](https://cloud.google.com/container-optimized-os/docs/how-to/create-configure-instance#using_cloud-init) |
| `deploys.*.labels` | A set of key/value label pairs to assign to instances. |
| `deploys.*.metadata` | A set of key/value metadata pairs to make available from within instances. |
| `deploys.*.tags` | A list of tags to assign to instances. |
| `deploys.*.vars` | A set of additional key/value variables which will be available in either startup_script, shutdown_script or cloud_init. They take precedence over ENV vars. |
| `delete_instance_templates_after` | Delete old instance templates after duration, default '336h' (14 days). Set to 'false' to disable. |
| Variable | Description |
|--------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `deploys.*.name` | ***Required*** Name of the deploy |
| `deploys.*.project` | Name of the Google Cloud project |
| `deploys.*.creds` | Either a path or the contents of a Service Account JSON Key. Required, if not specified in Github action. |
| `deploys.*.region` | ***Required*** Region of the instance group. |
| `deploys.*.instance_group` | ***Required*** Name of the instance group. |
| `deploys.*.instance_template_base` | ***Required*** Instance template to be used as base. |
| `deploys.*.instance_template` | ***Required*** Name of the newly created instance template. |
| `deploys.*.startup_script` | Path to script to run when VM boots. [Read more](https://cloud.google.com/compute/docs/startupscript) |
| `deploys.*.shutdown_script` | Path to script to run when VM shuts down. [Read more](https://cloud.google.com/compute/docs/shutdownscript) |
| `deploys.*.cloud_init` | Path to cloud-init file. [Read more](https://cloud.google.com/container-optimized-os/docs/how-to/create-configure-instance#using_cloud-init) |
| `deploys.*.labels` | A set of key/value label pairs to assign to instances. |
| `deploys.*.metadata` | A set of key/value metadata pairs to make available from within instances. |
| `deploys.*.tags` | A list of tags to assign to instances. |
| `deploys.*.vars` | A set of additional key/value variables which will be available in either startup_script, shutdown_script or cloud_init. They take precedence over ENV vars. |
| `deploys.*.update_policy.type` | The type of update process, must be either `PROACTIVE` (default) or `OPPORTUNISTIC`. [Read more](https://cloud.google.com/compute/docs/instance-groups/rolling-out-updates-to-managed-instance-groups#starting_an_opportunistic_or_proactive_update) |
| `deploys.*.update_policy.replacement_method` | What action should be used to replace instances, must be either `SUBSTITUTE` (default) or `RECREATE`. [Read more](https://cloud.google.com/compute/docs/instance-groups/rolling-out-updates-to-managed-instance-groups#replacement_method) |
| `deploys.*.update_policy.minimal_action` | Minimal action to be taken on an instance, possible values are `NONE`, `REFRESH`, `REPLACE` (default) or `RESTART`. [Read more](https://cloud.google.com/compute/docs/instance-groups/rolling-out-updates-to-managed-instance-groups#minimal_action) |
| `deploys.*.update_policy.min_ready_sec` | Time to wait between consecutive instance updates, default is 10 seconds. [Read more](https://cloud.google.com/compute/docs/instance-groups/updating-managed-instance-groups#minimum_wait_time) |
| `deploys.*.update_policy.max_surge` | Maximum number (or percentage, i.e. `15%`) of temporary instances to add while updating. Default is 3. [Read more](https://cloud.google.com/compute/docs/instance-groups/updating-managed-instance-groups#max_surge) |
| `deploys.*.update_policy.max_unavailable` | Maximum number (or percentage, i.e. `100%`) of instances that can be offline at the same time while updating. Default is 0. [Read more](https://cloud.google.com/compute/docs/instance-groups/updating-managed-instance-groups#max_unavailable) |
| `delete_instance_templates_after` | Delete old instance templates after duration, defaults to `336h` (14 days). Set to `false` to disable. |


### Variables
Expand Down
86 changes: 86 additions & 0 deletions config.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,21 @@ type Deploy struct {
Labels map[string]string `yaml:"labels"`
Metadata map[string]string `yaml:"metadata"`
Tags []string `yaml:"tags"`
UpdatePolicy UpdatePolicy `yaml:"update_policy"`
}

type UpdatePolicy struct {
Type string `yaml:"type"`
ReplacementMethod string `yaml:"replacement_method"`
MinimalAction string `yaml:"minimal_action"`
MinReadySec string `yaml:"min_ready_sec"`
minReadySec int
MaxSurge string `yaml:"max_surge"`
maxSurge int
maxSurgeInPercent bool
MaxUnavailable string `yaml:"max_unavailable"`
maxUnavailable int
maxUnavailableInPercent bool
}

func ParseConfig(b io.Reader) (*Config, error) {
Expand Down Expand Up @@ -173,6 +188,77 @@ func ParseConfig(b io.Reader) (*Config, error) {
for j := range dy.Tags {
dy.Tags[j] = expandShellRe(dy.Tags[j], getEnv(nil))
}

// expand vars in update policy
dy.UpdatePolicy.Type = expandShellRe(dy.UpdatePolicy.Type, getEnv(nil))
dy.UpdatePolicy.MinimalAction = expandShellRe(dy.UpdatePolicy.MinimalAction, getEnv(nil))
dy.UpdatePolicy.ReplacementMethod = expandShellRe(dy.UpdatePolicy.ReplacementMethod, getEnv(nil))
dy.UpdatePolicy.MinReadySec = expandShellRe(dy.UpdatePolicy.MinReadySec, getEnv(nil))
dy.UpdatePolicy.MaxSurge = expandShellRe(dy.UpdatePolicy.MaxSurge, getEnv(nil))
dy.UpdatePolicy.MaxUnavailable = expandShellRe(dy.UpdatePolicy.MaxUnavailable, getEnv(nil))

if strings.TrimSpace(dy.UpdatePolicy.Type) == "" {
dy.UpdatePolicy.Type = "PROACTIVE"
}

if strings.TrimSpace(dy.UpdatePolicy.MinimalAction) == "" {
dy.UpdatePolicy.MinimalAction = "REPLACE"
}

if strings.TrimSpace(dy.UpdatePolicy.ReplacementMethod) == "" {
dy.UpdatePolicy.ReplacementMethod = "SUBSTITUTE"
}

// parse update policy vars
if dy.UpdatePolicy.MinReadySec != "" {
minReadySec, err := strconv.Atoi(dy.UpdatePolicy.MinReadySec)
if err != nil {
return nil, fmt.Errorf("update_policy.min_ready_sec: %v", err)
}
dy.UpdatePolicy.minReadySec = minReadySec
} else {
dy.UpdatePolicy.minReadySec = 10 // set default
}

if dy.UpdatePolicy.MaxSurge != "" {
dy.UpdatePolicy.MaxSurge = strings.TrimSpace(dy.UpdatePolicy.MaxSurge)
if strings.HasSuffix(dy.UpdatePolicy.MaxSurge, "%") {
maxSurge, err := strconv.Atoi(strings.TrimSuffix(dy.UpdatePolicy.MaxSurge, "%"))
if err != nil {
return nil, fmt.Errorf("update_policy.max_surge: %v", err)
}
dy.UpdatePolicy.maxSurge = maxSurge
dy.UpdatePolicy.maxSurgeInPercent = true
} else {
maxSurge, err := strconv.Atoi(dy.UpdatePolicy.MaxSurge)
if err != nil {
return nil, fmt.Errorf("update_policy.max_surge: %v", err)
}
dy.UpdatePolicy.maxSurge = maxSurge
}
} else {
dy.UpdatePolicy.maxSurge = 3 // set default
}

if dy.UpdatePolicy.MaxUnavailable != "" {
dy.UpdatePolicy.MaxUnavailable = strings.TrimSpace(dy.UpdatePolicy.MaxUnavailable)
if strings.HasSuffix(dy.UpdatePolicy.MaxUnavailable, "%") {
maxUnavailable, err := strconv.Atoi(strings.TrimSuffix(dy.UpdatePolicy.MaxUnavailable, "%"))
if err != nil {
return nil, fmt.Errorf("update_policy.max_unavailable: %v", err)
}
dy.UpdatePolicy.maxUnavailable = maxUnavailable
dy.UpdatePolicy.maxUnavailableInPercent = true
} else {
maxUnavailable, err := strconv.Atoi(dy.UpdatePolicy.MaxUnavailable)
if err != nil {
return nil, fmt.Errorf("update_policy.max_unavailable: %v", err)
}
dy.UpdatePolicy.maxUnavailable = maxUnavailable
}
} else {
dy.UpdatePolicy.maxUnavailable = 0 // set default
}
}

// read contents of scripts and expand env vars
Expand Down
22 changes: 22 additions & 0 deletions config_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,19 @@ deploys:
metadatakey: metadatavalue-$BAR-${BAR}
tags:
- tagvalue-$BAR-${BAR}
update_policy:
type: type-$BAR-${BAR}
minimal_action: minimal-action-$BAR-${BAR}
replacement_method: replacement-method-$BAR-${BAR}
min_ready_sec: $MIN_READY_SEC
max_surge: $MAX_SURGE
max_unavailable: $MAX_UNAVAILABLE
`

environ = append(environ, "BAR=FOO")
environ = append(environ, "MIN_READY_SEC=2")
environ = append(environ, "MAX_SURGE=15%")
environ = append(environ, "MAX_UNAVAILABLE=14")
c, err := ParseConfig(strings.NewReader(config))
require.NoError(t, err)

Expand Down Expand Up @@ -72,6 +82,18 @@ deploys:
assert.Equal(t, "labelvalue-FOO-FOO", c.Deploys[0].Labels["labelkey"])
assert.Equal(t, "metadatavalue-FOO-FOO", c.Deploys[0].Metadata["metadatakey"])
assert.Equal(t, "tagvalue-FOO-FOO", c.Deploys[0].Tags[0])

assert.Equal(t, "type-FOO-FOO", c.Deploys[0].UpdatePolicy.Type)
assert.Equal(t, "minimal-action-FOO-FOO", c.Deploys[0].UpdatePolicy.MinimalAction)
assert.Equal(t, "replacement-method-FOO-FOO", c.Deploys[0].UpdatePolicy.ReplacementMethod)
assert.Equal(t, "2", c.Deploys[0].UpdatePolicy.MinReadySec)
assert.Equal(t, 2, c.Deploys[0].UpdatePolicy.minReadySec)
assert.Equal(t, "15%", c.Deploys[0].UpdatePolicy.MaxSurge)
assert.Equal(t, 15, c.Deploys[0].UpdatePolicy.maxSurge)
assert.Equal(t, true, c.Deploys[0].UpdatePolicy.maxSurgeInPercent)
assert.Equal(t, "14", c.Deploys[0].UpdatePolicy.MaxUnavailable)
assert.Equal(t, 14, c.Deploys[0].UpdatePolicy.maxUnavailable)
assert.Equal(t, false, c.Deploys[0].UpdatePolicy.maxUnavailableInPercent)
}

func TestExpandShellRe(t *testing.T) {
Expand Down
15 changes: 13 additions & 2 deletions deploy.go
View file
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package main

import (
"fmt"
"net/http"

computeBeta "google.golang.org/api/compute/v0.beta"
Expand Down Expand Up @@ -55,13 +56,23 @@ func Run(githubActionConfig *GithubActionConfig, config *Config, deploy Deploy)

Infof("%v: Created new instance template '%v/%v'", deploy.Name, deploy.Project, deploy.InstanceTemplate)

maxSurge := fmt.Sprintf("%v", deploy.UpdatePolicy.maxSurge)
if deploy.UpdatePolicy.maxSurgeInPercent {
maxSurge += "%"
}
maxUnavailable := fmt.Sprintf("%v", deploy.UpdatePolicy.maxUnavailable)
if deploy.UpdatePolicy.maxUnavailableInPercent {
maxUnavailable += "%"
}

Infof("%v: Started rolling deploy for instance group '%v/%v' with Update Type: %v, Minimal Action: %v, Replacement Method: %v, Min Ready: %vsec, Max Surge: %v, Max Unavailable: %v",
deploy.Name, deploy.Project, deploy.InstanceGroup, deploy.UpdatePolicy.Type, deploy.UpdatePolicy.MinimalAction, deploy.UpdatePolicy.ReplacementMethod, deploy.UpdatePolicy.minReadySec, maxSurge, maxUnavailable)

// start rolling update via instance group manager
if err := StartRollingUpdate(computeBetaService, deploy, instanceTemplateURL); err != nil {
return err
}

Infof("%v: Started rolling deploy for instance group '%v/%v'", deploy.Name, deploy.Project, deploy.InstanceGroup)

if config.deleteInstanceTemplatesAfter > 0 {
if err := CleanupInstanceTemplates(computeService, deploy.Project, config.deleteInstanceTemplatesAfter); err != nil {
LogWarning(err.Error(), map[string]string{"project": deploy.Project})
Expand Down
2 changes: 2 additions & 0 deletions example/deploy.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,6 @@ deploys:
- app123
metadata:
github_run_number: $GITHUB_RUN_NUMBER
update_policy:
min_ready_sec: 20

Loading

0 comments on commit e80fd42

Please sign in to comment.