Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merging Go Driver branch into main #79

Merged
merged 76 commits into from
Jan 16, 2025
Merged
Show file tree
Hide file tree
Changes from 74 commits
Commits
Show all changes
76 commits
Select commit Hold shift + click to select a range
ce03c37
adding basic go skele
zjrgov Nov 14, 2024
2a2abc6
adding simple makefile
zjrgov Nov 14, 2024
b5895ef
feat: parse VCAP & use to connect with cloud.gov
zjrgov Nov 14, 2024
c7fc20b
feat: parse creds from VCAP_SERVICES env var
zjrgov Nov 15, 2024
8c407b5
refactor: simplify GetCredentials test
zjrgov Nov 15, 2024
ab70e74
refactor: better error handling
zjrgov Nov 19, 2024
50eaa2a
refactor: slightly better read flow for getting config
zjrgov Nov 19, 2024
32f98a4
chore: add standard ignores for Go
zjrgov Nov 19, 2024
bfea6b8
chore: move go code into its proper place
zjrgov Nov 21, 2024
d6be17b
refactor: simplify error check for json.Unmarshal
zjrgov Nov 25, 2024
e749436
test: more robust testing of GetCredentials
zjrgov Nov 25, 2024
49d3151
test: skip Test_main because it's just meant for experiments right now
zjrgov Nov 25, 2024
5fb7f30
refactor: decouple GetCfClient from GetCredentials
zjrgov Nov 25, 2024
67d8249
feat: allow explicit config of CF user/pass w/ envvars
zjrgov Nov 25, 2024
18f9f3f
fix: move & comment out the CfClient test that snuck in
zjrgov Nov 25, 2024
b76d62f
refactor: Credentials -> CfCredentials for specificity
zjrgov Nov 25, 2024
13151dd
refactor: move Makefile & go gitignore
zjrgov Nov 25, 2024
d59e1dd
feat: add godotenv, load in main
zjrgov Dec 4, 2024
4c60148
refactor: wrapping cfclient poc
zjrgov Dec 4, 2024
0aac409
refactor: move relevant cloud.gov logic w/ adapter for cf-goclient
zjrgov Dec 6, 2024
450af89
refactor: move creds code into its own file
zjrgov Dec 6, 2024
e5a7e55
chore: add launch.json for debug config
zjrgov Dec 6, 2024
f74cfd5
refactor: colocate creds tests and fix names
zjrgov Dec 9, 2024
7cc5d7c
fix: add json tag for CGSrvAct.Creds, make isEmpty an OR
zjrgov Dec 9, 2024
6e47235
chore: rearrange Test_getCreds fields for better data alignment
zjrgov Dec 10, 2024
59bb468
test: add tests & test stubs for CG
zjrgov Dec 10, 2024
1b705bc
test: adding tests for rest of the CG stubs
zjrgov Dec 11, 2024
a110cb7
fix: issue with nil creds
zjrgov Dec 11, 2024
b2ae777
chore: delete commented out stub test from main_test.go
zjrgov Dec 11, 2024
97f35aa
refactor: Adapter -> CloudI, more descriptive and fun
zjrgov Dec 12, 2024
9a3c0c4
chore: better error msg for go-cfclient.conn()
zjrgov Dec 12, 2024
34fe7a5
refactor: extract the cf-client adapter's app casting
zjrgov Dec 12, 2024
fcda405
style: indirect deps should be in own require block
zjrgov Dec 12, 2024
8ad38f7
refactor: more descriptive name for cf_adapter file
zjrgov Dec 12, 2024
c9ed52e
style: the default root url should prob be a const
zjrgov Dec 12, 2024
f1ccccf
style: short CloudI shoulda been `i`
zjrgov Dec 12, 2024
8ebc0b4
feat: add test target to Makefile
zjrgov Dec 13, 2024
ab56585
fix: clear tested env vars before testing creds.go
zjrgov Dec 13, 2024
d936757
docs: add README.md to the go driver with info RE deps & testing
zjrgov Dec 16, 2024
53c7af5
chore: update the go.sum
zjrgov Dec 16, 2024
6da90b1
chore: executor should be driver!
zjrgov Dec 16, 2024
5c5d7a0
feat: add basic CI workflow for cf-driver-go
zjrgov Dec 18, 2024
de7b06f
fix: using the right file path would help
zjrgov Dec 18, 2024
c04918c
fix: how about setting a working directory
zjrgov Dec 18, 2024
07a2d08
feat: have the go fmt step check for diffs
zjrgov Dec 18, 2024
6e2ff19
chore: spec a go version in case GitHub likes that better
zjrgov Dec 18, 2024
45e2b16
chore: set a specific step to install deps
zjrgov Dec 18, 2024
3cfa5d2
fix: supply setup-go w/ go.sum path
zjrgov Dec 18, 2024
64dcab3
chore: test failing go fmt
zjrgov Dec 18, 2024
dda1a50
fix: gofmt doesn't take the ./... style arg
zjrgov Dec 18, 2024
629f28d
fix: foiled by quoting!
zjrgov Dec 18, 2024
4436697
chore: fix the formatting
zjrgov Dec 18, 2024
f2e842e
chore: validate that go vet will fail
zjrgov Dec 18, 2024
e1ba619
chore: remove my go vet break
zjrgov Dec 18, 2024
d87eb3d
chore: add a bug to validate test failure
zjrgov Dec 18, 2024
7f8ba86
Revert "chore: add a bug to validate test failure"
zjrgov Dec 18, 2024
ec98588
feat: upgrade setup-go to v5
zjrgov Dec 18, 2024
287cbb4
feat: get go version for action from go.mod
zjrgov Dec 18, 2024
e717240
chore: not necessary to pin a patch version for go
zjrgov Dec 18, 2024
6e69483
feat: add integration test for go driver's CG
zjrgov Dec 18, 2024
c31c6f2
fix: have to name package pkgname_test
zjrgov Dec 19, 2024
874c440
fix: forgot t.Error/t.Fail doesn't exit
zjrgov Dec 19, 2024
ac02ee5
docs: note about integration test in readme, helpful error message
zjrgov Dec 19, 2024
4b6dc62
fix: prevent caching integration test which defeats the purpose
zjrgov Dec 19, 2024
f6e41b1
feat: structure as CLI tool, with `drive` subcmd for each stage
zjrgov Dec 26, 2024
aea1f0a
chore: should have run `tidy`
zjrgov Dec 26, 2024
e35c5d9
chore: trigger pr workflow
zjrgov Dec 27, 2024
fa02c73
refactor: rename cf-driver to cfd
zjrgov Dec 27, 2024
899d25a
fix: forgot to update path in GH action
zjrgov Dec 27, 2024
4e789fb
feat: add types & functions to parse env config
zjrgov Jan 15, 2025
f12009b
refactor: should have used pointers
zjrgov Jan 15, 2025
736d2f0
refactor: use more descriptive variable names
zjrgov Jan 16, 2025
c2abd96
chore: delete unused parse for CF_DOCKER_PASSWORD
zjrgov Jan 16, 2025
2c65099
chore: delete unused parse for CF_DOCKER_PASSWORD, Pt II
zjrgov Jan 16, 2025
9d31e9b
chore: forgot the runner dir moved!
zjrgov Jan 16, 2025
1647675
fix: tell go driver workflow to look in runner-manager
zjrgov Jan 16, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 37 additions & 0 deletions .github/workflows/cf-driver-go.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# This workflow will build a golang project
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-go

name: "CF Driver: Go Build & Test"

on: [pull_request]

defaults:
run:
working-directory: runner/cfd

jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- name: Set up Go
uses: actions/setup-go@v5
with:
go-version-file: runner/cfd/go.mod
cache-dependency-path: runner/cfd/go.sum

- name: Install dependencies
run: go get .

- name: Check formatting
run: test -z "$(gofmt -l .)"

- name: Vet
run: go vet ./...

- name: Build
run: go build -v ./...

- name: Test
run: go test -v ./...
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -36,3 +36,6 @@ override.tf.json

# Include tfplan files to ignore the plan output of command: terraform plan -out=tfplan
# example: *tfplan*

**/testdata/.*
!**/testdata/.*.sample
22 changes: 22 additions & 0 deletions runner/cfd/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# Binaries for programs and plugins
*.exe
*.exe~
*.dll
*.so
*.dylib

# Test binary, built with `go test -c`
*.test

# Output of the go coverage tool, specifically when used with LiteIDE
*.out

# Dependency directories (remove the comment below to include it)
# vendor/

# Go workspace file
go.work
go.work.sum

# env file
.env
11 changes: 11 additions & 0 deletions runner/cfd/.vscode/launch.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
{
"version": "0.2.0",
"configurations": [
{
"name": "Debug main.go",
"type": "go",
"request": "launch",
"program": "main.go"
}
]
}
17 changes: 17 additions & 0 deletions runner/cfd/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
.DEFAULT_GOAL := build

.PHONY:fmt vet build test integration
fmt:
go fmt ./...

vet: fmt
go vet ./...

test: vet
go test ./...

integration: vet
go test -count=1 --tags=integration ./...

build: vet
go build
62 changes: 62 additions & 0 deletions runner/cfd/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# CloudFoundry (cloud.gov) driver

This is a _driver_ developed for use with `gitlab-runner`’s [Custom executor](https://docs.gitlab.com/runner/executors/custom.html) to prepare, run, and clean up runner managers, workers, and services.

## Go setup

### Install Go

You can use Homebrew to install Go on MacOS (it is generally unnecessary to use a version manager). You could also download Go packages for Linux, Mac, and Windows from [Go's website](https://go.dev/doc/install).

```sh
brew install go
```

### Manage dependencies

Manage project dependencies with `go` & `go mod`. Go programs are divided first into _modules_ and then into _packages_. Dependencies are listed in `./go.mod`.

#### To download dependencies

To download deps as listed in `./go.mod`, run:

```sh
go mod download
```

Or, to walk the project and as needed install & remove modules while updating `./go.mod` & `./go.sum` accordingly:

```sh
go mod tidy
```

#### To `get` new dependencies

While `go mod tidy` can install dependencies you've already imported in your packages, you can also install them explicitly:

```sh
go get github.com/google/some-mod/pkg
```

## Running tests

### Unit tests

The simplest way to run tests—or the one with the least typing, at least—is with `make`.

```sh
make test
```

### Integration tests

We only have one integration test right now, and to get it running you'll need to do a bit of local setup.

1. You will need to first get a username & password for some space on cloud.gov that has at least one app.
1. Then you can add those credentials to `./cg/testdata/.cg_creds` in the style of the `.cg_creds.sample` file there.
1. Run the test with `make integration`, which should give you an error and, in its output, show you what the resulting JSON looks like.
1. Copy that JSON result over to the last line of your `.cg_creds` file and run `make integration` again, this time it should succeed.

## Builds

You can also run a build with `make` (`make build`, or simply `make`), but it won't do you much good because we aren't set up to do anything with an executable yet.
51 changes: 51 additions & 0 deletions runner/cfd/cg/cf_adapter.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
package cg

import (
"context"

"github.com/cloudfoundry/go-cfclient/v3/client"
"github.com/cloudfoundry/go-cfclient/v3/config"
"github.com/cloudfoundry/go-cfclient/v3/resource"
)

type GoCFClientAdapter struct {
_con *client.Client
}

func (cf *GoCFClientAdapter) connect(url string, creds *Creds) error {
cfg, err := config.New(url, config.UserPassword(creds.Username, creds.Password))
if err != nil {
return err
}

con, err := client.New(cfg)
if err != nil {
return err
}

cf._con = con
return nil
}

func (cf *GoCFClientAdapter) conn() *client.Client {
if cf._con != nil {
return cf._con
}
panic("go-cfclient adapter is not connected")
}

func castApps(apps []*resource.App) []*App {
Apps := make([]*App, len(apps))
for idx, app := range apps {
Apps[idx] = &(App{app.GUID, app.Name, app.State})
}
return Apps
}

func (cf *GoCFClientAdapter) getApps() ([]*App, error) {
apps, err := cf.conn().Applications.ListAll(context.Background(), nil)
if err != nil {
return nil, err
}
return castApps(apps), nil
}
78 changes: 78 additions & 0 deletions runner/cfd/cg/cg.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
package cg

type App struct {
Id string
Name string
State string
}

// Stuff we'll need to implement, for ref
//
// mapRoute()
//
// addNetworkPolicy()
// removeNetworkPolicy()
//
// appGet()
// appCmd()
// appPush()
// appDelete()
type CloudI interface {
getApps() (apps []*App, err error)
connect(url string, creds *Creds) error
}

type CredI interface {
getCreds() (*Creds, error)
}

type Opts struct {
CredI
Creds *Creds

APIRootURL string
}

type CG struct {
CloudI
*Opts
}

const apiRootURLDefault = "https://api.fr.cloud.gov"

func New(i CloudI, o *Opts) (*CG, error) {
if o == nil {
o = &Opts{CredI: EnvCredsGetter{}}
}
cg := &CG{i, o}
return cg.Connect()
}

func (c *CG) apiRootURL() string {
if c.APIRootURL == "" {
return apiRootURLDefault
}
return c.APIRootURL
}

func (c *CG) creds() (*Creds, error) {
if c.Creds.isEmpty() {
return c.getCreds()
}
return c.Creds, nil
}

func (c *CG) Connect() (*CG, error) {
creds, err := c.creds()
if err != nil {
return nil, err
}
if err := c.connect(c.apiRootURL(), creds); err != nil {
return nil, err
}
return c, nil
}

func (c *CG) GetApps() ([]*App, error) {
return c.getApps()
}
87 changes: 87 additions & 0 deletions runner/cfd/cg/cg_integration_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
//go:build integration

package cg_test

import (
"bufio"
"encoding/json"
"os"
"testing"

"github.com/GSA-TTS/gitlab-runner-cloudgov/runner/cfd/cg"
"github.com/google/go-cmp/cmp"
)

func Test_CFAdapter_GetApps(t *testing.T) {
var u, p, want string
var l int

path := "./testdata/.cg_creds"
f, err := os.Open(path)
if err != nil {
t.Errorf(
"Error opening testdata file = %v\n\033[1;33mDid you forget to create `%v`?\033[0m",
err, path,
)
return
}
defer f.Close()

scanner := bufio.NewScanner(f)

scanning:
for scanner.Scan() {
text := scanner.Text()

if text[0] == '#' {
continue
}

switch l {
case 0:
u = text
case 1:
p = text
case 2:
want = text
break scanning
}

l++
}

if err := scanner.Err(); err != nil {
t.Errorf("Error scanning testdata file = %v", err)
return
}

if u == "" || p == "" || want == "" {
t.Error("Could not load variables from testdata")
return
}

cgClient, err := cg.New(&cg.GoCFClientAdapter{}, &cg.Opts{
Creds: &cg.Creds{Username: u, Password: p},
})
if err != nil {
t.Errorf("Error getting cgClient = %v", err)
return
}

apps, err := cgClient.GetApps()
if err != nil {
t.Errorf("Error running GetApps() = %v", err)
return
}

got, err := json.Marshal(apps)
if err != nil {
t.Errorf("Error marshalling apps to json = %v", err)
return
}

if diff := cmp.Diff(string(got), want); diff != "" {
t.Errorf("mismatch (-got +want):\n%s", diff)
return
}
}
Loading
Loading