Suggestions to omri

docs/configuration/environment_variables.md

@@ -5,56 +5,59 @@ The helm chart populate them using [values.yaml](https://github.com/quickube/pip
 
 ### Git
 
-* GIT_PROVIDER
-  The git provider that Piper will use, possible variables: GitHub (will support bitbucket and gitlab)
+- GIT_PROVIDER
+  The git provider that Piper will use, possible variables: GitHub | Gitlab (will support bitbucket)
 
-* GIT_TOKEN
+- GIT_TOKEN
   The git token that will be used.
 
-* GIT_ORG_NAME
+- GIT_URL
+  the git url that will be used, only relevant when running gitlab self hosted
+
+- GIT_ORG_NAME
   The organization name.
 
-* GIT_ORG_LEVEL_WEBHOOK
+- GIT_ORG_LEVEL_WEBHOOK
   Boolean variable, whether to config webhook in organization level. default `false`
 
-* GIT_WEBHOOK_REPO_LIST
+- GIT_WEBHOOK_REPO_LIST
   List of repositories to configure webhooks to.
 
-* GIT_WEBHOOK_URL
+- GIT_WEBHOOK_URL
   URL of piper ingress, to configure webhooks.
 
-* GIT_WEBHOOK_AUTO_CLEANUP
-  Will cleanup all webhook that were created with piper. 
-  Notice that there will be a race conditions between pod that being terminated and the new one. 
+- GIT_WEBHOOK_AUTO_CLEANUP
+  Will cleanup all webhook that were created with piper.
+  Notice that there will be a race conditions between pod that being terminated and the new one.
 
-* GIT_ENFORCE_ORG_BELONGING
+- GIT_ENFORCE_ORG_BELONGING
   Boolean variable, whether to enforce organizational belonging of git event creator. default `false`
 
-* GIT_FULL_HEALTH_CHECK
+- GIT_FULL_HEALTH_CHECK
   Enables full health check of webhook. Full health check contains expecting and validating ping event from a webhook.
   Doesn't work for bitbucket, because the API call doesn't exists.
 
-
 ### Argo Workflows Server
-* ARGO_WORKFLOWS_TOKEN
+
+- ARGO_WORKFLOWS_TOKEN
   The token of Argo Workflows server.
 
-* ARGO_WORKFLOWS_ADDRESS
+- ARGO_WORKFLOWS_ADDRESS
   The address of Argo Workflows Server.
-
-* ARGO_WORKFLOWS_CREATE_CRD
+- ARGO_WORKFLOWS_CREATE_CRD
   Whether to directly send Workflows instruction or create a CRD in the Cluster.
 
-* ARGO_WORKFLOWS_NAMESPACE
+- ARGO_WORKFLOWS_NAMESPACE
   The namespace of Workflows creation for Argo Workflows.
 
-* KUBE_CONFIG
+- KUBE_CONFIG
   Used to configure Argo Workflows client with local kube configurations.
 
 ### Rookout
-* ROOKOUT_TOKEN
+
+- ROOKOUT_TOKEN
   The token used to configure Rookout agent. If not provided, will not start the agent.
-* ROOKOUT_LABELS 
+- ROOKOUT_LABELS
   The labels to label instances at Rookout, default to "service:piper"
-* ROOKOUT_REMOTE_ORIGIN
-  The repo URL for source code fetching, default:"https://github.com/quickube/piper.git".
+- ROOKOUT_REMOTE_ORIGIN
+  The repo URL for source code fetching, default:"https://github.com/quickube/piper.git".

docs/getting_started/installation.md

@@ -1,16 +1,18 @@
 ## Instalation
 
-Piper should be deployed in the cluster with Argo Workflows. 
-Piper will create a CRD that Argo Workflows will pick, so install or configure Piper to create those CRDs in the right namespace. 
+Piper should be deployed in the cluster with Argo Workflows.
+Piper will create a CRD that Argo Workflows will pick, so install or configure Piper to create those CRDs in the right namespace.
 
 Please check out [values.yaml](https://github.com/quickube/piper/tree/main/helm-chart/values.yaml) file of the helm chart configurations.
 
 To add piper helm repo run:
+
 ```bash
 helm repo add piper https://piper.quickube.com
 ```
 
 After configuring Piper [values.yaml](https://github.com/quickube/piper/tree/main/helm-chart/values.yaml), run the following command for installation:
+
 ```bash
 helm upgrade --install piper piper/piper \
 -f YOUR_VALUES_FILE.yaml
@@ -22,7 +24,7 @@ helm upgrade --install piper piper/piper \
 
 ### Ingress
 
-Piper should listen to webhooks from your git provider. 
+Piper should listen to webhooks from your git provider.
 Expose it using ingress or service, then provide the address to `piper.webhook.url` as followed:
 `https://PIPER_EXPOESED_URL/webhook`
 
@@ -32,9 +34,9 @@ Checkout [values.yaml](https://github.com/quickube/piper/tree/main/helm-chart/va
 
 Piper will use git for fetching `.workflows` folder and receiving events using webhooks.
 
-To pick which git provider you are using provide `gitProvider.name` configuration in helm chart (Now only supports GitHub and Bitbucket).
+To pick which git provider you are using provide `gitProvider.name` configuration in helm chart (supports GitHub, Bitbucket and Gitlab).
 
-Also configure you organization (Github) or workspace (Bitbucket) name using `gitProvider.organization.name` in helm chart.
+Also configure you organization (Github), workspace (Bitbucket) or group (Gitlab) name using `gitProvider.organization.name` in helm chart.
 
 #### Git Token Permissions
 
@@ -44,11 +46,12 @@ For Bitbucket configure `Repositories:read`, `Webhooks:read and write` and `Pull
 
 #### Token
 
-The git token should be passed as secret in the helm chart at `gitProvider.token`. 
+The git token should be passed as secret in the helm chart at `gitProvider.token`.
 Can be passed as parameter in helm install command using `--set piper.gitProvider.token=YOUR_GIT_TOKEN`
 
 Alternatively, you can consume already existing secret and fill up `piper.gipProvider.existingSecret`.
-The key should be name `token`. Can be created using 
+The key should be name `token`. Can be created using
+
 ```bash
 kubectl create secret generic piper-git-token --from-literal=token=YOUR_GIT_OKEN
 ```
@@ -61,13 +64,13 @@ Configure `piper.webhook.url` the address of piper that exposed with ingress wit
 
 For organization level configure: `gitProvider.webhook.orgLevel` to `true`.
 
-For granular repo webhook provide list of repos at: `gitProvider.webhook.repoList`. 
+For granular repo webhook provide list of repos at: `gitProvider.webhook.repoList`.
 
-Piper implements graceful shutdown, it will delete all the webhooks when terminated. 
+Piper implements graceful shutdown, it will delete all the webhooks when terminated.
 
 #### Status check
 
-Piper will handle status checks for you. 
+Piper will handle status checks for you.
 It will notify the GitProvider for the status of Workflow for specific commit that triggered Piper.
 For linking provide valid URL of your Argo Workflows server address at: `argoWorkflows.server.address`
 
@@ -81,4 +84,4 @@ To lint the workflow before submitting it, please configure the internal address
 
 #### Skip CRD Creation (On development)
 
-Piper can communicate directly to Argo Workflow using ARGO_WORKFLOWS_CREATE_CRD environment variable, if you want to skip the creation of CRD change `argoWorkflows.crdCreation` to `false`.
+Piper can communicate directly to Argo Workflow using ARGO_WORKFLOWS_CREATE_CRD environment variable, if you want to skip the creation of CRD change `argoWorkflows.crdCreation` to `false`.

go.mod

@@ -45,6 +45,8 @@ require (
 	github.com/google/uuid v1.3.0 // indirect
 	github.com/gorilla/websocket v1.5.0 // indirect
 	github.com/grpc-ecosystem/grpc-gateway v1.16.0 // indirect
+	github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
+	github.com/hashicorp/go-retryablehttp v0.7.2 // indirect
 	github.com/hashicorp/golang-lru v0.5.4 // indirect
 	github.com/imdario/mergo v0.3.13 // indirect
 	github.com/josharian/intern v1.0.0 // indirect
@@ -67,6 +69,7 @@ require (
 	github.com/tidwall/pretty v1.2.0 // indirect
 	github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
 	github.com/ugorji/go/codec v1.2.11 // indirect
+	github.com/xanzy/go-gitlab v0.105.0 // indirect
 	github.com/yhirose/go-peg v0.0.0-20210804202551-de25d6753cf1 // indirect
 	golang.org/x/arch v0.3.0 // indirect
 	golang.org/x/crypto v0.17.0 // indirect

go.sum

@@ -879,6 +879,11 @@ github.com/grpc-ecosystem/grpc-gateway v1.16.0 h1:gmcG1KaJ57LophUzW0Hy8NmPhnMZb4
 github.com/grpc-ecosystem/grpc-gateway v1.16.0/go.mod h1:BDjrQk3hbvj6Nolgz8mAMFbcEtjT1g+wF4CSlocrBnw=
 github.com/grpc-ecosystem/grpc-gateway/v2 v2.7.0/go.mod h1:hgWBS7lorOAVIJEQMi4ZsPv9hVvWI6+ch50m39Pf2Ks=
 github.com/grpc-ecosystem/grpc-gateway/v2 v2.11.3/go.mod h1:o//XUCC/F+yRGJoPO/VU0GSB0f8Nhgmxx0VIRUvaC0w=
+github.com/hashicorp/go-cleanhttp v0.5.2 h1:035FKYIWjmULyFRBKPs8TBQoi0x6d9G4xc9neXJWAZQ=
+github.com/hashicorp/go-cleanhttp v0.5.2/go.mod h1:kO/YDlP8L1346E6Sodw+PrpBSV4/SoxCXGY6BqNFT48=
+github.com/hashicorp/go-hclog v0.9.2/go.mod h1:5CU+agLiy3J7N7QjHK5d05KxGsuXiQLrjA0H7acj2lQ=
+github.com/hashicorp/go-retryablehttp v0.7.2 h1:AcYqCvkpalPnPF2pn0KamgwamS42TqUDDYFRKq/RAd0=
+github.com/hashicorp/go-retryablehttp v0.7.2/go.mod h1:Jy/gPYAdjqffZ/yFGCFV2doI5wjtH1ewM9u8iYVjtX8=
 github.com/hashicorp/golang-lru v0.5.0/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
 github.com/hashicorp/golang-lru v0.5.1/go.mod h1:/m3WP610KZHVQ1SGc6re/UDhFvYD7pJ4Ao+sR/qLZy8=
 github.com/hashicorp/golang-lru v0.5.4 h1:YDjusn29QI/Das2iO9M0BHnIbxPeyuCHsjMW+lJfyTc=
@@ -1032,6 +1037,8 @@ github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS
 github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
 github.com/ugorji/go/codec v1.2.11 h1:BMaWp1Bb6fHwEtbplGBGJ498wD+LKlNSl25MjdZY4dU=
 github.com/ugorji/go/codec v1.2.11/go.mod h1:UNopzCgEMSXjBc6AOMqYvWC1ktqTAfzJZUZgYf6w6lg=
+github.com/xanzy/go-gitlab v0.105.0 h1:3nyLq0ESez0crcaM19o5S//SvezOQguuIHZ3wgX64hM=
+github.com/xanzy/go-gitlab v0.105.0/go.mod h1:ETg8tcj4OhrB84UEgeE8dSuV/0h4BBL1uOV/qK0vlyI=
 github.com/yhirose/go-peg v0.0.0-20210804202551-de25d6753cf1 h1:7iTmQ0lZwTtfm4XMgP5ezzWMDCjo7GTS0ZgCj6jpVzM=
 github.com/yhirose/go-peg v0.0.0-20210804202551-de25d6753cf1/go.mod h1:q2QWLflHsZxT6ixYcXveTYicEvxGh5Uv6CnI7f7BfjQ=
 github.com/yuin/goldmark v1.1.25/go.mod h1:3hX8gzYuyVAZsxl0MRgGTJEmQBFcNTphYh9decYSb74=

helm-chart/templates/deployment.yaml

@@ -100,6 +100,8 @@ spec:
                 key: token
           - name: GIT_ORG_NAME
             value: {{ .Values.piper.gitProvider.organization.name | quote }}
+          - name: GIT_URL
+            value: {{ .Values.piper.gitProvider.url | quote }}
           - name: GIT_WEBHOOK_URL
             value: {{ .Values.piper.gitProvider.webhook.url | quote }}
           - name: GIT_WEBHOOK_SECRET

helm-chart/values.yaml

@@ -4,7 +4,7 @@
 # Map of Piper configurations.
 piper:
   gitProvider:
-    # -- Name of your git provider (github/bitbucket).
+    # -- Name of your git provider (github/bitbucket/gitlab).
     name: github
     # -- The token for authentication with the Git provider.
     # -- This will create a secret named <RELEASE_NAME>-git-token and with the key 'token'
@@ -13,9 +13,12 @@ piper:
     # -- Reference to existing token with 'token' key.
     # -- can be created with `kubectl create secret generic piper-git-token --from-literal=token=YOUR_TOKEN`
     existingSecret: #piper-git-token
+    # -- git provider url
+    # -- relevant when using gitlab self hosted
+    url: ""
     # Map of organization configurations.
     organization:
-      # -- Name of your Git Organization (GitHub) or Workspace (Bitbucket)
+      # -- Name of your Git Organization (GitHub) / Workspace (Bitbucket) or Group (Gitlab)
       name: ""
     # Map of webhook configurations.
     webhook:
@@ -31,7 +34,7 @@ piper:
       url: "" #https://piper.example.local/webhook
       # -- Whether config webhook on org level (GitHub) or at workspace level (Bitbucket - not supported yet)
       orgLevel: false
-      # -- (Github) Used of orgLevel=false, to configure webhook for each of the repos provided.
+      # -- (Github/Gitlab) Used of orgLevel=false, to configure webhook for each of the repos provided.
       repoList: []
 
   # Map of Argo Workflows configurations.
@@ -51,8 +54,9 @@ piper:
       existingSecret: #piper-argo-token
     # -- Whether create Workflow CRD or send direct commands to Argo Workflows server.
     crdCreation: true
-
-  workflowsConfig: {}
+
+  workflowsConfig:
+    {}
     # default: |
     #   spec:
     #     volumes:
@@ -126,22 +130,21 @@ podAnnotations: {}
 
 # -- Security Context to set on the pod level
 podSecurityContext:
-   fsGroup: 1001
-   runAsUser: 1001
-   runAsGroup: 1001
+  fsGroup: 1001
+  runAsUser: 1001
+  runAsGroup: 1001
 
 # -- Security Context to set on the container level
 securityContext:
   runAsUser: 1001
   capabilities:
     drop:
-     - ALL
+      - ALL
   readOnlyRootFilesystem: true
   runAsNonRoot: true
 
-
 service:
-    # -- Sets the type of the Service
+  # -- Sets the type of the Service
   type: ClusterIP
   # -- Service port
   # For TLS mode change the port to 443
@@ -158,10 +161,11 @@ ingress:
   # -- Piper ingress class name
   className: ""
   # -- Piper ingress annotations
-  annotations: {}
+  annotations:
+    {}
     # kubernetes.io/ingress.class: nginx
     # kubernetes.io/tls-acme: "true"
-  
+
   # -- Piper ingress hosts
   ## Hostnames must be provided if Ingress is enabled.
   hosts:
@@ -215,4 +219,4 @@ volumes: []
 volumeMounts: []
 
 # -- Specify postStart and preStop lifecycle hooks for Piper container
-lifecycle: {}
+lifecycle: {}

pkg/conf/git_provider.go

@@ -1,21 +1,22 @@
 package conf
 
 import (
-	"fmt"
+	"fmt" 
 
 	"github.com/kelseyhightower/envconfig"
 )
 
 type GitProviderConfig struct {
 	Provider           string `envconfig:"GIT_PROVIDER" required:"true"`
 	Token              string `envconfig:"GIT_TOKEN" required:"true"`
-	OrgName            string `envconfig:"GIT_ORG_NAME" required:"true"`
+	Url			       string `envconfig:"GIT_URL" required:"false"`
+	OrgName           string `envconfig:"GIT_ORG_NAME" required:"true"`
 	OrgLevelWebhook    bool   `envconfig:"GIT_ORG_LEVEL_WEBHOOK" default:"false" required:"false"`
 	RepoList           string `envconfig:"GIT_WEBHOOK_REPO_LIST" required:"false"`
 	WebhookURL         string `envconfig:"GIT_WEBHOOK_URL" required:"false"`
 	WebhookSecret      string `envconfig:"GIT_WEBHOOK_SECRET" required:"false"`
 	WebhookAutoCleanup bool   `envconfig:"GIT_WEBHOOK_AUTO_CLEANUP" default:"false" required:"false"`
-  EnforceOrgBelonging bool   `envconfig:"GIT_ENFORCE_ORG_BELONGING" default:"false" required:"false"`
+    EnforceOrgBelonging bool   `envconfig:"GIT_ENFORCE_ORG_BELONGING" default:"false" required:"false"`
 	OrgID               int64
 	FullHealthCheck    bool   `envconfig:"GIT_FULL_HEALTH_CHECK" default:"false" required:"false"`
 }

pkg/git_provider/github.go

@@ -22,6 +22,7 @@ func NewGithubClient(cfg *conf.GlobalConfig) (Client, error) {
 	ctx := context.Background()
 
 	client := github.NewTokenClient(ctx, cfg.GitProviderConfig.Token)
+
 	err := ValidatePermissions(ctx, client, cfg)
 	if err != nil {
 		return nil, fmt.Errorf("failed to validate permissions: %v", err)