Convert pipelines to obsah

bin/forklift

@@ -0,0 +1,51 @@
+#!/usr/bin/env python3
+# PYTHON_ARGCOMPLETE_OK
+
+from pathlib import Path
+
+import obsah
+
+
+class ApplicationConfig(obsah.ApplicationConfig):
+    """
+    A class describing the where to find various files
+    """
+
+    @staticmethod
+    def name() -> str:
+        """
+        Return the name as shown to the user in the ArgumentParser
+        """
+        return 'forklift'
+
+    @staticmethod
+    def target_name() -> str:
+        """
+        Return the name of the target in the playbook if the playbook takes a parameter.
+        """
+        return 'pipeline'
+
+    @staticmethod
+    def data_path() -> Path:
+        """
+        Returns the root of the project
+        """
+        return Path(__file__).absolute().parent.parent
+
+    @classmethod
+    def playbooks_path(cls) -> str:
+        """
+        Return the data path. Houses playbooks and configs.
+        """
+        return (cls.data_path() / 'pipelines').as_posix()
+
+    @classmethod
+    def inventory_path(cls) -> str:
+        """
+        Return the inventory path
+        """
+        return (cls.data_path() / 'package_manifest.yaml').as_posix()
+
+
+if __name__ == '__main__':
+    obsah.main(application_config=ApplicationConfig)

docs/testing.md

@@ -61,57 +61,62 @@ centos7-foreman-bats-ci:
 
 ## Pipeline Testing
 
-Under `pipelines` are a series of playbooks designed around testing scenarios for various version of the Foreman and Katello stack. To run one:
+Under `pipelines` are a series of playbooks designed around testing scenarios for various version of the Foreman and Katello stack. Use `./bin/forklift --help` to find out which ones and their aguments.
 
-    ansible-playbook pipelines/<pipeline>.yml -e forklift_state=up -e <vars required by pipeline>
+```console
+$ ./bin/forklift --help
+usage: forklift [-h] action ...
 
-When you are finished with the test, you can tear down the associated infrastructure:
+positional arguments:
+  action      which action to execute
+    install   Run an install pipeline
+    upgrade   Run an upgrade pipeline
 
-    ansible-playbook pipelines/<pipeline>.yml -e forklift_state=destroy -e <vars required by pipeline>
-
-### Existing Pipelines
-
-* `install_pipeline` - Installs a Server and a Proxy VMs and runs the `foreman_testing` role to verify the setup.
-  Expects the `pipeline_os` variable to be set to a known OS (currently: centos7, debian9)
-  Expects the `pipeline_type` variable to be set to a known type (currently: foreman, katello, luna)
-  Expects the `pipeline_version` variable to be set to a known version (currently: 3.8, 3.9, 3.10, 3.11, nightly)
-* `upgrade_pipeline` - Installs a VM, upgrades it twice and runs the `foreman_testing` role to verify the final upgrade.
-  Expects the `pipeline_os` variable to be set to a known OS (currently: centos7, debian9)
-  Expects the `pipeline_type` variable to be set to a known type (currently: foreman, katello, luna)
-  Expects the `pipeline_version` variable to be set to a known version (currently: 3.8, 3.9, 3.10, 3.11, nightly).
-
-#### Examples
-
-    ansible-playbook pipelines/install_pipeline.yml -e forklift_state=up -e pipeline_os=debian9 -e pipeline_type=foreman -e pipeline_version=nightly
-    ansible-playbook pipelines/upgrade_pipeline.yml -e forklift_state=up -e pipeline_os=centos7 -e pipeline_type=katello -e pipeline_version=3.10
+optional arguments:
+  -h, --help  show this help message and exit
+```
 
-### Creating Pipelines
+Individual pipelines also have help texts:
+```console
+usage: forklift install [-h] [-v] [-e EXTRA_VARS] [--state FORKLIFT_STATE]
+                        [--os PIPELINE_OS] [--type PIPELINE_TYPE]
+                        [--version PIPELINE_VERSION]
+
+Run an install pipeline
+
+optional arguments:
+  -h, --help            show this help message and exit
+  -v, --verbose         verbose output
+  --state FORKLIFT_STATE
+                        Forklift state to ensure
+  --os PIPELINE_OS      Operating system to install, like centos7, debian10 or
+                        ubuntu1804
+  --type PIPELINE_TYPE  Type of pipeline, like foreman, katello or luna
+  --version PIPELINE_VERSION
+                        Version to install, like nightly, 1.24 or 3.14
+
+advanced arguments:
+  -e EXTRA_VARS, --extra-vars EXTRA_VARS
+                        set additional variables as key=value or YAML/JSON, if
+                        filename prepend with @
+```
 
-If you wish to add a new version of an existing pipeline (e.g. a new Katello release), you only have to add the corresponding vars files to `pipelines/vars/`.
+Pipelines typically have a state which defaults to `up`. Other valid values are `rebuild` and `destroy`. The latter one is useful to clean up which pipelines don't do by themselves.
 
-For Katello 3.11, you'd be adding the following two files:
+For example to run a Foreman Nightly installation pipeline on Debian Buster:
 
-`pipelines/vars/katello_3.11.yml`:
-```yaml
-forklift_name: pipeline-katello-3.11
-forklift_boxes:
-  pipeline-katello-3.11-centos7:
-    box: centos7
-    memory: 8192
-  pipeline-proxy-3.11-centos7:
-    box: centos7
-    memory: 3072
-katello_repositories_version: '3.11'
-katello_repositories_pulp_version: '2.19'
-foreman_repositories_version: '1.21'
-foreman_client_repositories_version: "{{ foreman_repositories_version }}"
+```console
+$ ./bin/forklift install --os debian10 --type foreman --version nightly
+... lots of output
+$ ./bin/forklift install --os debian10 --type foreman --version nightly --state destroy
 ```
 
-`pipelines/vars/katello_upgrade_3.11.yml`:
-```yaml
-katello_version_start: '3.9'
-katello_version_intermediate: '3.10'
-katello_version_final: '{{ katello_version }}'
+Similarly a Katello Nightly upgrade pipeline on CentOS 7:
+
+```console
+$ ./bin/forklift upgrade --os centos7 --type katello --version nightly
+... lots of output
+$ ./bin/forklift upgrade --os centos7 --type katello --version nightly --state destroy
 ```
 
 ## Running Robottelo Tests

pipelines/install_pipeline.yml → pipelines/install/install.yaml

@@ -1,8 +1,9 @@
 - name: create vagrant boxes
-  hosts: localhost
+  hosts:
+    - localhost
   vars_files:
-    - vars/install_base.yml
-    - vars/{{ pipeline_type }}_base.yml
+    - vars/base.yml
+    - ../vars/{{ pipeline_type }}_base.yml
   roles:
     - forklift
 
@@ -14,9 +15,9 @@
   vars:
     foreman_installer_skip_installer: true
   vars_files:
-    - vars/install_base.yml
-    - vars/{{ pipeline_type }}_base.yml
-    - vars/{{ pipeline_type }}_base_staging.yml
+    - vars/base.yml
+    - ../vars/{{ pipeline_type }}_base.yml
+    - ../vars/{{ pipeline_type }}_base_staging.yml
   roles:
     - role: forklift_versions
       scenario: "{{ pipeline_type }}"
@@ -34,10 +35,10 @@
    - "{{ forklift_server_name }}"
   become: yes
   vars_files:
-    - vars/install_base.yml
-    - vars/{{ pipeline_type }}_base.yml
-    - vars/{{ pipeline_type }}_base_staging.yml
-    - vars/{{ pipeline_type }}_server.yml
+    - vars/base.yml
+    - ../vars/{{ pipeline_type }}_base.yml
+    - ../vars/{{ pipeline_type }}_base_staging.yml
+    - ../vars/{{ pipeline_type }}_server.yml
   roles:
     - role: forklift_versions
       scenario: "{{ pipeline_type }}"
@@ -49,9 +50,9 @@
     - "{{ forklift_proxy_name }}"
   become: yes
   vars_files:
-    - vars/install_base.yml
-    - vars/{{ pipeline_type }}_base_staging.yml
-    - vars/{{ pipeline_type }}_proxy.yml
+    - vars/base.yml
+    - ../vars/{{ pipeline_type }}_base_staging.yml
+    - ../vars/{{ pipeline_type }}_proxy.yml
   vars:
     foreman_proxy_content_server: "{{ forklift_server_name }}"
     foreman_installer_scenario: foreman-proxy-content
@@ -77,9 +78,9 @@
     - "{{ forklift_server_name }}"
   become: true
   vars_files:
-    - vars/install_base.yml
-    - vars/{{ pipeline_type }}_base.yml
-    - vars/{{ pipeline_type }}_base_staging.yml
+    - vars/base.yml
+    - ../vars/{{ pipeline_type }}_base.yml
+    - ../vars/{{ pipeline_type }}_base_staging.yml
   environment:
     FOREMAN_EXPECTED_VERSION: "{{ foreman_expected_version | default('') }}"
   roles:

pipelines/install/metadata.obsah.yaml

@@ -0,0 +1,18 @@
+help: |
+  Run an install pipeline
+variables:
+  forklift_state:
+    parameter: --state
+    help: Forklift state to ensure
+    # TODO: up/rebuild/destroy
+  pipeline_os:
+    parameter: --os
+    help: Operating system to install, like centos7, debian10 or ubuntu1804
+  pipeline_type:
+    parameter: --type
+    help: Type of pipeline, like foreman, katello or luna
+    # TODO: choices: foreman/katello/luna
+  pipeline_version:
+    parameter: --version
+    help: Version to install, like nightly, 1.24 or 3.14
+

pipelines/vars/install_base.yml → pipelines/install/vars/base.yml

@@ -1,3 +1,4 @@
+forklift_directory: "{{ playbook_dir | dirname | dirname }}"
 forklift_name: "pipeline-{{ pipeline_type }}-{{ pipeline_version }}-{{ pipeline_os }}"
 forklift_server_name: "pipeline-{{ pipeline_type }}-server-{{ pipeline_version }}-{{ pipeline_os }}"
 forklift_proxy_name: "pipeline-{{ pipeline_type }}-proxy-{{ pipeline_version }}-{{ pipeline_os }}"

pipelines/upgrade/metadata.obsah.yaml

@@ -0,0 +1,18 @@
+help: |
+  Run an upgrade pipeline
+variables:
+  forklift_state:
+    parameter: --state
+    help: Forklift state to ensure
+    # TODO: up/rebuild/destroy
+  pipeline_os:
+    parameter: --os
+    help: Operating system to install, like centos7, debian10 or ubuntu1804
+  pipeline_type:
+    parameter: --type
+    help: Type of pipeline, like foreman, katello or luna
+    # TODO: choices: foreman/katello/luna
+  pipeline_version:
+    parameter: --version
+    help: Version to install, like nightly, 1.24 or 3.14
+

pipelines/upgrade_pipeline.yml → pipelines/upgrade/upgrade.yaml

@@ -1,8 +1,8 @@
 - name: create vagrant boxes
   hosts: localhost
   vars_files:
-    - vars/upgrade_base.yml
-    - vars/{{ pipeline_type }}_base.yml
+    - vars/base.yml
+    - ../vars/{{ pipeline_type }}_base.yml
   roles:
     - forklift
 
@@ -14,9 +14,9 @@
   vars:
     foreman_installer_skip_installer: true
   vars_files:
-    - vars/upgrade_base.yml
-    - vars/{{ pipeline_type }}_base.yml
-    - vars/{{ pipeline_type }}_base_release.yml
+    - vars/base.yml
+    - ../vars/{{ pipeline_type }}_base.yml
+    - ../vars/{{ pipeline_type }}_base_release.yml
   roles:
     - role: forklift_versions
       scenario: "{{ pipeline_type }}"
@@ -39,10 +39,10 @@
     - "{{ forklift_server_name }}"
   become: yes
   vars_files:
-    - vars/upgrade_base.yml
-    - vars/{{ pipeline_type }}_base.yml
-    - vars/{{ pipeline_type }}_base_release.yml
-    - vars/{{ pipeline_type }}_server.yml
+    - vars/base.yml
+    - ../vars/{{ pipeline_type }}_base.yml
+    - ../vars/{{ pipeline_type }}_base_release.yml
+    - ../vars/{{ pipeline_type }}_server.yml
   roles:
     - role: forklift_versions
       scenario: "{{ pipeline_type }}"
@@ -58,8 +58,8 @@
     - "{{ forklift_proxy_name }}"
   become: yes
   vars_files:
-    - vars/upgrade_base.yml
-    - vars/{{ pipeline_type }}_proxy.yml
+    - vars/base.yml
+    - ../vars/{{ pipeline_type }}_proxy.yml
   vars:
     foreman_proxy_content_server: "{{ forklift_server_name }}"
     foreman_installer_disable_system_checks: true
@@ -88,10 +88,10 @@
     - "{{ forklift_server_name }}"
   become: true
   vars_files:
-    - vars/upgrade_base.yml
-    - vars/{{ pipeline_type }}_base.yml
-    - vars/{{ pipeline_type }}_base_release.yml
-    - vars/{{ pipeline_type }}_server.yml
+    - vars/base.yml
+    - ../vars/{{ pipeline_type }}_base.yml
+    - ../vars/{{ pipeline_type }}_base_release.yml
+    - ../vars/{{ pipeline_type }}_server.yml
   vars:
     foreman_installer_upgrade: True
   roles:
@@ -111,10 +111,10 @@
     - "{{ forklift_proxy_name }}"
   become: true
   vars_files:
-    - vars/upgrade_base.yml
-    - vars/{{ pipeline_type }}_base.yml
-    - vars/{{ pipeline_type }}_base_release.yml
-    - vars/{{ pipeline_type }}_proxy.yml
+    - vars/base.yml
+    - ../vars/{{ pipeline_type }}_base.yml
+    - ../vars/{{ pipeline_type }}_base_release.yml
+    - ../vars/{{ pipeline_type }}_proxy.yml
   vars:
     foreman_installer_upgrade: True
     foreman_proxy_content_upgrade: True
@@ -139,10 +139,10 @@
   vars:
     foreman_installer_upgrade: True
   vars_files:
-    - vars/upgrade_base.yml
-    - vars/{{ pipeline_type }}_base.yml
-    - vars/{{ pipeline_type }}_base_staging.yml
-    - vars/{{ pipeline_type }}_server.yml
+    - vars/base.yml
+    - ../vars/{{ pipeline_type }}_base.yml
+    - ../vars/{{ pipeline_type }}_base_staging.yml
+    - ../vars/{{ pipeline_type }}_server.yml
   roles:
     - role: forklift_versions
       scenario: "{{ pipeline_type }}"
@@ -163,10 +163,10 @@
     foreman_proxy_content_upgrade: True
     foreman_proxy_content_server: "{{ forklift_server_name }}"
   vars_files:
-    - vars/upgrade_base.yml
-    - vars/{{ pipeline_type }}_base.yml
-    - vars/{{ pipeline_type }}_base_staging.yml
-    - vars/{{ pipeline_type }}_proxy.yml
+    - vars/base.yml
+    - ../vars/{{ pipeline_type }}_base.yml
+    - ../vars/{{ pipeline_type }}_base_staging.yml
+    - ../vars/{{ pipeline_type }}_proxy.yml
   roles:
     - role: forklift_versions
       scenario: "{{ pipeline_type }}"
@@ -184,9 +184,9 @@
     - "{{ forklift_server_name }}"
   become: yes
   vars_files:
-    - vars/upgrade_base.yml
-    - vars/{{ pipeline_type }}_base.yml
-    - vars/{{ pipeline_type }}_base_staging.yml
+    - vars/base.yml
+    - ../vars/{{ pipeline_type }}_base.yml
+    - ../vars/{{ pipeline_type }}_base_staging.yml
   environment:
     FOREMAN_EXPECTED_VERSION: "{{ foreman_expected_version | default('') }}"
   roles:

pipelines/vars/upgrade_base.yml → pipelines/upgrade/vars/base.yml

@@ -1,3 +1,4 @@
+forklift_directory: "{{ playbook_dir | dirname | dirname }}"
 forklift_name: "pipeline-upgrade-{{ pipeline_type }}-{{ pipeline_version}}-{{ pipeline_os }}"
 forklift_server_name: "pipeline-upgrade-{{ pipeline_type }}-{{ pipeline_version}}-{{ pipeline_os }}"
 forklift_proxy_name: "pipeline-upgrade-{{ pipeline_type }}-proxy-{{ pipeline_version}}-{{ pipeline_os }}"