Skip to content
This repository has been archived by the owner on Oct 7, 2024. It is now read-only.

Commit

Permalink
148 update all missing documentation (#240)
Browse files Browse the repository at this point in the history
* Add a app diagram. Rename ADR

* Update a bunch of documents

* Commit arch doc work

* Change the format of logging to avoid useless information

* Update the logging proposal

* Update the exceptions proposal

* Update FunctionRegistry proposal

* Update config path proposal

* Add the config proposal

* Add function invocation document

* Update the flows and actions document

* Update templates

* Add the development branch ADR

* Update the gcloud vs python API ADR

* Update github PR templates

* Add a glossary document

* Update the http example. Update the build default values for the logs bool option

* Add a deploying to GCP example

* Add an example of adding an existing function to the registry
  • Loading branch information
Piotr Katolik authored Feb 27, 2022
1 parent 1ce0d3a commit e8196ca
Show file tree
Hide file tree
Showing 32 changed files with 728 additions and 570 deletions.
1 change: 1 addition & 0 deletions .github/PULL_REQUEST_TEMPLATE/documentation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Resolves #
17 changes: 0 additions & 17 deletions .github/PULL_REQUEST_TEMPLATE/pull_request_template.md

This file was deleted.

17 changes: 1 addition & 16 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,7 @@
# Pull Request

## Related Issue
## Issue

Resolves #

## Description of the Change

<!-- Write an extensive description of the change so that a person reviewing have the best possible view over your work. -->

## Prerequisites

<!-- Extra steps required before testing the change itself. -->

## Testing Instructions

<!-- List all the steps and information needed to comprehensively test the proposed solution. -->


## Checklist

- [ ] Did you include necessary logging?
Expand Down
7 changes: 4 additions & 3 deletions docs/adrs/0001_initial_setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ The code wasn't necessarily clearly contained within a container like the `confi

Here is a view from the command line at this time.

![23_12_2021_CLI_view](../assets/23_12_2021_CLI_view.png)
![23_12_2021_CLI_view](../assets/screenshots/23_12_2021_CLI_view.png)

Here is some additional context on these root commands:

Expand All @@ -57,5 +57,6 @@ The initial concept has been designed as a single scenario concept (due to a pro
* creating the correct structure in config for components like local, gcp and any future ones
* various components in the code were not prepared to be modularized in a way to easily accept extensions from new components (like AWS extensions)

<!-- Shortcuts for links -->
* [Piotr](https://github.com/Katolus)
<!-- Identifiers, in alphabetical order -->

[Piotr]: https://github.com/Katolus
137 changes: 137 additions & 0 deletions docs/adrs/0002_architecture_layout.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
# `functions` - architecture overview

* Status: Proposed
* Deciders: [Piotr]
* Date: 2022-02-23

## Context and Problem Statement

This document describes all the architectural components used in the layout of the app.

It describe the assumed flow of the app under certain user stories and how different components fall into a full composition.

The architecture of the project has been developed in iterative process of improvement to the base concept (ADR-0001) and this document will formalize the decisions and provide shape and form context.

## Users

We are taking into the account following user stories:

* user has never worked with function as a service
* user has existing functions and wants to use the tool for unification and easy of use
* locally on the machine of work
* remotely in a form of a repository
* machine use, as an example being part of a CI/CD

## Goals

### Seamless installation

**Problem**: We want our users to have a single line and minimal complexity installation journey.

In the driver of *simplicity* the installation should be a quick and easy process. This should include all the ways of managing functions.

### Hight modularity

**Problem**: We want to allow user to decide which parts of the package are required and which one's are not.

This means that if the user only needs the local setup, then installing `docker` should be the only thing required to run those commands.

Some of the questions we need to answer with the solution:

* How do we figure out which components are available?
* How do we store that information?
* How do we change that state?
* Do we remove or update that information?
* Do we run any validation? If so, then when? How often?

### Components sync

**Problem**: We need to have a consistent way of knowing what is happening with various parts of the application at the lowest operational cost.

This means we want to know which functions are currently deployed, but we don't want to run a check every time a script is run, because that is very inefficient in the CLI way.

Some of the questions we need to answer with the solution:

* How and when do we query and store the current state of the `Docker` component? Is a function running, stopped, built?

### Assumptions

Our users are technically skilled individuals or organisation that have necessary skills to operate a CLI tool.

### Constraints

No constrains at the moment.

## Decision Drivers

* Implementation complexity
* Simplicity of use
* Configurability


## Design decision

The architecture is build in a iterative process with improving current design.

In the following sections we describe components, their purpose and how they may interact with other parts of the design.

### Overview

TBU

### App Diagram

*Insert a diagram representing different interactions between components.*

![Functions app diagram](../assets/diagrams/functional_layout.png)

### Flow

TBU

Example interaction for basic case of CLI command.

1. User types a `functions` command in the terminal.
2. `functions` will evaluate available components and store this information in the config file. Command will raise exceptions if required components like `docker` are not present.
3. Next, it will evaluate the registry if information about this function is available and if said registry needs to be updated.
4. Depending on the command type, `docker` will be used to run or serve a targeted function.
5. If a command deploys resources to a cloud provider, we will use the correct provider to manipulate resources. Subject to authorization and sub-tool (like `gcloud`) availability.

TBU

- Internal state saved in a function registry
- Component management
- Functions Config
- Logging
- Exceptions
- Managing Docker resources
- Managing GCP resources
- Creating new resources
- Adding existing resources to the scope of the tool
- defaults
-
- Command validation and autocomplete
- Flows and Actions


### Disclaimer

All these sections describe an idea components. What is real and present in the code often escapes that. These are the best ideas that we came up with trying to handle code scalability, tool's functionality while maintaining an experimental mindset.

## Decision Impact

| Interested Parties/Groups | Informed |
| ------------------------- | -------- |
| All Users | N/A |

### How will this decision be communicated to the public

There is no need to communicate it to any channels as it is a documentation written prior to any releases. It is bound to the first release.

## Links

* [ADR-0001](0001_initial_setup.md)

<!-- Identifiers, in alphabetical order -->

[Piotr]: https://github.com/Katolus
90 changes: 90 additions & 0 deletions docs/adrs/0002_development_branch.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
# What are we using as our main development branch?

* Status: Accepted
* Deciders: [Piotr]
* Date: 2021-11-23

## Context and Problem Statement

[Describe the context and problem statement, e.g., in free form using two to three sentences. You may want to articulate the problem in form of a question.]

## Details


### Constraints

* Adhere to an understandable standard.

## Decision Drivers

* Keep it simple.
* Keep it flexible.
* Keep it cheap.
* Keep it safe.

## Considered Options

* `master`/`main`
* `development`

## Decision Outcome

Chosen option: `development`, because we think that it will be the easiest branch names to drive ongoing development efforts. It semantically separate from `master` or `main` and has a clear commitment.

The idea behind this decision is a attempt to create a more accessible development stream.

The vision being that the `master` branch should be focused on the valuable releases associate with package versions releases.

All the tooling that needs to be run to enforce quality should be associated with the `master` branch.

The `master` branch should be protected.

All this should enable a more flexible work around smaller issues in a singular pool of developer(s).

All the PRs should target the `development` branch as base and a release PR should be a combination of sub PRs on a per ready basis.

This means that the tests for various python versions of the package and other heavy duty actions will only be trigger on merges to master.

### Positive Consequences

* low-cost of ongoing work.
* faster development.
* more control over release stages.

### Negative Consequences

* untested changes in `development` might be hard to discover prior to `master` merges.
* slightly more complicated contribution process.

## Pros and Cons of the Options

### `master` / `main`

Use `master` or `main` as the main development branch.

* Good, because it is a well understand pattern.
* Bad, because it will cost us more to run all the check in every merge event.

### `development`

Use `development` as the main development branch.

* Good, because it separates development actions from main tooling branch.
* Good, because it has a clear meaning.
* Good, because it should allow for making quick changes.
* Bad, because it might be a bit confusing.
* Bad, because it might lead untested changes into the main `development` branch.

## Decision Impact

| Interested Parties/Groups | Informed |
| ------------------------- | -------- |
| Contributors | Yes |

### How will this decision be communicated to the public

This information is communication via the `CONTRIBUTING.md` document.

<!-- Identifiers, in alphabetical order -->

[Piotr]: https://github.com/Katolus
82 changes: 77 additions & 5 deletions docs/adrs/0003_gcloud_vs_python_api.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,81 @@
# `gcloud` vs GCP's Python APIs
# What interface should we use it interact with GCP resources?

## Problem statement
* Status: Accepted
* Deciders: [Piotr]
* Date: 2021-11-23

In order to work with GCP, we need to find a way of interacting with GCP resources. There are two know ways of doing that. One way is to use the `gcloud` tool. A command line tool that can be used to interact with GCP via a terminal. Second way is to use GCP's Python API libraries. A set of libraries developed by Google to make it possible to interact with GCP's components from a python scope.
## Context and Problem Statement

## Attempt
Since we want to enable users to be able to deploy functions to GCP, we need to have an interface that we can call within our code.

History attempt on using the libraries.
This interface needs to be capable of bridging us from a source files to a deployed and working cloud resource.

### Assumptions

The tool is not meant to be ideal, but be a working proof of concept at this point in time.

### Constraints

* Limited development capacity
* Scope of the initial release

## Decision Drivers

* Development complexity
* Short vs long term solution

## Considered Options

* `gcloud`
* `Python API`

### `gcloud`

Use the `gcloud` command line interface to move our resources from the source to cloud.

Pros:

* Good, because it is a simple solution that allows us to provide this functionality.
* Good, because it handles a lot of the complexity, like authorization, outputs, logging.

Cons:

* Bad, because it feels more like a hack than a solution.
* Bad, because it is not a long term solution.
* Bad, because it requires an external interface - `gcloud` - to handle the tasks.

**Spike**: This works relatively well, but feels like a hacky way to solve this problem.

### Python APIs

Use resource [specific](https://cloud.google.com/python/docs/reference/cloudfunctions/latest) as well as [generic](https://github.com/googleapis/google-api-python-client) python SDKs to interact with GCP resources.

Pros:

* Good, because it gives us a lot of flexibility over our implementation.
* Good, because we don't have to rely on a user having to install a tool within the environment.
* Good, because there is place for factoring out common code from different providers and exploring unification efforts more achievable.

Cons:

* Bad, because it requires a lot of additional work to write logic to handle all the resources.
* Bad, because it is much more complex to do so. There are no single call APIs for cloud functions.
* Bad, because it feels like reinventing parts of the `gcloud` tool.

**Spike**: What seems like a simple task of deploying a folder of files into a cloud function scope turns out to be more complex than that. You need to have a place to store the files, build the function and then connect all these resources. Cleaning and managing artefacts becomes our scope as well.

## Decision Outcome

Chosen option: `gcloud`, because the development effort is minimal and allows us to perform really powerful stuff. It offloads the effort onto another tool and allows us to test and move at relatively fast pace. At a cost of not being a future proof solution.

## Decision Impact

No one is impacted by this decision. Interface will stay without a change. This change does have a long term impact of being vulnerable to any `gcloud` version changes.

### How will this decision be communicated to the public

No need.

<!-- Identifiers, in alphabetical order -->

[Piotr]: https://github.com/Katolus
Loading

0 comments on commit e8196ca

Please sign in to comment.