This repository has been archived by the owner on Oct 7, 2024. It is now read-only.
-
-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
148 update all missing documentation (#240)
* Add a app diagram. Rename ADR * Update a bunch of documents * Commit arch doc work * Change the format of logging to avoid useless information * Update the logging proposal * Update the exceptions proposal * Update FunctionRegistry proposal * Update config path proposal * Add the config proposal * Add function invocation document * Update the flows and actions document * Update templates * Add the development branch ADR * Update the gcloud vs python API ADR * Update github PR templates * Add a glossary document * Update the http example. Update the build default values for the logs bool option * Add a deploying to GCP example * Add an example of adding an existing function to the registry
- Loading branch information
Piotr Katolik
authored
Feb 27, 2022
1 parent
1ce0d3a
commit e8196ca
Showing
32 changed files
with
728 additions
and
570 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
Resolves # |
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,137 @@ | ||
# `functions` - architecture overview | ||
|
||
* Status: Proposed | ||
* Deciders: [Piotr] | ||
* Date: 2022-02-23 | ||
|
||
## Context and Problem Statement | ||
|
||
This document describes all the architectural components used in the layout of the app. | ||
|
||
It describe the assumed flow of the app under certain user stories and how different components fall into a full composition. | ||
|
||
The architecture of the project has been developed in iterative process of improvement to the base concept (ADR-0001) and this document will formalize the decisions and provide shape and form context. | ||
|
||
## Users | ||
|
||
We are taking into the account following user stories: | ||
|
||
* user has never worked with function as a service | ||
* user has existing functions and wants to use the tool for unification and easy of use | ||
* locally on the machine of work | ||
* remotely in a form of a repository | ||
* machine use, as an example being part of a CI/CD | ||
|
||
## Goals | ||
|
||
### Seamless installation | ||
|
||
**Problem**: We want our users to have a single line and minimal complexity installation journey. | ||
|
||
In the driver of *simplicity* the installation should be a quick and easy process. This should include all the ways of managing functions. | ||
|
||
### Hight modularity | ||
|
||
**Problem**: We want to allow user to decide which parts of the package are required and which one's are not. | ||
|
||
This means that if the user only needs the local setup, then installing `docker` should be the only thing required to run those commands. | ||
|
||
Some of the questions we need to answer with the solution: | ||
|
||
* How do we figure out which components are available? | ||
* How do we store that information? | ||
* How do we change that state? | ||
* Do we remove or update that information? | ||
* Do we run any validation? If so, then when? How often? | ||
|
||
### Components sync | ||
|
||
**Problem**: We need to have a consistent way of knowing what is happening with various parts of the application at the lowest operational cost. | ||
|
||
This means we want to know which functions are currently deployed, but we don't want to run a check every time a script is run, because that is very inefficient in the CLI way. | ||
|
||
Some of the questions we need to answer with the solution: | ||
|
||
* How and when do we query and store the current state of the `Docker` component? Is a function running, stopped, built? | ||
|
||
### Assumptions | ||
|
||
Our users are technically skilled individuals or organisation that have necessary skills to operate a CLI tool. | ||
|
||
### Constraints | ||
|
||
No constrains at the moment. | ||
|
||
## Decision Drivers | ||
|
||
* Implementation complexity | ||
* Simplicity of use | ||
* Configurability | ||
|
||
|
||
## Design decision | ||
|
||
The architecture is build in a iterative process with improving current design. | ||
|
||
In the following sections we describe components, their purpose and how they may interact with other parts of the design. | ||
|
||
### Overview | ||
|
||
TBU | ||
|
||
### App Diagram | ||
|
||
*Insert a diagram representing different interactions between components.* | ||
|
||
 | ||
|
||
### Flow | ||
|
||
TBU | ||
|
||
Example interaction for basic case of CLI command. | ||
|
||
1. User types a `functions` command in the terminal. | ||
2. `functions` will evaluate available components and store this information in the config file. Command will raise exceptions if required components like `docker` are not present. | ||
3. Next, it will evaluate the registry if information about this function is available and if said registry needs to be updated. | ||
4. Depending on the command type, `docker` will be used to run or serve a targeted function. | ||
5. If a command deploys resources to a cloud provider, we will use the correct provider to manipulate resources. Subject to authorization and sub-tool (like `gcloud`) availability. | ||
|
||
TBU | ||
|
||
- Internal state saved in a function registry | ||
- Component management | ||
- Functions Config | ||
- Logging | ||
- Exceptions | ||
- Managing Docker resources | ||
- Managing GCP resources | ||
- Creating new resources | ||
- Adding existing resources to the scope of the tool | ||
- defaults | ||
- | ||
- Command validation and autocomplete | ||
- Flows and Actions | ||
|
||
|
||
### Disclaimer | ||
|
||
All these sections describe an idea components. What is real and present in the code often escapes that. These are the best ideas that we came up with trying to handle code scalability, tool's functionality while maintaining an experimental mindset. | ||
|
||
## Decision Impact | ||
|
||
| Interested Parties/Groups | Informed | | ||
| ------------------------- | -------- | | ||
| All Users | N/A | | ||
|
||
### How will this decision be communicated to the public | ||
|
||
There is no need to communicate it to any channels as it is a documentation written prior to any releases. It is bound to the first release. | ||
|
||
## Links | ||
|
||
* [ADR-0001](0001_initial_setup.md) | ||
|
||
<!-- Identifiers, in alphabetical order --> | ||
|
||
[Piotr]: https://github.com/Katolus |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,90 @@ | ||
# What are we using as our main development branch? | ||
|
||
* Status: Accepted | ||
* Deciders: [Piotr] | ||
* Date: 2021-11-23 | ||
|
||
## Context and Problem Statement | ||
|
||
[Describe the context and problem statement, e.g., in free form using two to three sentences. You may want to articulate the problem in form of a question.] | ||
|
||
## Details | ||
|
||
|
||
### Constraints | ||
|
||
* Adhere to an understandable standard. | ||
|
||
## Decision Drivers | ||
|
||
* Keep it simple. | ||
* Keep it flexible. | ||
* Keep it cheap. | ||
* Keep it safe. | ||
|
||
## Considered Options | ||
|
||
* `master`/`main` | ||
* `development` | ||
|
||
## Decision Outcome | ||
|
||
Chosen option: `development`, because we think that it will be the easiest branch names to drive ongoing development efforts. It semantically separate from `master` or `main` and has a clear commitment. | ||
|
||
The idea behind this decision is a attempt to create a more accessible development stream. | ||
|
||
The vision being that the `master` branch should be focused on the valuable releases associate with package versions releases. | ||
|
||
All the tooling that needs to be run to enforce quality should be associated with the `master` branch. | ||
|
||
The `master` branch should be protected. | ||
|
||
All this should enable a more flexible work around smaller issues in a singular pool of developer(s). | ||
|
||
All the PRs should target the `development` branch as base and a release PR should be a combination of sub PRs on a per ready basis. | ||
|
||
This means that the tests for various python versions of the package and other heavy duty actions will only be trigger on merges to master. | ||
|
||
### Positive Consequences | ||
|
||
* low-cost of ongoing work. | ||
* faster development. | ||
* more control over release stages. | ||
|
||
### Negative Consequences | ||
|
||
* untested changes in `development` might be hard to discover prior to `master` merges. | ||
* slightly more complicated contribution process. | ||
|
||
## Pros and Cons of the Options | ||
|
||
### `master` / `main` | ||
|
||
Use `master` or `main` as the main development branch. | ||
|
||
* Good, because it is a well understand pattern. | ||
* Bad, because it will cost us more to run all the check in every merge event. | ||
|
||
### `development` | ||
|
||
Use `development` as the main development branch. | ||
|
||
* Good, because it separates development actions from main tooling branch. | ||
* Good, because it has a clear meaning. | ||
* Good, because it should allow for making quick changes. | ||
* Bad, because it might be a bit confusing. | ||
* Bad, because it might lead untested changes into the main `development` branch. | ||
|
||
## Decision Impact | ||
|
||
| Interested Parties/Groups | Informed | | ||
| ------------------------- | -------- | | ||
| Contributors | Yes | | ||
|
||
### How will this decision be communicated to the public | ||
|
||
This information is communication via the `CONTRIBUTING.md` document. | ||
|
||
<!-- Identifiers, in alphabetical order --> | ||
|
||
[Piotr]: https://github.com/Katolus |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,9 +1,81 @@ | ||
# `gcloud` vs GCP's Python APIs | ||
# What interface should we use it interact with GCP resources? | ||
|
||
## Problem statement | ||
* Status: Accepted | ||
* Deciders: [Piotr] | ||
* Date: 2021-11-23 | ||
|
||
In order to work with GCP, we need to find a way of interacting with GCP resources. There are two know ways of doing that. One way is to use the `gcloud` tool. A command line tool that can be used to interact with GCP via a terminal. Second way is to use GCP's Python API libraries. A set of libraries developed by Google to make it possible to interact with GCP's components from a python scope. | ||
## Context and Problem Statement | ||
|
||
## Attempt | ||
Since we want to enable users to be able to deploy functions to GCP, we need to have an interface that we can call within our code. | ||
|
||
History attempt on using the libraries. | ||
This interface needs to be capable of bridging us from a source files to a deployed and working cloud resource. | ||
|
||
### Assumptions | ||
|
||
The tool is not meant to be ideal, but be a working proof of concept at this point in time. | ||
|
||
### Constraints | ||
|
||
* Limited development capacity | ||
* Scope of the initial release | ||
|
||
## Decision Drivers | ||
|
||
* Development complexity | ||
* Short vs long term solution | ||
|
||
## Considered Options | ||
|
||
* `gcloud` | ||
* `Python API` | ||
|
||
### `gcloud` | ||
|
||
Use the `gcloud` command line interface to move our resources from the source to cloud. | ||
|
||
Pros: | ||
|
||
* Good, because it is a simple solution that allows us to provide this functionality. | ||
* Good, because it handles a lot of the complexity, like authorization, outputs, logging. | ||
|
||
Cons: | ||
|
||
* Bad, because it feels more like a hack than a solution. | ||
* Bad, because it is not a long term solution. | ||
* Bad, because it requires an external interface - `gcloud` - to handle the tasks. | ||
|
||
**Spike**: This works relatively well, but feels like a hacky way to solve this problem. | ||
|
||
### Python APIs | ||
|
||
Use resource [specific](https://cloud.google.com/python/docs/reference/cloudfunctions/latest) as well as [generic](https://github.com/googleapis/google-api-python-client) python SDKs to interact with GCP resources. | ||
|
||
Pros: | ||
|
||
* Good, because it gives us a lot of flexibility over our implementation. | ||
* Good, because we don't have to rely on a user having to install a tool within the environment. | ||
* Good, because there is place for factoring out common code from different providers and exploring unification efforts more achievable. | ||
|
||
Cons: | ||
|
||
* Bad, because it requires a lot of additional work to write logic to handle all the resources. | ||
* Bad, because it is much more complex to do so. There are no single call APIs for cloud functions. | ||
* Bad, because it feels like reinventing parts of the `gcloud` tool. | ||
|
||
**Spike**: What seems like a simple task of deploying a folder of files into a cloud function scope turns out to be more complex than that. You need to have a place to store the files, build the function and then connect all these resources. Cleaning and managing artefacts becomes our scope as well. | ||
|
||
## Decision Outcome | ||
|
||
Chosen option: `gcloud`, because the development effort is minimal and allows us to perform really powerful stuff. It offloads the effort onto another tool and allows us to test and move at relatively fast pace. At a cost of not being a future proof solution. | ||
|
||
## Decision Impact | ||
|
||
No one is impacted by this decision. Interface will stay without a change. This change does have a long term impact of being vulnerable to any `gcloud` version changes. | ||
|
||
### How will this decision be communicated to the public | ||
|
||
No need. | ||
|
||
<!-- Identifiers, in alphabetical order --> | ||
|
||
[Piotr]: https://github.com/Katolus |
Oops, something went wrong.