Skip to content

Commit

Permalink
feat: adds versions description and upgrade guide
Browse files Browse the repository at this point in the history
  • Loading branch information
claymcleod committed Jan 24, 2025
1 parent 6f23c95 commit dd17a4e
Show file tree
Hide file tree
Showing 3 changed files with 215 additions and 1 deletion.
7 changes: 6 additions & 1 deletion .vitepress/config.mts
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,7 @@ export default defineConfig({
{ text: "Tasks", link: "/language-guide/tasks.md" },
{ text: "Workflows", link: "/language-guide/workflows.md" },
{ text: "Imports", link: "/language-guide/imports.md" },
{ text: "Versions", link: "/language-guide/versions.md" },
],
},
{
Expand All @@ -93,7 +94,11 @@ export default defineConfig({
text: "Reference",
items: [
{
text: "Standard Library",
text: "Upgrade guide",
link: "/reference/upgrade-guide"
},
{
text: "Standard library",
items: [
{
text: "Numeric functions",
Expand Down
47 changes: 47 additions & 0 deletions language-guide/versions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# Verisons

The Workflow Description Language has two concepts of versions within the project:

* The WDL _language_ has a two-number version (e.g., `1.2`). An increase in the
minor (second) version number (e.g., `1.1` to `1.2`) indicates the addition
of, or non-breaking changes to, the language or standard library functions. An
increase in the major (first) version number (e.g., `1.0` to `2.0`) indicates
that breaking changes have been made.

* The WDL _specification_ has a three-number version (e.g., `1.2.0`). The
specification version tracks the language version, but there may also be patch
releases (indicated by a change to the patch, or third, version number) that
include fixes for typos, additional examples, or non-breaking clarifications
of ambiguous language.

In general, users of WDL only need to care about the version of the WDL
_language_—you'll rarely, if ever, need to care about the version of the
_specification_ itself.

## Upgrading

If you're interested in learning more about the finer details of upgrading
between versions, such as what features new versions introduce, common pitfalls
of upgrading, and how to get help, see the [Upgrade guide] in the Reference section.

## Version specification

The WDL version is a required statement at the top of every WDL document—it
**must** be the first non-comment within the document.

You can specify the version of your WDL document like so:

```wdl
version 1.2
# ... other document contents ...
```

## Compatability considerations

Documents may only import other WDL documents of the same version. This is
because the imported documents are effectively comingled within their importer's
context and processed holistically (instead of, for example, being compiled
independently).

[Upgrade guide]: ../reference/upgrade-guide.md
162 changes: 162 additions & 0 deletions reference/upgrade-guide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
---
outline: [1, 2]
---

# Overview

The Workflow Description Language has a number of **versions** that are
constantly improving. This guide lists all major versions of the language—the
major improvements for each version, common pitfalls when upgrading, and how to
get help.

If you're interested in learning more about how versioning works and how you can
specify the version of your WDL documents, see the [Versioning section] within
the Language Guide.

::: tip
Changes are grouped in terms of their **impact**. The word "impact" here is a
combination of "how important is the change" along with "how many existing WDL
documents are likely to be affected by this change". They are subjective
determinations made by the upgrade guide curator during the curator process, but
they are often helpful in quickly scanning upgrade elements to see which might
apply to your situation.
:::

# WDL v1.2

WDL v1.2 introduces a great deal of new syntax and changes to the specification.
Notably, changes were backwards compatible with all previous `v1.x` releases.
This sections only covers the high points related to upgrading—the full release
notes can be found [here][wdl-v1.2-release-notes].

### Checklist

Use this checklist to ensure you hit all of the sections.

#### Moderate impact changes

<input type="checkbox" /> New `Directory` type ([link](#new-directory-type)).<br />
<input type="checkbox" /> Replacement of `runtime` section with `requirements` and `hints` sections ([link](#replacement-of-runtime-with-requirements-and-hints)). <br />
<input type="checkbox" /> New standard library functions
([link](#new-standard-library-functions)). <br />
<input type="checkbox" /> Multi-line strings ([link](#multi-line-strings)). <br />
<input type="checkbox" /> Optional `input:` statement ([link](#optional-input-statement)). <br />

#### Low impact changes

<input type="checkbox" /> Addition of workflow `hints` ([link](#addition-of-workflow-hints)). <br />
<input type="checkbox" /> New requirements and hints keys ([link](#new-requirements-and-hints-keys)). <br />
<input type="checkbox" /> Metadata sections for structs ([link](#metadata-sections-for-structs)). <br />
<input type="checkbox" /> Exponentiation operator ([link](#exponentiation-operator)). <br />

## Moderate impact changes

### New `Directory` type

The `Directory` type was introduced in
[#641](https://github.com/openwdl/wdl/pull/641) to better semantically indicate
the use of a directory. If the intention of any of your arguments is to be used
to refer to a directory on the filesystem, you are encouraged to update the
parameters to a `Directory` type.

### Replacement of section `runtime` with `requirements` and `hints` sections

The `runtime` section, which previously held both requirement contraints and
hints to the execution engine, has now been split into the `requirements`
([#540](https://github.com/openwdl/wdl/issues/540)) section and `hints`
([#541](https://github.com/openwdl/wdl/issues/541)) section respectively. You
should split out these keys based on the definitions in the
specification<sup>[1](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-requirements-section),
[2](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-hints-section)</sup>.

🗑️ This change deprecates the `runtime` section, which will be removed in WDL
v2.0.

### New standard library functions

The following are new standard library functions and their definitions. You are
encouraged to read throught them and replace any custom functionality that would
now be duplicated in favor of these functions.

- `contains_key`: whether or not a `Map` or `Object` contain a specific member
([link](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-contains_key)).
- `values`: get the values from a `Map`
([link](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-values)).
- `find`: search for a regular expression in a string
([link](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-find)).
- `matches`: whether a string match a regular expression
([link](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-matches)).
- `chunk`: split an array into sub-arrays
([link](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-chunk)).
- `join_paths`: join two or more paths
([link](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-join_paths)).
- `contains`: whether an array contains a specified value
([link](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-contains)).


### Multi-line strings

Multi-line strings were introduced with the following syntax.

```wdl
String foo = <<<
my
multi-line
string
>>>
```

### Optional `input:` statement

As noted in [#524](https://github.com/openwdl/wdl/pull/524), the `input:`
statement that preceeds call bodies is unnecessary historical boilerplate. This
statement is now optional in `call` bodies. You are encouraged to remove these
from your `call` bodies.

## Low impact changes

### Addition of workflow `hints`

Workflows gained a `hints` section in
[#543](https://github.com/openwdl/wdl/issues/543). You are encouraged to go read
the supported workflow
hints<sup>[1](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#workflow-hints)</sup>
and apply them to your workflow if any are relevant.

### New requirements and hints keys

- The following keys were added to the task `requirements` section: `fpga`
<sup>[1](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#hardware-accelerators-gpu-and--fpga)</sup>.
- The following keys were added to the task `hints` section: `disks`
<sup>[1](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-disks)</sup>,
`gpu`
<sup>[2](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-gpu-and--fpga)</sup>,
and `fpga`
<sup>[2](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#-gpu-and--fpga)</sup>.

You are encouraged to examine these keys are determine if any of these should be
specified for your tasks.

### Metadata sections for structs

Similarly to tasks and workflows, structs now have `meta` and `parameter_meta`
sections. You are encouraged to use these fields according to the definition in
the
specification<sup>[1](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#struct-definition)</sup>.

### Exponentiation operator

The exponentiation operator (`**`) was added in this release. You are encouraged
to go and update any manual exponentiation to use this operator instead.

You are encouraged to go read the specification section on this concept
([link](https://github.com/openwdl/wdl/blob/wdl-1.2/SPEC.md#multi-line-strings))
and use them where appropriate.

# Prior WDL versions

Versions of WDL prior to the ones outlined in this guide did not exist at the
time the upgrade guide was created. As such, they are not included in the guide.

[wdl-v1.2-release-notes]: https://github.com/openwdl/wdl/releases/tag/v1.2.0
[Versioning section]: ./language-guide/versions.md

0 comments on commit dd17a4e

Please sign in to comment.