Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs(misc): update Custom DTE to Manual DTE #30180

Merged
merged 2 commits into from
Feb 26, 2025
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
522 changes: 222 additions & 300 deletions docs/generated/manifests/ci.json

Large diffs are not rendered by default.

399 changes: 171 additions & 228 deletions docs/generated/manifests/menus.json

Large diffs are not rendered by default.

26 changes: 13 additions & 13 deletions docs/generated/manifests/nx.json
Original file line number Diff line number Diff line change
Expand Up @@ -5748,14 +5748,14 @@
"tags": []
},
{
"id": "dte",
"name": "Large Repo and DTE",
"id": "nx-agents",
"name": "Large Repo and Nx Agents",
"description": "",
"mediaImage": "",
"file": "shared/examples/dte",
"file": "shared/examples/nx-agents",
"itemList": [],
"isExternal": false,
"path": "/showcase/benchmarks/dte",
"path": "/showcase/benchmarks/nx-agents",
"tags": []
}
],
Expand Down Expand Up @@ -6162,14 +6162,14 @@
"tags": []
},
{
"id": "dte",
"name": "Large Repo and DTE",
"id": "nx-agents",
"name": "Large Repo and Nx Agents",
"description": "",
"mediaImage": "",
"file": "shared/examples/dte",
"file": "shared/examples/nx-agents",
"itemList": [],
"isExternal": false,
"path": "/showcase/benchmarks/dte",
"path": "/showcase/benchmarks/nx-agents",
"tags": []
}
],
Expand Down Expand Up @@ -6199,15 +6199,15 @@
"path": "/showcase/benchmarks/caching",
"tags": []
},
"/showcase/benchmarks/dte": {
"id": "dte",
"name": "Large Repo and DTE",
"/showcase/benchmarks/nx-agents": {
"id": "nx-agents",
"name": "Large Repo and Nx Agents",
"description": "",
"mediaImage": "",
"file": "shared/examples/dte",
"file": "shared/examples/nx-agents",
"itemList": [],
"isExternal": false,
"path": "/showcase/benchmarks/dte",
"path": "/showcase/benchmarks/nx-agents",
"tags": []
},
"/reference": {
Expand Down
80 changes: 40 additions & 40 deletions docs/map.json
Original file line number Diff line number Diff line change
Expand Up @@ -1262,9 +1262,9 @@
"file": "shared/examples/caching"
},
{
"name": "Large Repo and DTE",
"id": "dte",
"file": "shared/examples/dte"
"name": "Large Repo and Nx Agents",
"id": "nx-agents",
"file": "shared/examples/nx-agents"
}
]
}
Expand Down Expand Up @@ -1888,6 +1888,43 @@
}
]
},
{
"name": "Manual Distributed Task Execution",
"id": "dte",
"description": "Manual DTE on your own CI provider",
"itemList": [
{
"name": "GitHub Actions Manual DTE",
"id": "github-dte",
"file": "nx-cloud/enterprise/dte/github-dte"
},
{
"name": "Circle CI Manual DTE",
"id": "circle-ci-dte",
"file": "nx-cloud/enterprise/dte/circle-ci-dte"
},
{
"name": "Azure Pipelines Manual DTE",
"id": "azure-dte",
"file": "nx-cloud/enterprise/dte/azure-dte"
},
{
"name": "Bitbucket Pipelines Manual DTE",
"id": "bitbucket-dte",
"file": "nx-cloud/enterprise/dte/bitbucket-dte"
},
{
"name": "GitLab Manual DTE",
"id": "gitlab-dte",
"file": "nx-cloud/enterprise/dte/gitlab-dte"
},
{
"name": "Jenkins Manual DTE",
"id": "jenkins-dte",
"file": "nx-cloud/enterprise/dte/jenkins-dte"
}
]
},
{
"name": "Enterprise",
"id": "enterprise",
Expand Down Expand Up @@ -1936,43 +1973,6 @@
}
]
},
{
"name": "Custom Distributed Task Execution",
"id": "dte",
"description": "Custom DTE on your own CI provider",
"itemList": [
{
"name": "GitHub Actions Custom DTE",
"id": "github-dte",
"file": "nx-cloud/enterprise/dte/github-dte"
},
{
"name": "Circle CI Custom DTE",
"id": "circle-ci-dte",
"file": "nx-cloud/enterprise/dte/circle-ci-dte"
},
{
"name": "Azure Pipelines Custom DTE",
"id": "azure-dte",
"file": "nx-cloud/enterprise/dte/azure-dte"
},
{
"name": "Bitbucket Pipelines Custom DTE",
"id": "bitbucket-dte",
"file": "nx-cloud/enterprise/dte/bitbucket-dte"
},
{
"name": "GitLab Custom DTE",
"id": "gitlab-dte",
"file": "nx-cloud/enterprise/dte/gitlab-dte"
},
{
"name": "Jenkins Custom DTE",
"id": "jenkins-dte",
"file": "nx-cloud/enterprise/dte/jenkins-dte"
}
]
},
{
"name": "Conformance",
"id": "conformance",
Expand Down
6 changes: 3 additions & 3 deletions docs/nx-cloud/enterprise/dte/azure-dte.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Custom Distributed Task Execution on Azure Pipelines
# Manual Distributed Task Execution on Azure Pipelines

Using [Nx Agents](/ci/features/distribute-task-execution) is the easiest way to distribute task execution, but it your organization may not be able to use hosted Nx Agents. With an [enterprise license](/enterprise), you can set up distributed task execution on your own CI provider using the recipe below.
Using [Nx Agents](/ci/features/distribute-task-execution) is the easiest way to distribute task execution, but it your organization may not be able to use hosted Nx Agents. You can set up distributed task execution on your own CI provider using the recipe below.

## Run Custom Agents on Azure Pipelines
## Run agents on Azure Pipelines

Run agents directly on Azure Pipelines with the workflow below:

Expand Down
6 changes: 3 additions & 3 deletions docs/nx-cloud/enterprise/dte/bitbucket-dte.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Custom Distributed Task Execution on Bitbucket Pipelines
# Manual Distributed Task Execution on Bitbucket Pipelines

Using [Nx Agents](/ci/features/distribute-task-execution) is the easiest way to distribute task execution, but it your organization may not be able to use hosted Nx Agents. With an [enterprise license](/enterprise), you can set up distributed task execution on your own CI provider using the recipe below.
Using [Nx Agents](/ci/features/distribute-task-execution) is the easiest way to distribute task execution, but it your organization may not be able to use hosted Nx Agents. You can set up distributed task execution on your own CI provider using the recipe below.

## Run Custom Agents on Bitbucket Pipelines
## Run agents on Bitbucket Pipelines

Run agents directly on Bitbucket Pipelines with the workflow below:

Expand Down
6 changes: 3 additions & 3 deletions docs/nx-cloud/enterprise/dte/circle-ci-dte.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Custom Distributed Task Execution on Circle CI
# Manual Distributed Task Execution on Circle CI

Using [Nx Agents](/ci/features/distribute-task-execution) is the easiest way to distribute task execution, but it your organization may not be able to use hosted Nx Agents. With an [enterprise license](/enterprise), you can set up distributed task execution on your own CI provider using the recipe below.
Using [Nx Agents](/ci/features/distribute-task-execution) is the easiest way to distribute task execution, but it your organization may not be able to use hosted Nx Agents. You can set up distributed task execution on your own CI provider using the recipe below.

## Run Custom Agents on Circle CI
## Run agents on Circle CI

Run agents directly on Circle CI with the workflow below:

Expand Down
4 changes: 2 additions & 2 deletions docs/nx-cloud/enterprise/dte/github-dte.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Custom Distributed Task Execution on Github Actions
# Manual Distributed Task Execution on Github Actions

Using [Nx Agents](/ci/features/distribute-task-execution) is the easiest way to distribute task execution, but it your organization may not be able to use hosted Nx Agents. With an [enterprise license](/enterprise), you can set up distributed task execution on your own CI provider using the recipe below.
Using [Nx Agents](/ci/features/distribute-task-execution) is the easiest way to distribute task execution, but it your organization may not be able to use hosted Nx Agents. You can set up distributed task execution on your own CI provider using the recipe below.

## Run Custom Agents on GitHub

Expand Down
6 changes: 3 additions & 3 deletions docs/nx-cloud/enterprise/dte/gitlab-dte.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Custom Distributed Task Execution on GitLab
# Manual Distributed Task Execution on GitLab

Using [Nx Agents](/ci/features/distribute-task-execution) is the easiest way to distribute task execution, but it your organization may not be able to use hosted Nx Agents. With an [enterprise license](/enterprise), you can set up distributed task execution on your own CI provider using the recipe below.
Using [Nx Agents](/ci/features/distribute-task-execution) is the easiest way to distribute task execution, but it your organization may not be able to use hosted Nx Agents. You can set up distributed task execution on your own CI provider using the recipe below.

## Run Custom Agents on GitLab
## Run agents on GitLab

Run agents directly on GitLab with the workflow below:

Expand Down
6 changes: 3 additions & 3 deletions docs/nx-cloud/enterprise/dte/jenkins-dte.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Custom Distributed Task Execution on Jenkins
# Manual Distributed Task Execution on Jenkins

Using [Nx Agents](/ci/features/distribute-task-execution) is the easiest way to distribute task execution, but it your organization may not be able to use hosted Nx Agents. With an [enterprise license](/enterprise), you can set up distributed task execution on your own CI provider using the recipe below.
Using [Nx Agents](/ci/features/distribute-task-execution) is the easiest way to distribute task execution, but it your organization may not be able to use hosted Nx Agents. You can set up manual distributed task execution on your own CI provider using the recipe below.

## Distribute Tasks Across Custom Agents on Jenkins
## Distribute Tasks Across agents on Jenkins

Run agents directly on Jenkins with the workflow below:

Expand Down
10 changes: 5 additions & 5 deletions docs/nx-cloud/reference/assignment-rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Assignment rules allow you to control which tasks can run on which agents. Save on agent costs by provisioning different sizes of agents to suite the individual needs of your tasks. You can ensure resource intensive targets like `e2e-ci` and `build` have what they need by using larger agents. Lighter tasks like `lint` and `test` can run on smaller agents.

Assignment rules are defined in `yaml` files within your workspace's `.nx/workflows` directory. You can use assignment rules with self-hosted agents or with [dynamic Nx agents](/ci/features/dynamic-agents). Note that additional configuration is required when using self-hosted agents.
Assignment rules are defined in `yaml` files within your workspace's `.nx/workflows` directory. You can use assignment rules with [Manual istibuted Task Execution (DTE)](/ci/recipes/dte) or with [dynamic Nx agents](/ci/features/dynamic-agents). Note that additional configuration is required when using Manual DTE.

## How to Define an Assignment Rule

Expand All @@ -26,9 +26,9 @@ You can mix and match any of the criteria in an assignment rule provided that yo
- There is at least one [agent type](/ci/reference/launch-templates) specified in the `runs-on` field.
- Every changeset in your `distribute-on` field must include at **least one agent** that matches each agent type specified in the `runs-on` field across all assignment rules. For example, if your rules distribute tasks on `linux-small-js`, `linux-medium-js`, and `linux-large-js`, then at least one agent of each type must be available; otherwise, tasks associated with those rules cannot be executed.

{% callout type="note" title="If you are using self-hosted agents, you must define your own agent types" %}
You must define your own agent types and attach them to your self-hosted agents using the `NX_AGENT_LAUNCH_TEMPLATE` environment variable. Ensure that for each `runs-on` field in your assignment rules, you have corresponding agents in your agent pool that have the same agent type.
See below for an [example](#using-assignment-rules-with-selfhosted-agents) of how to define your own agent types when using self-hosted agents.
{% callout type="note" title="If you are using Manual DTE, you must define your own agent types" %}
You must define your own agent types and attach them to your agents using the `NX_AGENT_LAUNCH_TEMPLATE` environment variable. Ensure that for each `runs-on` field in your assignment rules, you have corresponding agents in your agent pool that have the same agent type.
See below for an [example](#using-assignment-rules-with-manual-dte) of how to define your own agent types when using Manual DTE.
{% /callout %}

## Assignment Rule Precedence
Expand Down Expand Up @@ -72,7 +72,7 @@ assignment-rules:
- linux-large-js
```

## Using Assignment Rules with Self-Hosted Agents
## Using Assignment Rules with Manual DTE

A typical `assignment-rules.yaml` file might look like this:

Expand Down
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
# Distributed Task Execution at Scale
# Nx Agents at Scale

Repo contains:

- 5 shared buildable packages/libraries with 250 components each
- 5 Next.js applications built out of 20 app-specific libraries. Each app-specific lib has 250 components each. Each library uses the shared components.

The repo shows how Nx distributed task execution can make the CI 16 times faster with a small configuration change.
The repo shows how Nx Agents can make the CI 16 times faster with a small configuration change.

{% github-repository url="https://github.com/vsavkin/interstellar" /%}
16 changes: 8 additions & 8 deletions docs/shared/reference/sitemap.md
Original file line number Diff line number Diff line change
Expand Up @@ -200,7 +200,7 @@
- [Benchmarks](/showcase/benchmarks)
- [Typescript Batch Mode Compilation](/showcase/benchmarks/tsc-batch-mode)
- [Large Repo and Caching](/showcase/benchmarks/caching)
- [Large Repo and DTE](/showcase/benchmarks/dte)
- [Large Repo and Nx Agents](/showcase/benchmarks/nx-agents)
- [Reference](/reference)
- [Commands](/reference/nx-commands)
- [Nx Configuration](/reference/nx-json)
Expand Down Expand Up @@ -301,6 +301,13 @@
- [Enable Bitbucket PR Integration](/ci/recipes/source-control-integration/bitbucket)
- [Enable GitLab MR Integration](/ci/recipes/source-control-integration/gitlab)
- [Enable Azure DevOps PR Integration](/ci/recipes/source-control-integration/azure-devops)
- [Manual Distributed Task Execution](/ci/recipes/dte)
- [GitHub Actions Manual DTE](/ci/recipes/dte/github-dte)
- [Circle CI Manual DTE](/ci/recipes/dte/circle-ci-dte)
- [Azure Pipelines Manual DTE](/ci/recipes/dte/azure-dte)
- [Bitbucket Pipelines Manual DTE](/ci/recipes/dte/bitbucket-dte)
- [GitLab Manual DTE](/ci/recipes/dte/gitlab-dte)
- [Jenkins Manual DTE](/ci/recipes/dte/jenkins-dte)
- [Enterprise](/ci/recipes/enterprise)
- [Single Tenant](/ci/recipes/enterprise/single-tenant)
- [Single Tenant Nx Cloud Hosting](/ci/recipes/enterprise/single-tenant/overview)
Expand All @@ -310,13 +317,6 @@
- [Authenticate with BitBucket Data Center](/ci/recipes/enterprise/single-tenant/auth-bitbucket-data-center)
- [Authenticate via SAML](/ci/recipes/enterprise/single-tenant/auth-saml)
- [Custom GitHub App](/ci/recipes/enterprise/single-tenant/custom-github-app)
- [Custom Distributed Task Execution](/ci/recipes/enterprise/dte)
- [GitHub Actions Custom DTE](/ci/recipes/enterprise/dte/github-dte)
- [Circle CI Custom DTE](/ci/recipes/enterprise/dte/circle-ci-dte)
- [Azure Pipelines Custom DTE](/ci/recipes/enterprise/dte/azure-dte)
- [Bitbucket Pipelines Custom DTE](/ci/recipes/enterprise/dte/bitbucket-dte)
- [GitLab Custom DTE](/ci/recipes/enterprise/dte/gitlab-dte)
- [Jenkins Custom DTE](/ci/recipes/enterprise/dte/jenkins-dte)
- [Conformance](/ci/recipes/enterprise/conformance)
- [Configure Conformance Rules in Nx Cloud](/ci/recipes/enterprise/conformance/configure-conformance-rules-in-nx-cloud)
- [Publish Conformance Rules to Nx Cloud](/ci/recipes/enterprise/conformance/publish-conformance-rules-to-nx-cloud)
Expand Down
11 changes: 11 additions & 0 deletions nx-dev/nx-dev/redirect-rules.js
Original file line number Diff line number Diff line change
Expand Up @@ -1183,6 +1183,16 @@ const enterpriseNxSection = {
'/nx-enterprise/activate-powerpack',
};

const manualDTEUpdate = {
'/ci/recipes/enterprise/dte': '/ci/recipes/dte',
'/ci/recipes/enterprise/dte/github-dte': '/ci/recipes/dte/github-dte',
'/ci/recipes/enterprise/dte/circle-ci-dte': '/ci/recipes/dte/circle-ci-dte',
'/ci/recipes/enterprise/dte/azure-dte': '/ci/recipes/dte/azure-dte',
'/ci/recipes/enterprise/dte/bitbucket-dte': '/ci/recipes/dte/bitbucket-dte',
'/ci/recipes/enterprise/dte/gitlab-dte': '/ci/recipes/dte/gitlab-dte',
'/ci/recipes/enterprise/dte/jenkins-dte': '/ci/recipes/dte/jenkins-dte',
'/showcase/benchmarks/dte': '/showcase/benchmarks/nx-agents',
};
/**
* Public export API
*/
Expand Down Expand Up @@ -1217,4 +1227,5 @@ module.exports = {
featurePagesUpdate,
marketing,
enterpriseNxSection,
manualDTEUpdate,
};
Loading