diff --git a/README.md b/README.md
index 58cebae52f..6e543a791c 100644
--- a/README.md
+++ b/README.md
@@ -89,6 +89,11 @@ site you're working on.
 For example, the [manifest for the docs section](https://github.com/cert-manager/website/blob/master/content/docs/manifest.json)
 contains the expected path for every file.
 
+If you're adding a top-level page which should only appear in the `docs/` section (such as the existing "contributing" section)
+then add `"x-only-docs": true` underneath the title in `manifest.json`. This will cause that section to be removed when a new versioned docs section.
+
+Likewise, if a folder shouldn't be copied from `docs/` to a versioned section, add a file called `.x-only-docs` to that folder, and it will be removed from any newly created versioned documentation.
+
 ### Task: Changing OpenGraph / social sharing tags
 
 These tags are defined in Next.js code and config.
diff --git a/content/docs/contributing/.x-only-docs b/content/docs/contributing/.x-only-docs
new file mode 100644
index 0000000000..3686018617
--- /dev/null
+++ b/content/docs/contributing/.x-only-docs
@@ -0,0 +1,3 @@
+This is a marker file indicating that this folder should not be copied when freezing docs and should only be visible in `content/docs`.
+
+See `scripts/freeze-docs` for more details.
diff --git a/content/docs/contributing/release-process.md b/content/docs/contributing/release-process.md
index 87825f01bc..198674947c 100644
--- a/content/docs/contributing/release-process.md
+++ b/content/docs/contributing/release-process.md
@@ -256,20 +256,16 @@ page if a step is missing or if it is outdated.
       +"cert_manager_latest_version": "v1.14.3",
       ```
 
-   5. (**final release only**) Freeze the `docs/` folder by creating a copy ,
-     removing the pages from that copy that don't make sense to be versioned,
-     and updating the `manifest.json` file:
+   5. (**final release only**) Freeze the `docs/` folder by running the following script:
 
       ```bash
-      export RELEASE=1.15
-      cp -r content/docs content/v${RELEASE}-docs
-      rm -rf content/v${RELEASE}-docs/{release-notes,contributing}
-      sed -i.bak "s|/docs/|/v${RELEASE}-docs/|g" content/v${RELEASE}-docs/manifest.json
-      jq < content/v${RELEASE}-docs/manifest.json >/tmp/manifest \
-        'del(.routes[0].routes[] | select(.title | test("Releases|Contributing")))'
-      mv /tmp/manifest content/v${RELEASE}-docs/manifest.json
+      # From the website repository, on the master branch.
+      ./scripts/freeze-docs 1.16
       ```
 
+      This copies the `docs/` folder to a versioned folder (e.g. `v1.15-docs`) and removes any
+      folders which should not be present in versioned docs.
+
    6. (**final + patch releases**) Update the [API docs](https://cert-manager.io/docs/reference/api-docs/) and [CLI docs](https://cert-manager.io/docs/cli//):
 
       ```bash
diff --git a/content/docs/manifest.json b/content/docs/manifest.json
index 58442bb795..b04c784690 100644
--- a/content/docs/manifest.json
+++ b/content/docs/manifest.json
@@ -14,6 +14,7 @@
         },
         {
           "title": "Releases",
+          "x-only-docs": true,
           "routes": [
             {
               "title": "Supported Releases",
@@ -696,6 +697,7 @@
         },
         {
           "title": "Contributing",
+          "x-only-docs": true,
           "routes": [
             {
               "title": "Introduction",
diff --git a/content/docs/releases/.x-only-docs b/content/docs/releases/.x-only-docs
new file mode 100644
index 0000000000..3686018617
--- /dev/null
+++ b/content/docs/releases/.x-only-docs
@@ -0,0 +1,3 @@
+This is a marker file indicating that this folder should not be copied when freezing docs and should only be visible in `content/docs`.
+
+See `scripts/freeze-docs` for more details.
diff --git a/scripts/freeze-docs b/scripts/freeze-docs
new file mode 100755
index 0000000000..ec0fc7b204
--- /dev/null
+++ b/scripts/freeze-docs
@@ -0,0 +1,52 @@
+#!/usr/bin/env bash
+
+# Copyright 2025 The cert-manager Authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+set -o errexit
+set -o nounset
+set -o pipefail
+
+# This script "freezes" the docs folder, copying docs which are relevant for a
+# specific cert-manager release into their own folder so they can be browsed
+# individually.
+#
+# For example, upon the release of cert-manager v1.17 we use this script to
+# "freeze" cert-manager v1.16, creating content/v1.16-docs and removing some
+# fields which aren't relevant (such as release notes or "contributing" docs)
+
+RELEASE=${1:-}
+
+if [[ -z "${RELEASE}" ]]; then
+    echo "usage: $0 <release-version>"
+    echo "e.g. $0 1.16"
+    exit 1
+fi
+
+cp -r content/docs content/v${RELEASE}-docs
+
+# Delete any folders in the new v$RELEASE-docs section which contain a file
+# named ".x-only-docs"
+rm -rf $(find content/v${RELEASE}-docs -name ".x-only-docs" -execdir pwd \;)
+
+TMP_FILE=$(mktemp)
+
+trap "rm -rf ${TMP_FILE}" EXIT
+
+# Replace paths containing /docs/ with a versioned path
+sed "s|/docs/|/v${RELEASE}-docs/|g" content/v${RELEASE}-docs/manifest.json > $TMP_FILE
+
+# Remove any parts of the manifest which have "x-only-docs": true
+jq <$TMP_FILE > content/v${RELEASE}-docs/manifest.json \
+    'del(.routes[0].routes[] | select(."x-only-docs" == true))'