Substitute dynamically generated content in Markdown files
Warning
- With
docsub, every documentation file may become executable. - Never use
docsubto process files from untrusted sources. - This project is in experimental state, syntax and functionality may change significantly.
- If still want to try it, use pinned package version
docsub==0.8.3
- Embed static files
- Embed command execution results
- Idempotent substitutions
- Invisible non-intrusive markup using comment blocks
- Plays nicely with other markups
- Extensible with project-local commands
- Configurable with config files and env vars
- Manage partially duplicate docs for multiple destinations
- Manage docs for monorepositories
- Embed CLI reference in docs
- Embed dynamically generated content:
- Project metadata
- Test reports
- Models evaluation results
Note
This file uses docsub itself. Dig into raw markup if interested.
- ...a documentation engine like Sphinx or MkDocs
- ...a full-featured static website generator like Pelican
- ...a templating engine like Jinja
- ...a replacement for Bump My Version
$ uv run docsub apply -i README.md
# Title
<!-- docsub: begin -->
<!-- docsub: include info.md -->
<!-- docsub: include features.md -->
...
<!-- docsub: end -->
## Table
<!-- docsub: begin -->
<!-- docsub: include data.md -->
<!-- docsub: lines after 2 -->
| Col 1 | Col 2 |
|-------|-------|
...
<!-- docsub: end -->
## Code
<!-- docsub: begin #code -->
<!-- docsub: include func.py -->
<!-- docsub: lines after 1 upto -1 -->
```python
...
```
<!-- docsub: end #code --> |
> Long description.* Feature 1
* Feature 2
* Feature 3| Key 1 | value 1 |
| Key 2 | value 2 |
| Key 3 | value 3 |def func():
pass |
and keep it updated!
# Title
<!-- docsub: begin -->
<!-- docsub: include info.md -->
<!-- docsub: include features.md -->
> Long description.
* Feature 1
* Feature 2
* Feature 3
<!-- docsub: end -->
## Table
<!-- docsub: begin -->
<!-- docsub: include data.md -->
<!-- docsub: lines after 2 -->
| Col 1 | Col 2 |
|-------|-------|
| Key 1 | value 1 |
| Key 2 | value 2 |
| Key 3 | value 3 |
<!-- docsub: end -->
## Code
<!-- docsub: begin #code -->
<!-- docsub: include func.py -->
<!-- docsub: lines after 1 upto -1 -->
```python
def func():
pass
```
<!-- docsub: end #code -->Recommended. The most flexible installation option, allowing project-local commands to utilize project codebase.
# pyproject.toml
[dependency-groups]
dev = [
"docsub==0.8.3",
]Works for simple cases.
uv tool install docsub==0.8.3The syntax is purposefully verbose. This is fine, you are not supposed to edit it often. But it's searchable and sticks in eye when scrolling down large documents.
Docsub uses line-based substitution syntax based on directives and substitution blocks.
Markdown directive is one-line comment:
<!-- docsub: <directive> [directive args] -->
There are multiple directive types.
Markdown substitution block is a sequence of lines, starting with begin directive and ending with end directive.
<!-- docsub: begin -->
<!-- docsub: help docsub -->
<!-- docsub: include CHANGELOG.md -->
Inner text will be replaced.
<!-- docsub: this whole line is treated as plain text -->
This text will be replaced too.
<!-- docsub: end -->One or many other directives must come at the top of the block, otherwise they are treated as plain text. Blocks without producing directives are not allowed. Block's inner text will be replaced upon substitution, unless modifier directives are used, e.g. lines.
If docsub substitution block lies inside markdown fenced code block, it is not substituted (example: fenced code blocks above and below this paragraph, see the raw markup). To put dynamic content into a fenced code block, place begin and end around it and use lines after N upto -M (example: Usage section).
For nested blocks, only top level substitution is performed. Use block #identifier to distinguish between nesting levels.
<!-- docsub: begin #top -->
<!-- docsub: include part.md -->
<!-- docsub: begin -->
<!-- docsub: include nested.md -->
<!-- docsub: end -->
<!-- docsub: end #top -->When substitution block is indented, the indentation is preserved:
* List item
<!-- docsub: begin -->
<!-- docsub: include sublist.md -->
* Sub-item 1
* Sub-item 2
* Sub-item 3
<!-- docsub: end -->- Block delimiters:
begin,end - Producing commands:
exec,help,include,x - Modifying commands:
lines,strip
begin [#identifier]
Open substitution target block. To distinguish between nesting levels, use block #identifier, starting with #.
end [#identifier]
Close substitution target block.
exec <shell commands>
Execute <shell commands> with sh -c and substitute stdout. Allows pipes and other shell functionality. If possible, avoid using this directive.
cmd.exec.work_dir— shell working directory, default'.'cmd.exec.env_vars— dict of additional environment variables, default{}
help <command> [subcommand...]
help python -m <command> [subcommand...]
Display help for CLI utility or Python module. Use this command to document CLI instead of exec. Runs command [subcommand...] --help or python -m command [subcommand...] --help respectively. Directive args must be a space-separated sequence of characters [-._a-zA-Z0-9].
cmd.help.env_vars— dict of additional environment variables, default{}
include path/to/file
Literally include file specified by path relative to base_dir config option.
cmd.include.base_dir— base directory for relative paths
lines [after N] [upto -M]
Upon substitution, keep original target block lines: first N and/or last M. Only one lines command is allowed inside the block.
strip
Strip whitespace in substitution result:
- initial and trailing blank lines
- trailing whitespace on every line
x <project-command> [args and --options]
Execute project-local command declared in docsubfile.py in project root. The naming is inspired by X- HTTP headers and x- convention for reusable YAML sections.
cmd.x.docsubfile— path to file with project-local commands, absolute or relative to project root (default:docsubfile.py)
When project root contains file docsubfile.py with commands defined as in example below, they can be used in docsub: x directive. Project commands must be defined as click command and gathered under x group. There is no need to install click separately as docsub depends on it.
If docsub is installed globally and called as uvx docsub, project commands in docsubfile.py have access to docsub dependencies only: click, loguru, rich (see docsub's pyproject.toml for details).
If docsub is installed as project dev dependency and called as uv run docsub, user commands also have access to project modules and dev dependencies. This allows more flexible scenarios.
Project command author can get access to docsub Environment object (including command configs) from click context object (see example below). The docsub Environment object has some useful methods (not documented yet).
$ uv run docsub apply -i sample.md<!-- docsub: begin -->
<!-- docsub: x say-hello Alice Bob -->
Hi there, Alice!
Hi there, Bob!
<!-- docsub: end -->from docsub import Environment, click, pass_env
@click.group()
def x() -> None:
pass
@x.command()
@click.argument('users', nargs=-1)
def say_hello(users: tuple[str, ...]) -> None:
for user in users:
click.echo(f'Hi there, {user}!')
@x.command()
@click.argument('users', nargs=-1)
@pass_env
def log_hello(env: Environment, users: tuple[str, ...]) -> None:
base = env.get_temp_dir('log_hello')
(base / 'hello.log').write_text(f'said hello to {users}')Docsub exposes x as CLI command, letting project commands to be executed with project settings:
$ uv run docsub x say-hello Alice Bob
Hi there, Alice!
Hi there, Bob!Configuration resolution order
- command line options (to be documented)
- environment variables (to be documented)
docsub.tomlconfig file in current working directorypyproject.toml, section[tool.docsub](to be implemented)- default config values
local_dir— internal working directory at the project root (default:.docsub)
See Commands.
(to be documented)
(to be documented)
All config keys are optional.
local_dir = ".docsub" # default
[logging]
#level = "DEBUG" # default: missing, logging disabled
[cmd.exec]
env_vars = {} # default
work_dir = "." # default
[cmd.help.env_vars]
COLUMNS = "60" # more compact
[cmd.include]
base_dir = "." # default
[cmd.x]
docsubfile = "docsubfile.py" # defaultWarning
In future releases config keys will be moved under [tool.docsub] root for both pyproject.toml and docsub.toml, this will be a breaking change.
Docsub uses loguru for logging. Logging is disabled by default. To enable logging, set config option level to one of logging levels supported by loguru.
(logging is rudimentary at the moment)
$ docsub --help
Usage: python -m docsub [OPTIONS] COMMAND [ARGS]...
╭─ Options ──────────────────────────────────────────────────────────╮
│ --config-file -c PATH │
│ --local-dir -l PATH │
│ --cmd-exec-work-dir PATH │
│ --cmd-exec-env-vars TEXT │
│ --cmd-help-env-vars TEXT │
│ --cmd-include-base-dir PATH │
│ --cmd-x-docsubfile -x PATH │
│ --version Show the version and exit. │
│ --help Show this message and exit. │
╰────────────────────────────────────────────────────────────────────╯
╭─ Commands ─────────────────────────────────────────────────────────╮
│ apply Update Markdown files with embedded content. │
│ x Project-local commands. │
╰────────────────────────────────────────────────────────────────────╯$ docsub apply --help
Usage: python -m docsub apply [OPTIONS] FILES...
Update Markdown files with embedded content.
Read FILES and perform substitutions one by one. If one file depends
on another, place it after that file.
╭─ Options ──────────────────────────────────────────────────────────╮
│ --in-place -i Process files in-place │
│ --help Show this message and exit. │
╰────────────────────────────────────────────────────────────────────╯$ docsub x --help
Usage: python -m docsub x [OPTIONS] COMMAND [ARGS]...
Project-local commands.
╭─ Options ──────────────────────────────────────────────────────────╮
│ --help Show this message and exit. │
╰────────────────────────────────────────────────────────────────────╯This project appeared to maintain docs for multipython project. You may check it up for usage examples.