From ce3b4348e2fcc3c7b88264d18d924cd49f107c40 Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Tue, 31 Dec 2024 08:31:08 +0800 Subject: [PATCH 01/21] docs: how to use api --- docs/how-to/index.md | 12 ++ docs/how-to/use-pebble-api.md | 281 ++++++++++++++++++++++++++++++++++ docs/reference/api.md | 25 ++- docs/reuse/api.md | 7 + 4 files changed, 311 insertions(+), 14 deletions(-) create mode 100644 docs/how-to/use-pebble-api.md create mode 100644 docs/reuse/api.md diff --git a/docs/how-to/index.md b/docs/how-to/index.md index 0e5725502..bf22db51e 100644 --- a/docs/how-to/index.md +++ b/docs/how-to/index.md @@ -37,3 +37,15 @@ Use named "identities" to allow additional users to access the API. Manage identities ``` + + +## API + +To integrate Pebble with your automated workflows, you can use Pebble API. + +```{toctree} +:titlesonly: +:maxdepth: 1 + +Use Pebble API +``` diff --git a/docs/how-to/use-pebble-api.md b/docs/how-to/use-pebble-api.md new file mode 100644 index 000000000..4728b18e4 --- /dev/null +++ b/docs/how-to/use-pebble-api.md @@ -0,0 +1,281 @@ +# How to use Pebble API + +Pebble, as a service manager, provides a command-line interface (CLI) for managing services, similar to systemd. However, Pebble distinguishes itself with its dedicated REST API. + +While you can interact with systemd programmatically, it's typically done through [D-Bus](https://en.wikipedia.org/wiki/D-Bus) (a message-oriented middleware mechanism that allows communication between multiple processes running concurrently on the same machine), which has a steeper learning curve and requires more specialized code compared to using a REST API. + +This is where Pebble shines, because in contrast, Pebble's REST API offers a simpler, more standardized approach to service management, making it easier to integrate into automated workflows like Continuous Integration/Continuous Deployment (CI/CD) pipelines, unlocking much more potential of Pebble. + +--- + +## API and automated workflows + +While the CLI is convenient for manual operations, the API enables integration with automated workflows. + +Consider a CI pipeline that needs to setup some services before running tests and teardown the environment afterwards. Rather than relying on shell scripts and CLI commands which are error-prone and hard to maintain, the Pebble API offers a more robust and structured (service orchestration as code) approach. You can programmatically start, stop, and query the status and health of services before running tests in your CI workflows, allowing for tighter integration and more reliable automation. + +For example, your CI workflow could start a service or even a group of inter-dependent services, verify services are running and health checks are OK, run integration tests against those services, and finally stop the services as a cleanup step. This automation reduces intervention, reduces risks of human errors, and ensures consistency and idempotency across different environments. + +The Pebble API offers a significant advantage over CLI-based approaches in CI/CD pipelines. First of all, the structured interface simplifies interactions, eliminating the need for parsing command output. Second, API enables more sophisticated error handling. Last but not least, using the API promotes code reusability and maintainability, because instead of hard-coding shell commands throughout the CI workflow definition, we can encapsulate service operations within dedicated functions and modules, which leads to cleaner and more maintainable workflows. + +--- + +## Pebble API + +```{include} /reuse/api.md + :start-after: Start: Pebble API overview + :end-before: End: Pebble API overview +``` + +For more information, see [API](/reference/api). + +--- + +## Using the API + +You can use different tools and clients to access the API. For more examples, see [API and clients](../explanation/api-and-clients) and the [API reference doc](../reference/api). + +### curl + +At the moment (v1.17.0), only a handful of Pebble APIs are exposed via HTTP, most APIs are only available via the linux websocket. With curl, there is a `H `--unix-socket ` parameter, which allows us to connect through this Unix domain socket instead of using the network. In this section, let's see how we can use curl to access Pebble API via the unix socket. + +Suppose we start the Pebble daemon with no default services and an empty layer: + +```{terminal} + :input: pebble run +2024-12-30T01:07:10.275Z [pebble] Started daemon. +2024-12-30T01:07:10.281Z [pebble] POST /v1/services 75.042µs 400 +2024-12-30T01:07:10.281Z [pebble] Cannot start default services: no default services +``` + +To get the services, run: + +```bash +curl --unix-socket $PEBBLE/.pebble.socket -XGET http://localhost/v1/services +``` + +And we should get the following JSON response showing that there is no service: + +```json +{"type":"sync","status-code":200,"status":"OK","result":[]} +``` + +To add a simple layer with one service. run: + +```bash +curl --unix-socket $PEBBLE/.pebble.socket -XPOST http://localhost/v1/layers -d '{"action": "add", "combine": true, "inner": true, "label": "ci", "format": "yaml", "layer": "summary: ci\nservices:\n svc1:\n override: replace\n command: sleep 100\n"}' +``` + +If we get the services again by running `curl --unix-socket $PEBBLE/.pebble.socket -XGET http://localhost/v1/services`, we will get a different response showing we have a service now but it's not started: + +```json +{ + "type": "sync", + "status-code": 200, + "status": "OK", + "result": [ + { + "name": "svc1", + "startup": "disabled", + "current": "inactive" + } + ] +} +``` + +To start the newly added service, run: + +```bash +curl --unix-socket $PEBBLE/.pebble.socket -XPOST http://localhost/v1/services -d '{"action": "start", "services": ["svc1"]}' +``` + +This endpoint is of type "async", returning a change ID instead: + +```json +{"type":"async","status-code":202,"status":"Accepted","change":"1","result":null} +``` + +To wait for the change to be finished, run: + +```bash +curl --unix-socket $PEBBLE/.pebble.socket -XGET http://localhost/v1/changes/1/wait +``` + +And we shall get a response similar to the following: + +```json +{ + "type": "sync", + "status-code": 200, + "status": "OK", + "result": { + "id": "1", + "kind": "start", + "summary": "Start service \"svc1\"", + "status": "Done", + "tasks": [ + { + "id": "1", + "kind": "start", + "summary": "Start service \"svc1\"", + "status": "Done", + "progress": { + "label": "", + "done": 1, + "total": 1 + }, + "spawn-time": "2024-12-30T08:57:43.194037353+08:00", + "ready-time": "2024-12-30T08:57:44.21087777+08:00" + } + ], + "ready": true, + "spawn-time": "2024-12-30T08:57:43.194065853+08:00", + "ready-time": "2024-12-30T08:57:44.210879687+08:00" + } +} +``` + +If we try to get the services again, we can see it's active now. + +We can integrate these steps in our CI workflow to start services and dependencies, and now we can run some tests against them. Of course, after the CI workflow is finished, we want to do some cleanup like stopping those services. + +To stop a service, run: + +```bash +curl --unix-socket $PEBBLE/.pebble.socket -XPOST http://localhost/v1/services -d '{"action": "stop", "services": ["svc1"]}' +``` + +As aforementioned, this endpoint is `async`. We will get a response similar to: + +```json +{"type":"async","status-code":202,"status":"Accepted","change":"2","result":null} +``` + +We can run the same command above to wait for the change to be finished and verify the status by getting the services. + +### Go client + +We can also use the Go client to achieve the same result: + +```go +package main + +import ( + "fmt" + + "github.com/canonical/pebble/client" +) + +func main() { + pebble, err := client.New(&client.Config{Socket: "/home/ubuntu/PEBBLE_HOME/.pebble.socket"}) + if err != nil { + panic(err) + } + + // get services + _, err = pebble.Services(&client.ServicesOptions{}) + if err != nil { + panic(err) + } + + // add layer + layerYAML := ` +services: + svc1: + override: replace + command: sleep 100 +` + err = pebble.AddLayer(&client.AddLayerOptions{ + Combine: true, + Label: "ci", + LayerData: []byte(layerYAML), + }) + if err != nil { + panic(err) + } + + // start services + changeID, err := pebble.Start(&client.ServiceOptions{Names: []string{"svc1"}}) + if err != nil { + panic(err) + } + + // wait for the change + _, err = pebble.WaitChange(changeID, &client.WaitChangeOptions{}) + if err != nil { + panic(err) + } + + // get services, the service svc1 should be active + services, err := pebble.Services(&client.ServicesOptions{}) + if err != nil { + panic(err) + } + for _, svc := range services { + fmt.Printf("The status of service %s is: %s.\n", svc.Name, svc.Current) + } + + // Now we can run some tests against those services. + + // stop services + _, err = pebble.Stop(&client.ServiceOptions{Names: []string{"svc1"}}) + if err != nil { + panic(err) + } + + // wait for the change + _, err = pebble.WaitChange(changeID, &client.WaitChangeOptions{}) + if err != nil { + panic(err) + } +} +``` + +### Python client + +Here is an example to achieve the same result with Python: + +```python +import ops + +client = ops.pebble.Client("/home/ubuntu/PEBBLE_HOME/.pebble.socket") + +# get services +services = client.get_services() +assert len(services) == 0 + +# add layer +layerYAML = """ +services: + svc1: + override: replace + command: sleep 100 +""" +client.add_layer(label="ci", layer=layerYAML, combine=True) + +# start services +changeID = client.start_services(["svc1"]) + +# wait for the change +client.wait_change(changeID) + +# get services, the service svc1 should be active +services = client.get_services() +for svc in services: + print(f"The status of service {svc.name} is: {svc.current}.") + +# stop services +changeID = client.stop_services(["svc1"]) + +# wait for the change +client.wait_change(changeID) +``` + +--- + +## See more + +- [Go client](https://pkg.go.dev/github.com/canonical/pebble/client) +- [Python client for Pebble API](https://ops.readthedocs.io/en/latest/reference/pebble.html). +- [API and clients](../explanation/api-and-clients) +- [API](../reference/api) diff --git a/docs/reference/api.md b/docs/reference/api.md index caa2f9486..35d603ded 100644 --- a/docs/reference/api.md +++ b/docs/reference/api.md @@ -1,30 +1,27 @@ # API -Pebble exposes an HTTP API that remote clients can use to interact with the daemon. The API has endpoints for starting and stopping services, adding configuration layers to the plan, and so on. - -The API uses HTTP over a Unix socket, with access to the API controlled by user ID. If `pebble run` is started with the `--http
` option, Pebble exposes a limited set of open-access endpoints (see {ref}`api-access-levels`) using the given TCP address. +```{include} /reuse/api.md + :start-after: Start: Pebble API overview + :end-before: End: Pebble API overview +``` ## Using the API You can use different tools and clients to access the API. -For more examples, see "How to use Pebble API". +For more examples, see [How to use Pebble API](../how-to/use-pebble-api). ### curl To access the API endpoints over the Unix socket, use the `--unix-socket` option of `curl`. For example: -* TODO - - ``` - curl ... - ``` - -* TODO +```bash +curl --unix-socket $PEBBLE/.pebble.socket -XGET http://localhost/v1/services +``` - ``` - curl ... - ``` +```bash +curl --unix-socket $PEBBLE/.pebble.socket -XPOST http://localhost/v1/services -d '{"action": "start", "services": ["svc1"]}' +``` ### Go client diff --git a/docs/reuse/api.md b/docs/reuse/api.md new file mode 100644 index 000000000..7923923e5 --- /dev/null +++ b/docs/reuse/api.md @@ -0,0 +1,7 @@ +Start: Pebble API overview + +Pebble exposes an HTTP API that remote clients can use to interact with the daemon. The API has endpoints for starting and stopping services, adding configuration layers to the plan, and so on. + +The API uses HTTP over a Unix socket, with access to the API controlled by user ID. If `pebble run` is started with the `--http
` option, Pebble exposes a limited set of open-access endpoints (see {ref}`api-access-levels`) using the given TCP address. + +End: Pebble API overview From be84674b7e58059811445e647a4349c823bfe364 Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Sat, 4 Jan 2025 13:21:36 +0800 Subject: [PATCH 02/21] chore: refactor according to review --- docs/how-to/index.md | 2 +- ...se-pebble-api.md => use-the-pebble-api.md} | 38 ++++++++----------- 2 files changed, 17 insertions(+), 23 deletions(-) rename docs/how-to/{use-pebble-api.md => use-the-pebble-api.md} (62%) diff --git a/docs/how-to/index.md b/docs/how-to/index.md index bf22db51e..0dcdcd3ce 100644 --- a/docs/how-to/index.md +++ b/docs/how-to/index.md @@ -41,7 +41,7 @@ Manage identities ## API -To integrate Pebble with your automated workflows, you can use Pebble API. +To integrate Pebble with your automated workflows, you can use the Pebble API. ```{toctree} :titlesonly: diff --git a/docs/how-to/use-pebble-api.md b/docs/how-to/use-the-pebble-api.md similarity index 62% rename from docs/how-to/use-pebble-api.md rename to docs/how-to/use-the-pebble-api.md index 4728b18e4..349bf1461 100644 --- a/docs/how-to/use-pebble-api.md +++ b/docs/how-to/use-the-pebble-api.md @@ -1,43 +1,37 @@ -# How to use Pebble API +# How to use the Pebble API to manage services -Pebble, as a service manager, provides a command-line interface (CLI) for managing services, similar to systemd. However, Pebble distinguishes itself with its dedicated REST API. +Pebble provides a command-line interface (CLI) for managing services, similar to systemd. However, Pebble distinguishes itself with its dedicated REST API. -While you can interact with systemd programmatically, it's typically done through [D-Bus](https://en.wikipedia.org/wiki/D-Bus) (a message-oriented middleware mechanism that allows communication between multiple processes running concurrently on the same machine), which has a steeper learning curve and requires more specialized code compared to using a REST API. - -This is where Pebble shines, because in contrast, Pebble's REST API offers a simpler, more standardized approach to service management, making it easier to integrate into automated workflows like Continuous Integration/Continuous Deployment (CI/CD) pipelines, unlocking much more potential of Pebble. +This guide demonstrates how to use the Pebble API to programmatically manage services as part of an automated workflow. --- ## API and automated workflows -While the CLI is convenient for manual operations, the API enables integration with automated workflows. +As an example scenario, consider an automated workflow that starts and tests a group of inter-dependent services as part of a continuous integration pipeline. The tests could include verifying that the services are running, querying their status and health, and running integration tests. After testing the services, the workflow stops the services. -Consider a CI pipeline that needs to setup some services before running tests and teardown the environment afterwards. Rather than relying on shell scripts and CLI commands which are error-prone and hard to maintain, the Pebble API offers a more robust and structured (service orchestration as code) approach. You can programmatically start, stop, and query the status and health of services before running tests in your CI workflows, allowing for tighter integration and more reliable automation. +Although you could use shell scripts and CLI commands to implement the workflow, the Pebble API enables a more robust and maintainable approach: -For example, your CI workflow could start a service or even a group of inter-dependent services, verify services are running and health checks are OK, run integration tests against those services, and finally stop the services as a cleanup step. This automation reduces intervention, reduces risks of human errors, and ensures consistency and idempotency across different environments. +- You don't need to parse command output. +- You can handle errors in a more sophisticated way. +- You can encapsulate service operations within reusable functions and modules. -The Pebble API offers a significant advantage over CLI-based approaches in CI/CD pipelines. First of all, the structured interface simplifies interactions, eliminating the need for parsing command output. Second, API enables more sophisticated error handling. Last but not least, using the API promotes code reusability and maintainability, because instead of hard-coding shell commands throughout the CI workflow definition, we can encapsulate service operations within dedicated functions and modules, which leads to cleaner and more maintainable workflows. +This approach reduces manual intervention and the risk of human error. It also supports consistency and idempotency across different environments. --- -## Pebble API - +## Using the API ```{include} /reuse/api.md :start-after: Start: Pebble API overview :end-before: End: Pebble API overview ``` - -For more information, see [API](/reference/api). +For reference information about the API, see [](../explanation/api-and-clients) and [](../reference/api). --- -## Using the API - -You can use different tools and clients to access the API. For more examples, see [API and clients](../explanation/api-and-clients) and the [API reference doc](../reference/api). - ### curl -At the moment (v1.17.0), only a handful of Pebble APIs are exposed via HTTP, most APIs are only available via the linux websocket. With curl, there is a `H `--unix-socket ` parameter, which allows us to connect through this Unix domain socket instead of using the network. In this section, let's see how we can use curl to access Pebble API via the unix socket. +Most API endpoints are only available via the Unix socket. With curl, there is a `--unix-socket ` parameter, which allows us to connect through this Unix socket instead of using the network. In this section, let's see how we can use curl to access the API via the Unix socket. Suppose we start the Pebble daemon with no default services and an empty layer: @@ -60,7 +54,7 @@ And we should get the following JSON response showing that there is no service: {"type":"sync","status-code":200,"status":"OK","result":[]} ``` -To add a simple layer with one service. run: +To add a simple layer with one service, run: ```bash curl --unix-socket $PEBBLE/.pebble.socket -XPOST http://localhost/v1/layers -d '{"action": "add", "combine": true, "inner": true, "label": "ci", "format": "yaml", "layer": "summary: ci\nservices:\n svc1:\n override: replace\n command: sleep 100\n"}' @@ -137,7 +131,7 @@ And we shall get a response similar to the following: If we try to get the services again, we can see it's active now. -We can integrate these steps in our CI workflow to start services and dependencies, and now we can run some tests against them. Of course, after the CI workflow is finished, we want to do some cleanup like stopping those services. +We can integrate these steps in our automated workflow to start services and their dependencies. Then we're able to run some tests against the services. As a final step in the workflow, we'll want to stop the services. To stop a service, run: @@ -145,7 +139,7 @@ To stop a service, run: curl --unix-socket $PEBBLE/.pebble.socket -XPOST http://localhost/v1/services -d '{"action": "stop", "services": ["svc1"]}' ``` -As aforementioned, this endpoint is `async`. We will get a response similar to: +This endpoint is `async`, so we'll get a response similar to: ```json {"type":"async","status-code":202,"status":"Accepted","change":"2","result":null} @@ -276,6 +270,6 @@ client.wait_change(changeID) ## See more - [Go client](https://pkg.go.dev/github.com/canonical/pebble/client) -- [Python client for Pebble API](https://ops.readthedocs.io/en/latest/reference/pebble.html). +- [Python client for Pebble API](https://ops.readthedocs.io/en/latest/reference/pebble.html) - [API and clients](../explanation/api-and-clients) - [API](../reference/api) From 092702cef95286c608fdc19ec32e1c58af71254a Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Mon, 6 Jan 2025 10:22:34 +0800 Subject: [PATCH 03/21] docs: how to manage remote systems draft --- docs/how-to/index.md | 12 +++ docs/how-to/manage-remote-systems.md | 135 +++++++++++++++++++++++++ docs/reference/pebble-in-containers.md | 19 ---- 3 files changed, 147 insertions(+), 19 deletions(-) create mode 100644 docs/how-to/manage-remote-systems.md delete mode 100644 docs/reference/pebble-in-containers.md diff --git a/docs/how-to/index.md b/docs/how-to/index.md index 0e5725502..b5d26548f 100644 --- a/docs/how-to/index.md +++ b/docs/how-to/index.md @@ -27,6 +27,18 @@ Manage service dependencies ``` +## Manage remote systems + +Pebble provides exec and file management APIs to coordinate with remote systems. + +```{toctree} +:titlesonly: +:maxdepth: 1 + +Manage remote systems +``` + + ## Identities Use named "identities" to allow additional users to access the API. diff --git a/docs/how-to/manage-remote-systems.md b/docs/how-to/manage-remote-systems.md new file mode 100644 index 000000000..a344dcd9a --- /dev/null +++ b/docs/how-to/manage-remote-systems.md @@ -0,0 +1,135 @@ +# How to use Pebble to manage remote systems + +Managing a cluster of servers remotely is no mean task. It is not only time-consuming since servers need regular software updates, configuration changes, file transfer, and command executions, but also presents security challenges because remote access requires opening ports which increases the potential attack surface. + +While it's possible to manually execute commands on remote servers and transfer files from/to remote servers, it is not the most efficient (or secure) approach. There exist several other methods for remote command execution and system administration. Secure Shell (SSH) is one of them, and there are also dedicated configuration management tools like Ansible. However, to use these tools, an extra port for SSH needs to be opened, which not only increases the operational overhead of managing public/private keys but also the potential attack surface. + +--- + +## Pebble in remote systems + +Pebble provides various commands and APIs to run commands and manage files and directories in remote systems. With these capabilities, if we run Pebble in a remote system like in a separate container, we can use Pebble APIs coordinate with the remote system without opening a new port. + +To know more about Pebble APIs, read [API and clients](/explanation/api-and-clients) and the [API spec](/reference/api). + +--- + +## Run commands in remote systems + +One common task in system administration is to update and install packages in remote servers. With Pebble, we can use the `pebble exec` command to achieve this. + +For example, if Pebble is running with a user with root privilege, we can use this command to update and install packages in remote systems: + +```{terminal} +:input: pebble exec apt update +Hit:1 http://ports.ubuntu.com/ubuntu-ports noble InRelease +Get:2 http://ports.ubuntu.com/ubuntu-ports noble-updates InRelease [126 kB] +Get:3 http://ports.ubuntu.com/ubuntu-ports noble-backports InRelease [126 kB] +... +Fetched 3121 kB in 38s (81.5 kB/s) +Reading package lists... Done +Building dependency tree... Done +Reading state information... Done +41 packages can be upgraded. Run 'apt list --upgradable' to see them. +``` + +To install a package, run: + +```bash +pebble exec apt install cowsay +``` + +To confirm the package is successfully installed in the remote system, run: + +```{terminal} +:input: pebble exec cowsay moo + _____ +< moo > + ----- + \ ^__^ + \ (oo)\_______ + (__)\ )\/\ + ||----w | + || || +``` + +For more information on the Pebble `exec` command, see {ref}`reference_pebble_exec_command`. + +--- + +## Manage files in remote systems + +Another common task in remote systems is configuration management, like updating configuration files, pushing files to the servers, pulling files generated by running services, and so on. + +With Pebble, this can be done easily. For example, if we want to install Nginx, create our own website, and serve it on a newly created virtual host, we can first install Nginx: + +```bash +pebble exec apt install nginx +``` + +Then create our website: + +```bash +sudo mkdir -p /var/www/pebble + +sudo bash -c 'cat < /var/www/pebble/index.html +Hello, Pebble! +EOF' +``` + +Create a virtual host configuration _locally_: + +``` +cat < ~/pebble +server { + listen 81; + listen [::]:81; + + server_name example.ubuntu.com; + + root /var/www/pebble; + index index.html; + + location / { + try_files $uri $uri/ =404; + } +} +EOF +``` + +Push the file to the remote system: + +```bash +pebble push ~/pebble /etc/nginx/sites-enabled/pebble +``` + +Activate the newly added virtual host: + +```bash +pebble exec service nginx restart +``` + +And test our result: + +```{terminal} +:input: curl localhost:81 +Hello, Pebble! +``` + +For more information on Pebble files related commands, see: + +- {ref}`reference_pebble_ls_command` +- {ref}`reference_pebble_mkdir_command` +- {ref}`reference_pebble_rm_command` +- {ref}`reference_pebble_push_command` +- {ref}`reference_pebble_pull_command` + +--- + +## Use Pebble APIs + +--- + +## See more + +A summary, more links to Pebble CLI reference docs. \ No newline at end of file diff --git a/docs/reference/pebble-in-containers.md b/docs/reference/pebble-in-containers.md deleted file mode 100644 index 7329b4b3b..000000000 --- a/docs/reference/pebble-in-containers.md +++ /dev/null @@ -1,19 +0,0 @@ -# Pebble in containers - -Pebble works well as a local service manager, but if running Pebble in a separate container, you can use the exec and file management APIs to coordinate with the remote system over the shared unix socket. - -## Run commands in a container - -To run commands in a container, see {ref}`reference_pebble_exec_command`. - -## File management - -Pebble provides various API calls and commands to manage files and directories on the server. - -The simplest way to use these is with the commands below, several of which should be familiar: - -- {ref}`reference_pebble_ls_command` -- {ref}`reference_pebble_mkdir_command` -- {ref}`reference_pebble_rm_command` -- {ref}`reference_pebble_push_command` -- {ref}`reference_pebble_pull_command` From 593715ac4c22de3c9b61ca1c4a3552e60a6cb0ae Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Mon, 6 Jan 2025 10:46:53 +0800 Subject: [PATCH 04/21] chore: fix links --- docs/how-to/index.md | 2 +- docs/reference/api.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/how-to/index.md b/docs/how-to/index.md index 0dcdcd3ce..fe9ef68aa 100644 --- a/docs/how-to/index.md +++ b/docs/how-to/index.md @@ -47,5 +47,5 @@ To integrate Pebble with your automated workflows, you can use the Pebble API. :titlesonly: :maxdepth: 1 -Use Pebble API +Use Pebble API ``` diff --git a/docs/reference/api.md b/docs/reference/api.md index 35d603ded..80e176dbb 100644 --- a/docs/reference/api.md +++ b/docs/reference/api.md @@ -9,7 +9,7 @@ You can use different tools and clients to access the API. -For more examples, see [How to use Pebble API](../how-to/use-pebble-api). +For more examples, see [How to use Pebble API](../how-to/use-the-pebble-api). ### curl From ea98b5768a055afe035b1ffa6f64a4849bcad52c Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Mon, 6 Jan 2025 10:49:44 +0800 Subject: [PATCH 05/21] chore: refactor after merging changes from how to use api --- docs/how-to/manage-remote-systems.md | 52 ++++++++++++++++++---------- 1 file changed, 34 insertions(+), 18 deletions(-) diff --git a/docs/how-to/manage-remote-systems.md b/docs/how-to/manage-remote-systems.md index a344dcd9a..a74c3ffeb 100644 --- a/docs/how-to/manage-remote-systems.md +++ b/docs/how-to/manage-remote-systems.md @@ -1,22 +1,20 @@ # How to use Pebble to manage remote systems -Managing a cluster of servers remotely is no mean task. It is not only time-consuming since servers need regular software updates, configuration changes, file transfer, and command executions, but also presents security challenges because remote access requires opening ports which increases the potential attack surface. +Managing a cluster of servers remotely is no mean task. It is not only time-consuming since servers need regular software updates, configuration changes, file transfers, and command executions, but it also presents security challenges because remote access requires opening ports, which increases the potential attack surface. -While it's possible to manually execute commands on remote servers and transfer files from/to remote servers, it is not the most efficient (or secure) approach. There exist several other methods for remote command execution and system administration. Secure Shell (SSH) is one of them, and there are also dedicated configuration management tools like Ansible. However, to use these tools, an extra port for SSH needs to be opened, which not only increases the operational overhead of managing public/private keys but also the potential attack surface. +While it's possible to manually execute commands on remote servers and transfer files from/to remote servers, it is not the most efficient (or secure) approach. Several other methods exist for remote command execution and system administration. Secure Shell (SSH) is one of them, and there are also dedicated configuration management tools like Ansible. However, to use these tools, an extra port for SSH needs to be opened, which not only increases the operational overhead of managing public/private keys but also the potential attack surface. --- ## Pebble in remote systems -Pebble provides various commands and APIs to run commands and manage files and directories in remote systems. With these capabilities, if we run Pebble in a remote system like in a separate container, we can use Pebble APIs coordinate with the remote system without opening a new port. - -To know more about Pebble APIs, read [API and clients](/explanation/api-and-clients) and the [API spec](/reference/api). +Pebble provides various commands and APIs to run commands and manage files and directories in remote systems. With these capabilities, if we run Pebble in a remote system like in a separate container, we can use Pebble APIs to coordinate with the remote system without opening a new port. --- ## Run commands in remote systems -One common task in system administration is to update and install packages in remote servers. With Pebble, we can use the `pebble exec` command to achieve this. +One common task in system administration is updating and installing packages on remote servers. With Pebble, we can use the `pebble exec` command to achieve this. For example, if Pebble is running with a user with root privilege, we can use this command to update and install packages in remote systems: @@ -67,20 +65,28 @@ With Pebble, this can be done easily. For example, if we want to install Nginx, pebble exec apt install nginx ``` -Then create our website: +Then create our website _locally_: ```bash -sudo mkdir -p /var/www/pebble +echo "Hello, Pebble!" > /tmp/index.html +``` -sudo bash -c 'cat < /var/www/pebble/index.html -Hello, Pebble! -EOF' +Create a directory in the remote system: + +```bash +pebble mkdir -p /var/www/pebble +``` + +Push our local website file to the remote system: + +```bash +pebble push /tmp/index.html /var/www/pebble/index.html ``` Create a virtual host configuration _locally_: ``` -cat < ~/pebble +cat < /tmp/pebble server { listen 81; listen [::]:81; @@ -91,7 +97,7 @@ server { index index.html; location / { - try_files $uri $uri/ =404; + try_files \$uri \$uri/ =404; } } EOF @@ -100,23 +106,23 @@ EOF Push the file to the remote system: ```bash -pebble push ~/pebble /etc/nginx/sites-enabled/pebble +pebble push /tmp/pebble /etc/nginx/sites-enabled/pebble ``` -Activate the newly added virtual host: +Activate the newly added virtual host in the remote system by restarting Nginx: ```bash pebble exec service nginx restart ``` -And test our result: +Finally, we can test the final result: ```{terminal} :input: curl localhost:81 Hello, Pebble! ``` -For more information on Pebble files related commands, see: +For more information on files related commands in Pebble, see: - {ref}`reference_pebble_ls_command` - {ref}`reference_pebble_mkdir_command` @@ -128,8 +134,18 @@ For more information on Pebble files related commands, see: ## Use Pebble APIs +In previous sections, we used Pebble commands to execute commands and manage files. For automation purposes, we can also do the same with Pebble API. + +For example, to push a file to the remote system, we can POST to the `/v1/files` endpoint: + +```bash +curl --unix-socket $PEBBLE/.pebble.socket -XPOST localhost:4000/v1/files -H "Content-Type: multipart/form-data" -F request='{"action": "write", "files": [{"path": "/var/www/pebble/index.html", "make-dirs": true, "permissions": "644"}]}' -F 'files=@/tmp/index.html;type=application/octet-stream;filename=/var/www/pebble/index.html' +``` + --- ## See more -A summary, more links to Pebble CLI reference docs. \ No newline at end of file +In this guide, we used Pebble commands and curl to access the APIs. We can also access Pebble API using the Go client and Python client. For more information, see [How to use the Pebble API to manage services](/how-to/use-the-pebble-api). + +For more information on the API, read the [API spec](/reference/api). From bdfe190abe66e193389a72f1461a9e3f7e39d60e Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Mon, 6 Jan 2025 10:54:46 +0800 Subject: [PATCH 06/21] chore: fix broken links --- docs/how-to/index.md | 2 +- docs/index.md | 1 - docs/reference/api.md | 2 +- docs/reference/cli-commands.md | 12 ++++++------ docs/reference/index.md | 8 -------- 5 files changed, 8 insertions(+), 17 deletions(-) diff --git a/docs/how-to/index.md b/docs/how-to/index.md index fcbd6fde6..b457e3d96 100644 --- a/docs/how-to/index.md +++ b/docs/how-to/index.md @@ -59,5 +59,5 @@ To integrate Pebble with your automated workflows, you can use the Pebble API. :titlesonly: :maxdepth: 1 -Use Pebble API +Use Pebble API ``` diff --git a/docs/index.md b/docs/index.md index 398138f04..3e654a920 100644 --- a/docs/index.md +++ b/docs/index.md @@ -34,7 +34,6 @@ Pebble is useful for developers who are building [Juju charms on Kubernetes](htt **Technical information** - [Layer specification](reference/layer-specification) - [CLI commands](reference/cli-commands) -- [Pebble in containers](reference/pebble-in-containers) ``` ```` diff --git a/docs/reference/api.md b/docs/reference/api.md index 85364649a..bd4ceed86 100644 --- a/docs/reference/api.md +++ b/docs/reference/api.md @@ -9,7 +9,7 @@ You can use different tools and clients to access the API. -For more examples, see [How to use Pebble API](../how-to/use-pebble-api). +For more examples, see [How to use Pebble API](../how-to/use-the-pebble-api). ### curl diff --git a/docs/reference/cli-commands.md b/docs/reference/cli-commands.md index 7a48494bb..2636a0176 100644 --- a/docs/reference/cli-commands.md +++ b/docs/reference/cli-commands.md @@ -191,7 +191,7 @@ error: cannot perform the following tasks: - exec command "sleep" (timed out after 1s: context deadline exceeded) ``` -Read more: [Use Pebble in containers](pebble-in-containers.md). +Read more: [How to use Pebble to manage remote systems](/how-to/manage-remote-systems.md). (reference_pebble_health_command)= @@ -410,7 +410,7 @@ may be specified for the last path element. ``` -Read more: [Use Pebble in containers](pebble-in-containers.md). +Read more: [How to use Pebble to manage remote systems](/how-to/manage-remote-systems.md). (reference_pebble_mkdir_command)= @@ -437,7 +437,7 @@ The mkdir command creates the specified directory. ``` -Read more: [Use Pebble in containers](pebble-in-containers.md). +Read more: [How to use Pebble to manage remote systems](/how-to/manage-remote-systems.md). (reference_pebble_notice_command)= @@ -640,7 +640,7 @@ The pull command retrieves a file from the remote system. ``` -Read more: [Use Pebble in containers](pebble-in-containers.md). +Read more: [How to use Pebble to manage remote systems](/how-to/manage-remote-systems.md). (reference_pebble_push_command)= @@ -666,7 +666,7 @@ The push command transfers a file to the remote system. ``` -Read more: [Use Pebble in containers](pebble-in-containers.md). +Read more: [How to use Pebble to manage remote systems](/how-to/manage-remote-systems.md). (reference_pebble_remove-identities_command)= @@ -810,7 +810,7 @@ The rm command removes a file or directory. ``` -Read more: [Use Pebble in containers](pebble-in-containers.md). +Read more: [How to use Pebble to manage remote systems](/how-to/manage-remote-systems.md). (reference_pebble_run_command)= diff --git a/docs/reference/index.md b/docs/reference/index.md index 92d327f06..d46af8908 100644 --- a/docs/reference/index.md +++ b/docs/reference/index.md @@ -20,7 +20,6 @@ Layers Layer specification Log forwarding Notices -Pebble in containers Service auto-restart ``` @@ -43,13 +42,6 @@ The `pebble` command has several subcommands. * [CLI commands](cli-commands) -## Pebble in containers - -When the Pebble daemon is running inside a remote system (for example, a separate container), you can manage the remote system using subcommands on the Pebble client. - -* [Pebble in containers](pebble-in-containers) - - ## Service failures Pebble provides two ways to automatically restart services when they fail. Auto-restart is based on exit codes from services. Health checks are a more sophisticated way to test and report the availability of services. From 59279ea1a1585d881719725199fb53d1aa13e14d Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Mon, 6 Jan 2025 16:28:58 +0800 Subject: [PATCH 07/21] chore: add links --- docs/how-to/manage-remote-systems.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/how-to/manage-remote-systems.md b/docs/how-to/manage-remote-systems.md index a74c3ffeb..8a817f40d 100644 --- a/docs/how-to/manage-remote-systems.md +++ b/docs/how-to/manage-remote-systems.md @@ -2,7 +2,7 @@ Managing a cluster of servers remotely is no mean task. It is not only time-consuming since servers need regular software updates, configuration changes, file transfers, and command executions, but it also presents security challenges because remote access requires opening ports, which increases the potential attack surface. -While it's possible to manually execute commands on remote servers and transfer files from/to remote servers, it is not the most efficient (or secure) approach. Several other methods exist for remote command execution and system administration. Secure Shell (SSH) is one of them, and there are also dedicated configuration management tools like Ansible. However, to use these tools, an extra port for SSH needs to be opened, which not only increases the operational overhead of managing public/private keys but also the potential attack surface. +While it's possible to manually execute commands on remote servers and transfer files from/to remote servers, it is not the most efficient (or secure) approach. Several other methods exist for remote command execution and system administration. Secure Shell (SSH) is one of them, and there are also dedicated configuration management tools like Ansible. However, to use these tools, an extra port for SSH needs to be opened, which not only increases the operational overhead of managing public/private keys but also the potential attack surface. Remember March 29, 2024 which seemed to be another normal Friday? A developer shocked the world by revealing an XZ Utils (data-compression utilities) backdoor which could potentially enable unauthorized access via SSH and remote code execution (read the full story [here](https://www.ssh.com/blog/a-recap-of-the-openssh-and-xz-liblzma-incident)). --- From f305d68c8639df2d1601e11995ac3187a7c8c5e7 Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Tue, 7 Jan 2025 17:05:08 +0800 Subject: [PATCH 08/21] chore: refactor after review --- docs/how-to/index.md | 2 +- docs/how-to/manage-remote-systems.md | 24 +++++++++++++----------- 2 files changed, 14 insertions(+), 12 deletions(-) diff --git a/docs/how-to/index.md b/docs/how-to/index.md index b457e3d96..871ee94f8 100644 --- a/docs/how-to/index.md +++ b/docs/how-to/index.md @@ -29,7 +29,7 @@ Manage service dependencies ## Manage remote systems -Pebble provides exec and file management APIs to coordinate with remote systems. +Pebble provides exec and file management functions to coordinate with remote systems. ```{toctree} :titlesonly: diff --git a/docs/how-to/manage-remote-systems.md b/docs/how-to/manage-remote-systems.md index 8a817f40d..ed3842e1b 100644 --- a/docs/how-to/manage-remote-systems.md +++ b/docs/how-to/manage-remote-systems.md @@ -1,14 +1,16 @@ # How to use Pebble to manage remote systems -Managing a cluster of servers remotely is no mean task. It is not only time-consuming since servers need regular software updates, configuration changes, file transfers, and command executions, but it also presents security challenges because remote access requires opening ports, which increases the potential attack surface. - -While it's possible to manually execute commands on remote servers and transfer files from/to remote servers, it is not the most efficient (or secure) approach. Several other methods exist for remote command execution and system administration. Secure Shell (SSH) is one of them, and there are also dedicated configuration management tools like Ansible. However, to use these tools, an extra port for SSH needs to be opened, which not only increases the operational overhead of managing public/private keys but also the potential attack surface. Remember March 29, 2024 which seemed to be another normal Friday? A developer shocked the world by revealing an XZ Utils (data-compression utilities) backdoor which could potentially enable unauthorized access via SSH and remote code execution (read the full story [here](https://www.ssh.com/blog/a-recap-of-the-openssh-and-xz-liblzma-incident)). +Managing a cluster of servers remotely is no mean feat. It is time-consuming because servers need regular software updates, configuration changes, file transfers, and command executions. It also presents security challenges because remote access requires opening ports, which increases the potential attack surface. --- -## Pebble in remote systems +## Why use Pebble? + +Pebble provides commands and an HTTP API to run commands and manage files and directories in remote systems. If we run Pebble in a remote system, for example in a separate container, we can use Pebble to coordinate with the remote system without opening a new port. + +While it's possible to manually run commands on remote servers and transfer files from/to remote servers, this isn't the most efficient or secure approach. Even a configuration management tool such as Ansible requires an extra port to be opened for Secure Shell (SSH), which not only increases the operational overhead of managing public/private keys but also the potential attack surface. -Pebble provides various commands and APIs to run commands and manage files and directories in remote systems. With these capabilities, if we run Pebble in a remote system like in a separate container, we can use Pebble APIs to coordinate with the remote system without opening a new port. +Remember March 29, 2024 which seemed to be another normal Friday? A developer shocked the world by revealing an XZ Utils (data-compression utilities) backdoor which could potentially enable unauthorized access via SSH and remote code execution (read the full story [here](https://en.wikipedia.org/wiki/XZ_Utils_backdoor)). --- @@ -16,7 +18,7 @@ Pebble provides various commands and APIs to run commands and manage files and d One common task in system administration is updating and installing packages on remote servers. With Pebble, we can use the `pebble exec` command to achieve this. -For example, if Pebble is running with a user with root privilege, we can use this command to update and install packages in remote systems: +For example, if Pebble is running as user with root privileges, we can use this command to update and install packages in remote systems: ```{terminal} :input: pebble exec apt update @@ -57,7 +59,7 @@ For more information on the Pebble `exec` command, see {ref}`reference_pebble_ex ## Manage files in remote systems -Another common task in remote systems is configuration management, like updating configuration files, pushing files to the servers, pulling files generated by running services, and so on. +Another common task in remote systems is configuration management. For example, updating configuration files, pushing files to servers, pulling files generated by running services, and so on. With Pebble, this can be done easily. For example, if we want to install Nginx, create our own website, and serve it on a newly created virtual host, we can first install Nginx: @@ -122,7 +124,7 @@ Finally, we can test the final result: Hello, Pebble! ``` -For more information on files related commands in Pebble, see: +For more information about related Pebble commands, see: - {ref}`reference_pebble_ls_command` - {ref}`reference_pebble_mkdir_command` @@ -132,9 +134,9 @@ For more information on files related commands in Pebble, see: --- -## Use Pebble APIs +## Use the API -In previous sections, we used Pebble commands to execute commands and manage files. For automation purposes, we can also do the same with Pebble API. +In the previous sections, we used Pebble commands to run commands and manage files. When automating these tasks, we can use the HTTP API that Pebble provides. For example, to push a file to the remote system, we can POST to the `/v1/files` endpoint: @@ -146,6 +148,6 @@ curl --unix-socket $PEBBLE/.pebble.socket -XPOST localhost:4000/v1/files -H "Con ## See more -In this guide, we used Pebble commands and curl to access the APIs. We can also access Pebble API using the Go client and Python client. For more information, see [How to use the Pebble API to manage services](/how-to/use-the-pebble-api). +In this guide, we used Pebble commands and curl to access the API. We can also access the API using Go and Python. For more information, see [How to use the Pebble API to manage services](/how-to/use-the-pebble-api). For more information on the API, read the [API spec](/reference/api). From c6f074655ee44b5766e661852b694050ad349069 Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Thu, 9 Jan 2025 09:58:21 +0800 Subject: [PATCH 09/21] chore: remove separating lines --- docs/how-to/use-the-pebble-api.md | 8 -------- 1 file changed, 8 deletions(-) diff --git a/docs/how-to/use-the-pebble-api.md b/docs/how-to/use-the-pebble-api.md index 349bf1461..efe88d9dc 100644 --- a/docs/how-to/use-the-pebble-api.md +++ b/docs/how-to/use-the-pebble-api.md @@ -4,8 +4,6 @@ Pebble provides a command-line interface (CLI) for managing services, similar to This guide demonstrates how to use the Pebble API to programmatically manage services as part of an automated workflow. ---- - ## API and automated workflows As an example scenario, consider an automated workflow that starts and tests a group of inter-dependent services as part of a continuous integration pipeline. The tests could include verifying that the services are running, querying their status and health, and running integration tests. After testing the services, the workflow stops the services. @@ -18,8 +16,6 @@ Although you could use shell scripts and CLI commands to implement the workflow, This approach reduces manual intervention and the risk of human error. It also supports consistency and idempotency across different environments. ---- - ## Using the API ```{include} /reuse/api.md :start-after: Start: Pebble API overview @@ -27,8 +23,6 @@ This approach reduces manual intervention and the risk of human error. It also s ``` For reference information about the API, see [](../explanation/api-and-clients) and [](../reference/api). ---- - ### curl Most API endpoints are only available via the Unix socket. With curl, there is a `--unix-socket ` parameter, which allows us to connect through this Unix socket instead of using the network. In this section, let's see how we can use curl to access the API via the Unix socket. @@ -265,8 +259,6 @@ changeID = client.stop_services(["svc1"]) client.wait_change(changeID) ``` ---- - ## See more - [Go client](https://pkg.go.dev/github.com/canonical/pebble/client) From 766af89b3a44265095f0103ef737f390ddae4d11 Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Thu, 9 Jan 2025 09:59:49 +0800 Subject: [PATCH 10/21] chore: remove separating lines --- docs/how-to/manage-remote-systems.md | 10 ---------- 1 file changed, 10 deletions(-) diff --git a/docs/how-to/manage-remote-systems.md b/docs/how-to/manage-remote-systems.md index ed3842e1b..d60c04a39 100644 --- a/docs/how-to/manage-remote-systems.md +++ b/docs/how-to/manage-remote-systems.md @@ -2,8 +2,6 @@ Managing a cluster of servers remotely is no mean feat. It is time-consuming because servers need regular software updates, configuration changes, file transfers, and command executions. It also presents security challenges because remote access requires opening ports, which increases the potential attack surface. ---- - ## Why use Pebble? Pebble provides commands and an HTTP API to run commands and manage files and directories in remote systems. If we run Pebble in a remote system, for example in a separate container, we can use Pebble to coordinate with the remote system without opening a new port. @@ -12,8 +10,6 @@ While it's possible to manually run commands on remote servers and transfer file Remember March 29, 2024 which seemed to be another normal Friday? A developer shocked the world by revealing an XZ Utils (data-compression utilities) backdoor which could potentially enable unauthorized access via SSH and remote code execution (read the full story [here](https://en.wikipedia.org/wiki/XZ_Utils_backdoor)). ---- - ## Run commands in remote systems One common task in system administration is updating and installing packages on remote servers. With Pebble, we can use the `pebble exec` command to achieve this. @@ -55,8 +51,6 @@ To confirm the package is successfully installed in the remote system, run: For more information on the Pebble `exec` command, see {ref}`reference_pebble_exec_command`. ---- - ## Manage files in remote systems Another common task in remote systems is configuration management. For example, updating configuration files, pushing files to servers, pulling files generated by running services, and so on. @@ -132,8 +126,6 @@ For more information about related Pebble commands, see: - {ref}`reference_pebble_push_command` - {ref}`reference_pebble_pull_command` ---- - ## Use the API In the previous sections, we used Pebble commands to run commands and manage files. When automating these tasks, we can use the HTTP API that Pebble provides. @@ -144,8 +136,6 @@ For example, to push a file to the remote system, we can POST to the `/v1/files` curl --unix-socket $PEBBLE/.pebble.socket -XPOST localhost:4000/v1/files -H "Content-Type: multipart/form-data" -F request='{"action": "write", "files": [{"path": "/var/www/pebble/index.html", "make-dirs": true, "permissions": "644"}]}' -F 'files=@/tmp/index.html;type=application/octet-stream;filename=/var/www/pebble/index.html' ``` ---- - ## See more In this guide, we used Pebble commands and curl to access the API. We can also access the API using Go and Python. For more information, see [How to use the Pebble API to manage services](/how-to/use-the-pebble-api). From 7cf294a798812cc6aade9f6e60d91ecb70a5b4f4 Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Mon, 13 Jan 2025 11:59:16 +0800 Subject: [PATCH 11/21] chore: add a space to trigger another doc build --- docs/how-to/use-the-pebble-api.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/how-to/use-the-pebble-api.md b/docs/how-to/use-the-pebble-api.md index efe88d9dc..1ad3bd4ee 100644 --- a/docs/how-to/use-the-pebble-api.md +++ b/docs/how-to/use-the-pebble-api.md @@ -265,3 +265,4 @@ client.wait_change(changeID) - [Python client for Pebble API](https://ops.readthedocs.io/en/latest/reference/pebble.html) - [API and clients](../explanation/api-and-clients) - [API](../reference/api) + From fda3f07a18aada3d055341d92ec60f7b2daf776e Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Mon, 13 Jan 2025 11:59:33 +0800 Subject: [PATCH 12/21] chore: and remove it --- docs/how-to/use-the-pebble-api.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/how-to/use-the-pebble-api.md b/docs/how-to/use-the-pebble-api.md index 1ad3bd4ee..efe88d9dc 100644 --- a/docs/how-to/use-the-pebble-api.md +++ b/docs/how-to/use-the-pebble-api.md @@ -265,4 +265,3 @@ client.wait_change(changeID) - [Python client for Pebble API](https://ops.readthedocs.io/en/latest/reference/pebble.html) - [API and clients](../explanation/api-and-clients) - [API](../reference/api) - From 3b6f3857f4f4686e75fee6fa5455edee1e98cf7f Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Mon, 20 Jan 2025 13:03:54 +0800 Subject: [PATCH 13/21] chore: refactor after discussion --- docs/how-to/use-the-pebble-api.md | 220 ++---------------------------- docs/reference/api.md | 2 + 2 files changed, 13 insertions(+), 209 deletions(-) diff --git a/docs/how-to/use-the-pebble-api.md b/docs/how-to/use-the-pebble-api.md index efe88d9dc..1aa7283c3 100644 --- a/docs/how-to/use-the-pebble-api.md +++ b/docs/how-to/use-the-pebble-api.md @@ -2,30 +2,16 @@ Pebble provides a command-line interface (CLI) for managing services, similar to systemd. However, Pebble distinguishes itself with its dedicated REST API. -This guide demonstrates how to use the Pebble API to programmatically manage services as part of an automated workflow. - -## API and automated workflows - -As an example scenario, consider an automated workflow that starts and tests a group of inter-dependent services as part of a continuous integration pipeline. The tests could include verifying that the services are running, querying their status and health, and running integration tests. After testing the services, the workflow stops the services. - -Although you could use shell scripts and CLI commands to implement the workflow, the Pebble API enables a more robust and maintainable approach: - -- You don't need to parse command output. -- You can handle errors in a more sophisticated way. -- You can encapsulate service operations within reusable functions and modules. - -This approach reduces manual intervention and the risk of human error. It also supports consistency and idempotency across different environments. +This guide demonstrates how to use the Pebble API to programmatically manage services as part of an automated workflow where we can start, test, and stop interdependent services in a CI pipeline. While shell scripts could be used for this purpose, Pebble API offers a more robust approach with easier error handling and reusable components. This reduces manual work, errors, and ensures consistency across environments. ## Using the API + ```{include} /reuse/api.md :start-after: Start: Pebble API overview :end-before: End: Pebble API overview ``` -For reference information about the API, see [](../explanation/api-and-clients) and [](../reference/api). -### curl - -Most API endpoints are only available via the Unix socket. With curl, there is a `--unix-socket ` parameter, which allows us to connect through this Unix socket instead of using the network. In this section, let's see how we can use curl to access the API via the Unix socket. +For reference information about the API, see [](../explanation/api-and-clients) and [](../reference/api). Suppose we start the Pebble daemon with no default services and an empty layer: @@ -36,197 +22,12 @@ Suppose we start the Pebble daemon with no default services and an empty layer: 2024-12-30T01:07:10.281Z [pebble] Cannot start default services: no default services ``` -To get the services, run: - -```bash -curl --unix-socket $PEBBLE/.pebble.socket -XGET http://localhost/v1/services -``` - -And we should get the following JSON response showing that there is no service: - -```json -{"type":"sync","status-code":200,"status":"OK","result":[]} -``` - -To add a simple layer with one service, run: - -```bash -curl --unix-socket $PEBBLE/.pebble.socket -XPOST http://localhost/v1/layers -d '{"action": "add", "combine": true, "inner": true, "label": "ci", "format": "yaml", "layer": "summary: ci\nservices:\n svc1:\n override: replace\n command: sleep 100\n"}' -``` - -If we get the services again by running `curl --unix-socket $PEBBLE/.pebble.socket -XGET http://localhost/v1/services`, we will get a different response showing we have a service now but it's not started: - -```json -{ - "type": "sync", - "status-code": 200, - "status": "OK", - "result": [ - { - "name": "svc1", - "startup": "disabled", - "current": "inactive" - } - ] -} -``` - -To start the newly added service, run: - -```bash -curl --unix-socket $PEBBLE/.pebble.socket -XPOST http://localhost/v1/services -d '{"action": "start", "services": ["svc1"]}' -``` - -This endpoint is of type "async", returning a change ID instead: - -```json -{"type":"async","status-code":202,"status":"Accepted","change":"1","result":null} -``` - -To wait for the change to be finished, run: - -```bash -curl --unix-socket $PEBBLE/.pebble.socket -XGET http://localhost/v1/changes/1/wait -``` - -And we shall get a response similar to the following: - -```json -{ - "type": "sync", - "status-code": 200, - "status": "OK", - "result": { - "id": "1", - "kind": "start", - "summary": "Start service \"svc1\"", - "status": "Done", - "tasks": [ - { - "id": "1", - "kind": "start", - "summary": "Start service \"svc1\"", - "status": "Done", - "progress": { - "label": "", - "done": 1, - "total": 1 - }, - "spawn-time": "2024-12-30T08:57:43.194037353+08:00", - "ready-time": "2024-12-30T08:57:44.21087777+08:00" - } - ], - "ready": true, - "spawn-time": "2024-12-30T08:57:43.194065853+08:00", - "ready-time": "2024-12-30T08:57:44.210879687+08:00" - } -} -``` - -If we try to get the services again, we can see it's active now. - -We can integrate these steps in our automated workflow to start services and their dependencies. Then we're able to run some tests against the services. As a final step in the workflow, we'll want to stop the services. - -To stop a service, run: - -```bash -curl --unix-socket $PEBBLE/.pebble.socket -XPOST http://localhost/v1/services -d '{"action": "stop", "services": ["svc1"]}' -``` - -This endpoint is `async`, so we'll get a response similar to: - -```json -{"type":"async","status-code":202,"status":"Accepted","change":"2","result":null} -``` - -We can run the same command above to wait for the change to be finished and verify the status by getting the services. - -### Go client - -We can also use the Go client to achieve the same result: - -```go -package main - -import ( - "fmt" - - "github.com/canonical/pebble/client" -) - -func main() { - pebble, err := client.New(&client.Config{Socket: "/home/ubuntu/PEBBLE_HOME/.pebble.socket"}) - if err != nil { - panic(err) - } - - // get services - _, err = pebble.Services(&client.ServicesOptions{}) - if err != nil { - panic(err) - } - - // add layer - layerYAML := ` -services: - svc1: - override: replace - command: sleep 100 -` - err = pebble.AddLayer(&client.AddLayerOptions{ - Combine: true, - Label: "ci", - LayerData: []byte(layerYAML), - }) - if err != nil { - panic(err) - } - - // start services - changeID, err := pebble.Start(&client.ServiceOptions{Names: []string{"svc1"}}) - if err != nil { - panic(err) - } - - // wait for the change - _, err = pebble.WaitChange(changeID, &client.WaitChangeOptions{}) - if err != nil { - panic(err) - } - - // get services, the service svc1 should be active - services, err := pebble.Services(&client.ServicesOptions{}) - if err != nil { - panic(err) - } - for _, svc := range services { - fmt.Printf("The status of service %s is: %s.\n", svc.Name, svc.Current) - } - - // Now we can run some tests against those services. - - // stop services - _, err = pebble.Stop(&client.ServiceOptions{Names: []string{"svc1"}}) - if err != nil { - panic(err) - } - - // wait for the change - _, err = pebble.WaitChange(changeID, &client.WaitChangeOptions{}) - if err != nil { - panic(err) - } -} -``` - -### Python client - -Here is an example to achieve the same result with Python: +Here is an example in Python: ```python import ops -client = ops.pebble.Client("/home/ubuntu/PEBBLE_HOME/.pebble.socket") +client = ops.pebble.Client("/path/to/.pebble.socket") # get services services = client.get_services() @@ -242,23 +43,24 @@ services: client.add_layer(label="ci", layer=layerYAML, combine=True) # start services -changeID = client.start_services(["svc1"]) - -# wait for the change -client.wait_change(changeID) +changeID = client.start_services(["svc1"]) # Python client also waits for change to finish # get services, the service svc1 should be active services = client.get_services() for svc in services: print(f"The status of service {svc.name} is: {svc.current}.") +# Now we can run some tests against those services. + # stop services -changeID = client.stop_services(["svc1"]) +changeID = client.stop_services(["svc1"]) # Python client also waits for change to finish # wait for the change client.wait_change(changeID) ``` +You can also use Go or curl to achieve the same result. For more information, see {ref}`api_go_client` and {ref}`api_curl`. + ## See more - [Go client](https://pkg.go.dev/github.com/canonical/pebble/client) diff --git a/docs/reference/api.md b/docs/reference/api.md index d5e7322e4..81abf47ee 100644 --- a/docs/reference/api.md +++ b/docs/reference/api.md @@ -11,6 +11,7 @@ You can use different tools and clients to access the API. For more examples, see [How to use Pebble API](../how-to/use-the-pebble-api). +(api_curl)= ### curl To access the API endpoints over the Unix socket, use the `--unix-socket` option of `curl`. For example: @@ -27,6 +28,7 @@ To access the API endpoints over the Unix socket, use the `--unix-socket` option {"type":"sync","status-code":200,"status":"OK","result":{...}} ``` +(api_go_client)= ### Go client To use the [Go client](https://pkg.go.dev/github.com/canonical/pebble/client) to access API endpoints over the Unix socket, first create a client using `New`, and then call the methods on the returned `Client` struct to interact with the API. For example, to stop a service named `mysvc`: From 0e0ae4e1292b325d203f145e82a594b20dca1e61 Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Mon, 20 Jan 2025 13:32:13 +0800 Subject: [PATCH 14/21] chore: refactor after discussion --- docs/how-to/manage-remote-systems.md | 26 ++++++++++++-------------- 1 file changed, 12 insertions(+), 14 deletions(-) diff --git a/docs/how-to/manage-remote-systems.md b/docs/how-to/manage-remote-systems.md index d60c04a39..bba72e203 100644 --- a/docs/how-to/manage-remote-systems.md +++ b/docs/how-to/manage-remote-systems.md @@ -1,20 +1,16 @@ -# How to use Pebble to manage remote systems +# How to use Pebble to manage a remote system -Managing a cluster of servers remotely is no mean feat. It is time-consuming because servers need regular software updates, configuration changes, file transfers, and command executions. It also presents security challenges because remote access requires opening ports, which increases the potential attack surface. +Managing server clusters remotely takes time and effort. Servers need regular updates, configuration changes, file transfers, and commands run. Directly managing remote servers or using tools like Ansible, which require SSH, increases overhead and security risks. The [XZ Utils backdoor incident](https://en.wikipedia.org/wiki/XZ_Utils_backdoor) highlighted vulnerabilities associated with SSH access. -## Why use Pebble? +Pebble offers commands and an HTTP API over unix socket for remote system management, avoiding the need for extra open ports. -Pebble provides commands and an HTTP API to run commands and manage files and directories in remote systems. If we run Pebble in a remote system, for example in a separate container, we can use Pebble to coordinate with the remote system without opening a new port. +> Note: "remote system" means clients can interact with the Pebble daemon via a unix socket. -While it's possible to manually run commands on remote servers and transfer files from/to remote servers, this isn't the most efficient or secure approach. Even a configuration management tool such as Ansible requires an extra port to be opened for Secure Shell (SSH), which not only increases the operational overhead of managing public/private keys but also the potential attack surface. +## Run commands in a remote system -Remember March 29, 2024 which seemed to be another normal Friday? A developer shocked the world by revealing an XZ Utils (data-compression utilities) backdoor which could potentially enable unauthorized access via SSH and remote code execution (read the full story [here](https://en.wikipedia.org/wiki/XZ_Utils_backdoor)). +One common task in system administration is updating and installing packages. With Pebble, we can use the `pebble exec` command to achieve this. -## Run commands in remote systems - -One common task in system administration is updating and installing packages on remote servers. With Pebble, we can use the `pebble exec` command to achieve this. - -For example, if Pebble is running as user with root privileges, we can use this command to update and install packages in remote systems: +For example, if Pebble is running as user with root privileges, we can use this command to update and install packages in a remote system: ```{terminal} :input: pebble exec apt update @@ -51,9 +47,9 @@ To confirm the package is successfully installed in the remote system, run: For more information on the Pebble `exec` command, see {ref}`reference_pebble_exec_command`. -## Manage files in remote systems +## Manage files in a remote system -Another common task in remote systems is configuration management. For example, updating configuration files, pushing files to servers, pulling files generated by running services, and so on. +Another common task in a remote system is configuration management. For example, updating configuration files, pushing files to servers, pulling files generated by running services, and so on. With Pebble, this can be done easily. For example, if we want to install Nginx, create our own website, and serve it on a newly created virtual host, we can first install Nginx: @@ -133,9 +129,11 @@ In the previous sections, we used Pebble commands to run commands and manage fil For example, to push a file to the remote system, we can POST to the `/v1/files` endpoint: ```bash -curl --unix-socket $PEBBLE/.pebble.socket -XPOST localhost:4000/v1/files -H "Content-Type: multipart/form-data" -F request='{"action": "write", "files": [{"path": "/var/www/pebble/index.html", "make-dirs": true, "permissions": "644"}]}' -F 'files=@/tmp/index.html;type=application/octet-stream;filename=/var/www/pebble/index.html' +curl --unix-socket /path/to/.pebble.socket -XPOST http://_/v1/files -H "Content-Type: multipart/form-data" -F request='{"action": "write", "files": [{"path": "/var/www/pebble/index.html", "make-dirs": true, "permissions": "644"}]}' -F 'files=@/tmp/index.html;type=application/octet-stream;filename=/var/www/pebble/index.html' ``` +We can also use {ref}`api_go_client` and {ref}`api_python_client` for API access. + ## See more In this guide, we used Pebble commands and curl to access the API. We can also access the API using Go and Python. For more information, see [How to use the Pebble API to manage services](/how-to/use-the-pebble-api). From 14345af8b5ed66fbba8c8ca8cd1ca1ccd20639e0 Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Mon, 20 Jan 2025 13:38:05 +0800 Subject: [PATCH 15/21] chore: refactor after discussion --- docs/how-to/index.md | 4 ++-- ...e-remote-systems.md => manage-a-remote-system.md} | 5 ++--- docs/reference/api.md | 3 +++ docs/reference/cli-commands.md | 12 ++++++------ 4 files changed, 13 insertions(+), 11 deletions(-) rename docs/how-to/{manage-remote-systems.md => manage-a-remote-system.md} (93%) diff --git a/docs/how-to/index.md b/docs/how-to/index.md index 871ee94f8..12ee608ba 100644 --- a/docs/how-to/index.md +++ b/docs/how-to/index.md @@ -27,7 +27,7 @@ Manage service dependencies ``` -## Manage remote systems +## Manage a remote system Pebble provides exec and file management functions to coordinate with remote systems. @@ -35,7 +35,7 @@ Pebble provides exec and file management functions to coordinate with remote sys :titlesonly: :maxdepth: 1 -Manage remote systems +Manage a remote system ``` diff --git a/docs/how-to/manage-remote-systems.md b/docs/how-to/manage-a-remote-system.md similarity index 93% rename from docs/how-to/manage-remote-systems.md rename to docs/how-to/manage-a-remote-system.md index bba72e203..199f0e895 100644 --- a/docs/how-to/manage-remote-systems.md +++ b/docs/how-to/manage-a-remote-system.md @@ -136,6 +136,5 @@ We can also use {ref}`api_go_client` and {ref}`api_python_client` for API access ## See more -In this guide, we used Pebble commands and curl to access the API. We can also access the API using Go and Python. For more information, see [How to use the Pebble API to manage services](/how-to/use-the-pebble-api). - -For more information on the API, read the [API spec](/reference/api). +- [How to use the Pebble API to manage services](/how-to/use-the-pebble-api). +- [API spec](/reference/api). diff --git a/docs/reference/api.md b/docs/reference/api.md index d5e7322e4..763a485a2 100644 --- a/docs/reference/api.md +++ b/docs/reference/api.md @@ -11,6 +11,7 @@ You can use different tools and clients to access the API. For more examples, see [How to use Pebble API](../how-to/use-the-pebble-api). +(api_curl)= ### curl To access the API endpoints over the Unix socket, use the `--unix-socket` option of `curl`. For example: @@ -27,6 +28,7 @@ To access the API endpoints over the Unix socket, use the `--unix-socket` option {"type":"sync","status-code":200,"status":"OK","result":{...}} ``` +(api_go_client)= ### Go client To use the [Go client](https://pkg.go.dev/github.com/canonical/pebble/client) to access API endpoints over the Unix socket, first create a client using `New`, and then call the methods on the returned `Client` struct to interact with the API. For example, to stop a service named `mysvc`: @@ -50,6 +52,7 @@ We try to never change the underlying HTTP API in a backwards-incompatible way. For more information, see the [Go client documentation](https://pkg.go.dev/github.com/canonical/pebble/client). +(api_python_client)= ### Python client The Ops library for writing and testing Juju charms includes a [Python client for Pebble API](https://ops.readthedocs.io/en/latest/reference/pebble.html). You can use the Python client to access the API endpoints over the Unix socket. For example: diff --git a/docs/reference/cli-commands.md b/docs/reference/cli-commands.md index 2636a0176..2a5912123 100644 --- a/docs/reference/cli-commands.md +++ b/docs/reference/cli-commands.md @@ -191,7 +191,7 @@ error: cannot perform the following tasks: - exec command "sleep" (timed out after 1s: context deadline exceeded) ``` -Read more: [How to use Pebble to manage remote systems](/how-to/manage-remote-systems.md). +Read more: [How to use Pebble to manage remote systems](/how-to/manage-a-remote-system.md). (reference_pebble_health_command)= @@ -410,7 +410,7 @@ may be specified for the last path element. ``` -Read more: [How to use Pebble to manage remote systems](/how-to/manage-remote-systems.md). +Read more: [How to use Pebble to manage remote systems](/how-to/manage-a-remote-system.md). (reference_pebble_mkdir_command)= @@ -437,7 +437,7 @@ The mkdir command creates the specified directory. ``` -Read more: [How to use Pebble to manage remote systems](/how-to/manage-remote-systems.md). +Read more: [How to use Pebble to manage remote systems](/how-to/manage-a-remote-system.md). (reference_pebble_notice_command)= @@ -640,7 +640,7 @@ The pull command retrieves a file from the remote system. ``` -Read more: [How to use Pebble to manage remote systems](/how-to/manage-remote-systems.md). +Read more: [How to use Pebble to manage remote systems](/how-to/manage-a-remote-system.md). (reference_pebble_push_command)= @@ -666,7 +666,7 @@ The push command transfers a file to the remote system. ``` -Read more: [How to use Pebble to manage remote systems](/how-to/manage-remote-systems.md). +Read more: [How to use Pebble to manage remote systems](/how-to/manage-a-remote-system.md). (reference_pebble_remove-identities_command)= @@ -810,7 +810,7 @@ The rm command removes a file or directory. ``` -Read more: [How to use Pebble to manage remote systems](/how-to/manage-remote-systems.md). +Read more: [How to use Pebble to manage remote systems](/how-to/manage-a-remote-system.md). (reference_pebble_run_command)= From fa9fb337882cd847dc53de8e98e988f5d3021e89 Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Tue, 21 Jan 2025 21:10:17 +0800 Subject: [PATCH 16/21] chore: refactor according to review --- docs/how-to/index.md | 2 +- docs/how-to/use-the-pebble-api.md | 15 ++++++--------- 2 files changed, 7 insertions(+), 10 deletions(-) diff --git a/docs/how-to/index.md b/docs/how-to/index.md index fe9ef68aa..09924c293 100644 --- a/docs/how-to/index.md +++ b/docs/how-to/index.md @@ -47,5 +47,5 @@ To integrate Pebble with your automated workflows, you can use the Pebble API. :titlesonly: :maxdepth: 1 -Use Pebble API +Use the Pebble API ``` diff --git a/docs/how-to/use-the-pebble-api.md b/docs/how-to/use-the-pebble-api.md index 1aa7283c3..1cd7e9905 100644 --- a/docs/how-to/use-the-pebble-api.md +++ b/docs/how-to/use-the-pebble-api.md @@ -1,10 +1,10 @@ # How to use the Pebble API to manage services -Pebble provides a command-line interface (CLI) for managing services, similar to systemd. However, Pebble distinguishes itself with its dedicated REST API. +Pebble provides a command-line interface for managing services, similar to systemd. However, Pebble distinguishes itself with its dedicated REST API. -This guide demonstrates how to use the Pebble API to programmatically manage services as part of an automated workflow where we can start, test, and stop interdependent services in a CI pipeline. While shell scripts could be used for this purpose, Pebble API offers a more robust approach with easier error handling and reusable components. This reduces manual work, errors, and ensures consistency across environments. +This guide demonstrates how to use the Pebble API to programmatically manage services as part of an automated workflow that starts, tests, and stops interdependent services in a CI pipeline. While shell scripts could be used for this purpose, the Pebble API offers a more robust approach with easier error handling and reusable components. This reduces manual work, errors, and ensures consistency across environments. -## Using the API +## Use the API ```{include} /reuse/api.md :start-after: Start: Pebble API overview @@ -22,7 +22,7 @@ Suppose we start the Pebble daemon with no default services and an empty layer: 2024-12-30T01:07:10.281Z [pebble] Cannot start default services: no default services ``` -Here is an example in Python: +We can then use a Python client to interact with Pebble: ```python import ops @@ -43,7 +43,7 @@ services: client.add_layer(label="ci", layer=layerYAML, combine=True) # start services -changeID = client.start_services(["svc1"]) # Python client also waits for change to finish +client.start_services(["svc1"]) # Python client also waits for change to finish # get services, the service svc1 should be active services = client.get_services() @@ -53,10 +53,7 @@ for svc in services: # Now we can run some tests against those services. # stop services -changeID = client.stop_services(["svc1"]) # Python client also waits for change to finish - -# wait for the change -client.wait_change(changeID) +client.stop_services(["svc1"]) # Python client also waits for change to finish ``` You can also use Go or curl to achieve the same result. For more information, see {ref}`api_go_client` and {ref}`api_curl`. From 720b793562b6230bc0f27d487c23cea43bad2adc Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Tue, 21 Jan 2025 21:14:40 +0800 Subject: [PATCH 17/21] chore: refactor according to review --- docs/how-to/manage-a-remote-system.md | 28 +++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/docs/how-to/manage-a-remote-system.md b/docs/how-to/manage-a-remote-system.md index 199f0e895..2947bc9f5 100644 --- a/docs/how-to/manage-a-remote-system.md +++ b/docs/how-to/manage-a-remote-system.md @@ -1,16 +1,16 @@ # How to use Pebble to manage a remote system -Managing server clusters remotely takes time and effort. Servers need regular updates, configuration changes, file transfers, and commands run. Directly managing remote servers or using tools like Ansible, which require SSH, increases overhead and security risks. The [XZ Utils backdoor incident](https://en.wikipedia.org/wiki/XZ_Utils_backdoor) highlighted vulnerabilities associated with SSH access. +Managing server clusters remotely takes time and effort. Servers need regular updates and configuration changes, which typically involves transferring files and running commands. Directly managing remote servers or using tools that require SSH access (such as Ansible) increases overhead and security risks. The [XZ Utils backdoor incident](https://en.wikipedia.org/wiki/XZ_Utils_backdoor) in 2024 highlighted vulnerabilities associated with SSH access. -Pebble offers commands and an HTTP API over unix socket for remote system management, avoiding the need for extra open ports. +Pebble offers commands and an HTTP API over Unix socket for remote system management, avoiding the need for extra open ports. -> Note: "remote system" means clients can interact with the Pebble daemon via a unix socket. +> Note: By "remote system", we mean that the Pebble daemon is running in a separate system and there's a Unix socket for clients to interact with the daemon. ## Run commands in a remote system One common task in system administration is updating and installing packages. With Pebble, we can use the `pebble exec` command to achieve this. -For example, if Pebble is running as user with root privileges, we can use this command to update and install packages in a remote system: +For example, if Pebble is running as a user with root privileges, we can use this command to update and install packages in a remote system: ```{terminal} :input: pebble exec apt update @@ -45,7 +45,7 @@ To confirm the package is successfully installed in the remote system, run: || || ``` -For more information on the Pebble `exec` command, see {ref}`reference_pebble_exec_command`. +For more information, see {ref}`reference_pebble_exec_command`. ## Manage files in a remote system @@ -66,26 +66,26 @@ echo "Hello, Pebble!" > /tmp/index.html Create a directory in the remote system: ```bash -pebble mkdir -p /var/www/pebble +pebble mkdir -p /var/www/demo ``` Push our local website file to the remote system: ```bash -pebble push /tmp/index.html /var/www/pebble/index.html +pebble push /tmp/index.html /var/www/demo/index.html ``` Create a virtual host configuration _locally_: ``` -cat < /tmp/pebble +cat < /tmp/demo server { listen 81; listen [::]:81; server_name example.ubuntu.com; - root /var/www/pebble; + root /var/www/demo; index index.html; location / { @@ -98,7 +98,7 @@ EOF Push the file to the remote system: ```bash -pebble push /tmp/pebble /etc/nginx/sites-enabled/pebble +pebble push /tmp/demo /etc/nginx/sites-enabled/demo ``` Activate the newly added virtual host in the remote system by restarting Nginx: @@ -107,7 +107,7 @@ Activate the newly added virtual host in the remote system by restarting Nginx: pebble exec service nginx restart ``` -Finally, we can test the final result: +Finally, we can test the result: ```{terminal} :input: curl localhost:81 @@ -132,9 +132,9 @@ For example, to push a file to the remote system, we can POST to the `/v1/files` curl --unix-socket /path/to/.pebble.socket -XPOST http://_/v1/files -H "Content-Type: multipart/form-data" -F request='{"action": "write", "files": [{"path": "/var/www/pebble/index.html", "make-dirs": true, "permissions": "644"}]}' -F 'files=@/tmp/index.html;type=application/octet-stream;filename=/var/www/pebble/index.html' ``` -We can also use {ref}`api_go_client` and {ref}`api_python_client` for API access. +We can also use the {ref}`api_python_client` and the {ref}`api_go_client` for API access. ## See more -- [How to use the Pebble API to manage services](/how-to/use-the-pebble-api). -- [API spec](/reference/api). +- [How to use the Pebble API to manage services](/how-to/use-the-pebble-api) +- [API](/reference/api) From dc34b6553f51d1d4b4bd04227f6fd98bd231a489 Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Tue, 21 Jan 2025 21:17:11 +0800 Subject: [PATCH 18/21] chore: refactor according to review --- docs/how-to/manage-a-remote-system.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/how-to/manage-a-remote-system.md b/docs/how-to/manage-a-remote-system.md index 2947bc9f5..c963b92cc 100644 --- a/docs/how-to/manage-a-remote-system.md +++ b/docs/how-to/manage-a-remote-system.md @@ -10,6 +10,8 @@ Pebble offers commands and an HTTP API over Unix socket for remote system manage One common task in system administration is updating and installing packages. With Pebble, we can use the `pebble exec` command to achieve this. +> Note: Set the environment variable `PEBBLE_SOCKET` to override the Unix socket used for the API (defaults to `$PEBBLE/.pebble.socket`). + For example, if Pebble is running as a user with root privileges, we can use this command to update and install packages in a remote system: ```{terminal} From 7092a657647178a0f88d7f3ecd5b8b01bf653321 Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Thu, 23 Jan 2025 08:59:56 +0800 Subject: [PATCH 19/21] chore: refactor according to review --- docs/how-to/manage-a-remote-system.md | 23 +++++------------------ 1 file changed, 5 insertions(+), 18 deletions(-) diff --git a/docs/how-to/manage-a-remote-system.md b/docs/how-to/manage-a-remote-system.md index c963b92cc..3f32649a9 100644 --- a/docs/how-to/manage-a-remote-system.md +++ b/docs/how-to/manage-a-remote-system.md @@ -10,7 +10,7 @@ Pebble offers commands and an HTTP API over Unix socket for remote system manage One common task in system administration is updating and installing packages. With Pebble, we can use the `pebble exec` command to achieve this. -> Note: Set the environment variable `PEBBLE_SOCKET` to override the Unix socket used for the API (defaults to `$PEBBLE/.pebble.socket`). +> Note: Set the environment variable `PEBBLE_SOCKET` to override the Unix socket used for the API (defaults to `$PEBBLE/.pebble.socket`, or to `/var/lib/pebble/default/.pebble.socket` if `PEBBLE` is not set). For example, if Pebble is running as a user with root privileges, we can use this command to update and install packages in a remote system: @@ -53,7 +53,7 @@ For more information, see {ref}`reference_pebble_exec_command`. Another common task in a remote system is configuration management. For example, updating configuration files, pushing files to servers, pulling files generated by running services, and so on. -With Pebble, this can be done easily. For example, if we want to install Nginx, create our own website, and serve it on a newly created virtual host, we can first install Nginx: +With Pebble, this can be done using its file commands. For example, if we want to install Nginx, create our own website, and serve it on a newly created virtual host, we can first install Nginx: ```bash pebble exec apt install nginx @@ -120,23 +120,10 @@ For more information about related Pebble commands, see: - {ref}`reference_pebble_ls_command` - {ref}`reference_pebble_mkdir_command` -- {ref}`reference_pebble_rm_command` -- {ref}`reference_pebble_push_command` - {ref}`reference_pebble_pull_command` - -## Use the API - -In the previous sections, we used Pebble commands to run commands and manage files. When automating these tasks, we can use the HTTP API that Pebble provides. - -For example, to push a file to the remote system, we can POST to the `/v1/files` endpoint: - -```bash -curl --unix-socket /path/to/.pebble.socket -XPOST http://_/v1/files -H "Content-Type: multipart/form-data" -F request='{"action": "write", "files": [{"path": "/var/www/pebble/index.html", "make-dirs": true, "permissions": "644"}]}' -F 'files=@/tmp/index.html;type=application/octet-stream;filename=/var/www/pebble/index.html' -``` - -We can also use the {ref}`api_python_client` and the {ref}`api_go_client` for API access. +- {ref}`reference_pebble_push_command` +- {ref}`reference_pebble_rm_command` ## See more -- [How to use the Pebble API to manage services](/how-to/use-the-pebble-api) -- [API](/reference/api) +You can also use the Pebble API to run command and manage files. This is normally done using the Go or Python client libraries. For more information, see [How to use the Pebble API](/how-to/use-the-pebble-api). From a5872759c82c63deb6679ad26f38f40f8c6dbc45 Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Thu, 23 Jan 2025 09:08:15 +0800 Subject: [PATCH 20/21] chore: refactor according to review --- docs/how-to/use-the-pebble-api.md | 13 +++++-------- docs/reference/api.md | 7 +++---- docs/reuse/api.md | 7 ------- 3 files changed, 8 insertions(+), 19 deletions(-) delete mode 100644 docs/reuse/api.md diff --git a/docs/how-to/use-the-pebble-api.md b/docs/how-to/use-the-pebble-api.md index 1cd7e9905..05798cd60 100644 --- a/docs/how-to/use-the-pebble-api.md +++ b/docs/how-to/use-the-pebble-api.md @@ -6,12 +6,9 @@ This guide demonstrates how to use the Pebble API to programmatically manage ser ## Use the API -```{include} /reuse/api.md - :start-after: Start: Pebble API overview - :end-before: End: Pebble API overview -``` +Pebble's API allows remote client interaction with the daemon. It uses HTTP over a Unix socket, with access controlled by user ID. -For reference information about the API, see [](../explanation/api-and-clients) and [](../reference/api). +For an explanation of API access levels, see [API and clients](/explanation/api-and-clients). For the full API reference, see [API](/reference/api). Suppose we start the Pebble daemon with no default services and an empty layer: @@ -40,7 +37,7 @@ services: override: replace command: sleep 100 """ -client.add_layer(label="ci", layer=layerYAML, combine=True) +client.add_layer("label1", layerYAML, combine=True) # start services client.start_services(["svc1"]) # Python client also waits for change to finish @@ -62,5 +59,5 @@ You can also use Go or curl to achieve the same result. For more information, se - [Go client](https://pkg.go.dev/github.com/canonical/pebble/client) - [Python client for Pebble API](https://ops.readthedocs.io/en/latest/reference/pebble.html) -- [API and clients](../explanation/api-and-clients) -- [API](../reference/api) +- [Explanation of API access control](/explanation/api-and-clients) +- [API reference](/reference/api) diff --git a/docs/reference/api.md b/docs/reference/api.md index 81abf47ee..acf7139d9 100644 --- a/docs/reference/api.md +++ b/docs/reference/api.md @@ -1,9 +1,8 @@ # API -```{include} /reuse/api.md - :start-after: Start: Pebble API overview - :end-before: End: Pebble API overview -``` +Pebble exposes an HTTP API that remote clients can use to interact with the daemon. The API has endpoints for starting and stopping services, adding configuration layers to the plan, and so on. + +The API uses HTTP over a Unix socket, with access to the API controlled by user ID. If `pebble run` is started with the `--http
` option, Pebble exposes a limited set of open-access endpoints (see {ref}`api-access-levels`) using the given TCP address. ## Using the API diff --git a/docs/reuse/api.md b/docs/reuse/api.md deleted file mode 100644 index 7923923e5..000000000 --- a/docs/reuse/api.md +++ /dev/null @@ -1,7 +0,0 @@ -Start: Pebble API overview - -Pebble exposes an HTTP API that remote clients can use to interact with the daemon. The API has endpoints for starting and stopping services, adding configuration layers to the plan, and so on. - -The API uses HTTP over a Unix socket, with access to the API controlled by user ID. If `pebble run` is started with the `--http
` option, Pebble exposes a limited set of open-access endpoints (see {ref}`api-access-levels`) using the given TCP address. - -End: Pebble API overview From c7ff16553c76b5f744c3e2f11c2506a251aa228c Mon Sep 17 00:00:00 2001 From: Tiexin Guo Date: Thu, 23 Jan 2025 09:24:53 +0800 Subject: [PATCH 21/21] Update docs/how-to/use-the-pebble-api.md Co-authored-by: Ben Hoyt --- docs/how-to/use-the-pebble-api.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/how-to/use-the-pebble-api.md b/docs/how-to/use-the-pebble-api.md index 05798cd60..e4fb25fab 100644 --- a/docs/how-to/use-the-pebble-api.md +++ b/docs/how-to/use-the-pebble-api.md @@ -6,7 +6,7 @@ This guide demonstrates how to use the Pebble API to programmatically manage ser ## Use the API -Pebble's API allows remote client interaction with the daemon. It uses HTTP over a Unix socket, with access controlled by user ID. +Pebble's API allows clients to interact remotely with the daemon. It uses HTTP over a Unix socket, with access controlled by user ID. For an explanation of API access levels, see [API and clients](/explanation/api-and-clients). For the full API reference, see [API](/reference/api).