future runtime dependencies / cockpit-ws rewrite

[JanZerebecki]

Questions at the end.

---

Current state from my limited investigation:

It seems there is a python rewrite of cockpit-ws in the source.

The spec file mentions python in some places where I don't have investigated the reason:
Even if the python bridge and pytest tests are disabled there is a BuildRequires:

cockpit/tools/cockpit.spec

Line 107 in 4975d24

BuildRequires: /usr/bin/python3

There is a suggests from cockpit-ws on it:

cockpit/tools/cockpit.spec

Lines 448 to 449 in 4975d24

	# for cockpit-desktop
	Suggests: python3

cockpit-storaged requires it:

cockpit/tools/cockpit.spec

Line 647 in 4975d24

Requires: %{__python3}

cockpit-packagekit has a recommneds on it:

cockpit/tools/cockpit.spec

Line 698 in 4975d24

Recommends: python3-tracer

I'm not sure if the pytests are only for cockpit-ws.

I have not done a comprehensive list, but a quick overview seems to indicate that the other dependencies are together smaller than the 40MB that python weighs.

---

#17483 (comment)
Mentions that a goal of rewriting cockpit-ws in Python or Rust is to use a webserver library and support http/2.

#17504
Mentions that the cockpit-auth-ssh-key script requires python which adds 40MB and "could eventually be ported to Rust".

I have not found anything in the docs or mailing list or github issues/discussions about the plans regarding it.

OpenSUSE Tumbleweed Micro currently doesn't have python in its image and would prefer to keep it that way for size reasons.
Fedora Coreos, might not want to pull in python either for similar reasons, but I have not checked.

---

(CC: @djoreilly @Nykseli @Lunarequest )
Maybe @martinpitt is the right person to ask.

Darragh did a proof of concept re-implementation: https://github.com/djoreilly/cockpit-auth-ssh-key
Would such an implementation be welcome to replace the existing script?

Are there any other goals of the cockpit-ws rewrite?
Is there any plan for other things to depend on python?
Besides cockpit-auth-ssh-key and rewritten cockpit-ws what else currently depends on python?
Is there any plan to rewrite other things that might increase runtime dependency size?
Would changing direction away from python as a runtime dependency be considered?

[Lunarequest]

@Nykseli and me have been experimenting with rewriting cockpit-ws in rust. Its written in actix and supports HTTP2. would upstream be interested in pursuing this?

[martinpitt]

I did something similar in https://piware.de/post/2022-12-11-rust-webserver-comparison/ for evaluating Rust, and https://piware.de/tmp/mockpit.py for Python. Please don't spend too much time on this yet -- this first needs to be discussed and planned. We almost certainly dont' want a straight one-to-one reimplementation, but clean up the ws architecture first. E.g. there is functionality which shouldn't be in that, but e.g. move to the login page.

[martinpitt]

It seems there is a python rewrite of cockpit-ws in the source.

No, that's not the webserver, but the bridge. Indeed we want to rewrite the web server in Rust or Python as well at some point, but (1) this still needs some more fundamental discussions about its architecture, and (2) we want to finish the bridge project first before we start another major project.

Even if the python bridge and pytest tests are disabled there is a BuildRequires:

Yes, but we have a lot of build tools/scripts which are in Python, and they will even increase in the future. That certainly shouldn't matter much as a build dep? But still, quite a lot of cockpit pages also run python helper scripts to implement functionality in a portable way, so we already have several Python runtime dependencies even before the bridge rewrite.

Would changing direction away from python as a runtime dependency be considered?

No -- Cockpit pages use D-Bus/shell/CLI where possible, but some functionality (e.g. in cockpit-machines) cannot be done with existing OS APIs, and requires something more strong than shell. Rust or other compiled languages don't work, as this needs to be architecture independent. Python is pretty much the only adequate choice here.

[JanZerebecki]

Rust or other compiled languages don't work, as this needs to be architecture independent. Python is pretty much the only adequate choice here.

I wanted to quickly write down how this AFAIK might not be quite true. I don't know whether that in practice means anything for cockpit. But the topic came up again elsewhere and I wanted to make sure to write it down to link to.

While python is often already available, it seems to becomes less so. WASM seems to me to be already more widely available, though that depends on the niche. Also that neglects that we can influence what is available.

AFAIK what cockpit is doing now is similar to salt-ssh: pack up the python code (from where the webserver/cockpit-ws run) to run it on the python runtime available on the target (where the bridge runs). (salt-ssh is in my experience brittle, not sure what causes this.)

Other choices that might be adequate (possibly in combination):

• Ansible style remote calls: Run the python code locally (where the webserver runs), but make it do remote calls through ssh to call on the target. (One might even define cockpit UIs on top of ansible, perhaps allowing export of an ansible script of what the user clicks.)
• Other runtimes: WASM has a runtime that is about 10MB and many languages can target it; lua its runtime is even smaller.
• Ship binary code for all supported arches, that is probably still less in size than the python runtime.
• Require a container runtime. Possibly while using the same python implementation strategy that is currently used, but when connecting to a remote system without python use a python container instead.

[martinpitt]

Ansible style remote calls: Run the python code locally (where the webserver runs), but make it do remote calls through ssh to call on the target.

That's not too far from what's actually happening now. Ansible also copies some python helpers to the remote machine, as there are a lot of things which can't be done with plain shell commands and no extra dependencies. "some python helpers to send D-Bus calls and listen to inotify events" is pretty much exactly what the bridge is.

Other runtimes

Sure, that would be an option in principle, but it's a design choice. We (our team) live in a Fedora/RHEL/Linux server world, where Python is ubiquitous, and we deliberately follow Ansible here.

Ship binary code for all supported arches

No, we won't do that. There are a lot of architectures, Debian alone has over 20.

Require a container runtime

Right, we already talked about that idea with @allisonkarlitskaya , especially as there is Fedora CoreOS. This would be a good bootstrapping fallback if python3 is not installed, but podman is (docker doesn't work). This idea is on the shelf for now, mainly due to lack of time/capacity.

Note that all of this applies to the bridge only, while the discussion topic is about cockpit-ws (where most of the above points don't apply).