Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

server/lib: add generic hypervisor enlightenment interface #846

Merged
merged 18 commits into from
Feb 3, 2025

Conversation

gjcolombo
Copy link
Contributor

@gjcolombo gjcolombo commented Jan 30, 2025

Implement some basic infrastructure to support hypervisor enlightenments. The general idea is as follows:

  • propolis-lib defines Enlightenment and EnlightenmentDevice traits that are implemented by components that provide a particular guest-hypervisor interface.
  • Library users instantiate their chosen enlightenment stack and pass the resulting Arc<dyn EnlightenmentDevice> to the lib's machine::Builder. The resulting Machine contains vCPU objects that hold a reference to the selected Enlightenment. This allows them to forward RDMSR/WRMSR exits to the enlightenment stack for (possible) processing.
  • Library users can call Enlightenment::add_cpuid to allow hypervisor interfaces to inject CPUID identifiers into the CpuidSets that are used to configure the VM's vCPUs.
  • Finally, library users can upcast an EnlightenmentDevice to an Arc<dyn Lifecycle> and use that reference to dispatch lifecycle and migration notifications to the active enlightenment stack.

Implement a BhyveGuestInterface enlightenment stack that implements these interfaces but does nothing beyond exposing its vendor string in CPUID leaf 0x4000_0000. This matches the hypervisor-related CPUID and MSR behavior guests would have gotten before this change. More interesting hypervisor interfaces are left for future PRs.

Add an instance spec field to allow propolis-server clients to select an enlightenment stack. If not specified this defaults to the bhyve interface; this provides compatibility when migrating to/from servers built without this code.

Tests: cargo test; PHD w/Alpine, Debian 11, and WS2022 guests; verified in an ad hoc VM that leaf 0x40000000 returns the correct data to a Debian guest.

Related to #328.

The `cpuid` field in an instance spec's board is optional. If it is
`None`, have propolis-server read bhyve's default CPUID values and
insert them into the spec as though the caller had specified them. This
has two benefits:

1. propolis-server can set up CPUID values in the hypervisor leaf range
   without requiring callers to set CPUID values in the standard and
   extended ranges.
2. When a VM migrates, the default CPUID settings from the source will
   transfer to the target, even if the target host's bhyve supplies
   different default CPUID values than the source host's.
@gjcolombo gjcolombo requested review from hawkw and iximeow January 30, 2025 00:41
bin/propolis-server/src/lib/initializer.rs Outdated Show resolved Hide resolved
lib/propolis/src/enlightenment/bhyve.rs Outdated Show resolved Hide resolved
lib/propolis/src/enlightenment/mod.rs Outdated Show resolved Hide resolved
lib/propolis/src/enlightenment/mod.rs Outdated Show resolved Hide resolved
lib/propolis/src/enlightenment/mod.rs Outdated Show resolved Hide resolved
lib/propolis/src/vmm/machine.rs Outdated Show resolved Hide resolved
lib/propolis/src/enlightenment/mod.rs Outdated Show resolved Hide resolved
@gjcolombo gjcolombo requested a review from hawkw January 31, 2025 03:52
lib/propolis/src/enlightenment/mod.rs Outdated Show resolved Hide resolved
lib/propolis/src/enlightenment/bhyve.rs Show resolved Hide resolved
lib/propolis/src/enlightenment/mod.rs Outdated Show resolved Hide resolved
@gjcolombo
Copy link
Contributor Author

@pfmooney - thanks for taking a look! 20f1204 should address all your comments.

lib/propolis/src/enlightenment/mod.rs Show resolved Hide resolved
lib/propolis/src/enlightenment/mod.rs Show resolved Hide resolved
@gjcolombo gjcolombo merged commit c0bb4cd into master Feb 3, 2025
11 checks passed
@gjcolombo gjcolombo deleted the gjcolombo/enlightenment-interface branch February 3, 2025 23:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants