Kernel panic in bento/ubuntu-22.04 for libvirt

[AndrewKvalheim]

Version

• bento/ubuntu-22.04 202309.08.0

Environment

• Vagrant 2.4.1
• libvirt 9.9.0
• QEMU 8.1.5
• Linux 6.1.81 x86_64
• NixOS 23.11.5328.51063ed4f234

Scenario

Run an Ubuntu LTS release in Vagrant

Steps to Reproduce

$ vagrant init 'bento/ubuntu-22.04'
$ vagrant up

Expected Result

Provisioned Vagrant machine

Actual Result

Kernel panic at boot:

$ vagrant init 'bento/ubuntu-22.04'
A `Vagrantfile` has been placed in this directory. You are now
ready to `vagrant up` your first virtual environment! Please read
the comments in the Vagrantfile as well as documentation on
`vagrantup.com` for more information on using Vagrant.
$ vagrant up
Bringing machine 'default' up with 'libvirt' provider...
==> default: Checking if box 'bento/ubuntu-22.04' version '202309.08.0' is up to date...
==> default: Creating image (snapshot of base box volume).
==> default: Creating domain with the following settings...
==> default:  -- Name:              bento_default
==> default:  -- Description:       Source: /home/user/example/Vagrantfile
==> default:  -- Domain type:       qemu
==> default:  -- Cpus:              1
==> default:  -- Feature:           acpi
==> default:  -- Feature:           apic
==> default:  -- Feature:           pae
==> default:  -- Clock offset:      utc
==> default:  -- Memory:            512M
==> default:  -- Base box:          bento/ubuntu-22.04
==> default:  -- Storage pool:      default
==> default:  -- Image(vda):        /var/lib/libvirt/images/bento_default.img, virtio, 64G
==> default:  -- Disk driver opts:  cache='default'
==> default:  -- Graphics Type:     vnc
==> default:  -- Video Type:        cirrus
==> default:  -- Video VRAM:        16384
==> default:  -- Video 3D accel:    false
==> default:  -- Keymap:            en-us
==> default:  -- TPM Backend:       passthrough
==> default:  -- INPUT:             type=mouse, bus=ps2
==> default: Creating shared folders metadata...
==> default: Starting domain.
==> default: Domain launching with graphics connection settings...
==> default:  -- Graphics Port:      5900
==> default:  -- Graphics IP:        127.0.0.1
==> default:  -- Graphics Password:  Not defined
==> default:  -- Graphics Websocket: 5700
==> default: Waiting for domain to get an IP address...

[skyscooby]

Did you try increasing the memory allocated to the VM? It looks like 512M is not enough for Ubuntu 22.04?

[Vonng]

also have this problem, with enough resource:

Specs = [

  { "name" => "meta"   , "ip" => "10.10.10.10" ,  "cpu" => "20" ,  "mem" => "40960" ,  "image" => "bento/ubuntu-22.04" },
  { "name" => "node-1" , "ip" => "10.10.10.11" ,  "cpu" => "10" ,  "mem" => "20480" ,  "image" => "bento/ubuntu-22.04" },
  { "name" => "node-2" , "ip" => "10.10.10.12" ,  "cpu" => "10" ,  "mem" => "20480" ,  "image" => "bento/ubuntu-22.04" },
  { "name" => "node-3" , "ip" => "10.10.10.13" ,  "cpu" => "10" ,  "mem" => "20480" ,  "image" => "bento/ubuntu-22.04" },

]

And hang at "Waiting for domain to get an IP addresses" for a long time

==> meta: Creating shared folders metadata...
==> node-2: Creating shared folders metadata...
==> meta: Starting domain.
==> node-2: Starting domain.
==> meta: Domain launching with graphics connection settings...
==> node-2: Domain launching with graphics connection settings...
==> meta:  -- Graphics Port:      5900
==> node-2:  -- Graphics Port:      5901
==> meta:  -- Graphics IP:        127.0.0.1
==> node-2:  -- Graphics IP:        127.0.0.1
==> meta:  -- Graphics Password:  Not defined
==> node-2:  -- Graphics Password:  Not defined
==> meta:  -- Graphics Websocket: 5700
==> node-2:  -- Graphics Websocket: 5701
==> meta: Waiting for domain to get an IP address...
==> node-2: Waiting for domain to get an IP address...
==> node-1: Creating shared folders metadata...
==> node-1: Starting domain.
==> node-1: Domain launching with graphics connection settings...
==> node-1:  -- Graphics Port:      5902
==> node-1:  -- Graphics IP:        127.0.0.1
==> node-1:  -- Graphics Password:  Not defined
==> node-1:  -- Graphics Websocket: 5702
==> node-1: Waiting for domain to get an IP address...
==> node-3: Creating shared folders metadata...
==> node-3: Starting domain.
==> node-3: Domain launching with graphics connection settings...
==> node-3:  -- Graphics Port:      5903
==> node-3:  -- Graphics IP:        127.0.0.1
==> node-3:  -- Graphics Password:  Not defined
==> node-3:  -- Graphics Websocket: 5703
==> node-3: Waiting for domain to get an IP address...

[skyscooby]

also have this problem, with enough resource:

This is probably not the same problem actually.. the libvirt image got busted with the latest release.. reported already as #1604.

You can pin back your box image to 202309.08.0 and it should boot fine with enough resources.

vm.box_version = 202309.08.0

[AndrewKvalheim]

Did you try increasing the memory allocated to the VM? It looks like 512M is not enough for Ubuntu 22.04?

Thanks, looks like it:

Box version	Default behavior	With override to 536 MB
202304.25.0	kernel panic	success
202309.08.0	kernel panic	success
202502.21.0	kernel panic	success

Can the box be configured with an appropriate default?

[skyscooby]

Can the box be configured with an appropriate default?

I wish I knew how.. from what I've read there is a Vagrantfile that is generated with vagrant package step that is included with the box itself and loaded first and that seems to be where one would place a default value for memory...

This project used to be a lot more straightforward with specific .json files for each build but now it's all gone into ruby and it seems like all the packer files are being dynamically generated.. I don't have time to figure it out nor does there seem to be any CI logs published so I guess they are internal to Progress on a private Github instance maybe?.. possibly just built locally by the current maintainer?.. anyhow there isn't even tags in this repository representing the builds since 2023 so I really think this whole project is likely becoming stale sadly.

[Stromweld]

The project has always been a ruby project, but the bento ruby gem is simply a wrapper for packer to loop through all the OS configs and build images for each OS type/version. Packer deprecated the .json config files in favor of their more universal HCL2 language and all the templates in the $BENTO_DIR/packer_templates and in $BENTO_DIR/os_pkrvars folders have been upgraded to hcl2 standards now.

Everything is public here, this repo is using Github Actions CI system. We don't have any internal CI system for this project. But due to limitations the CI system is still a work in progess. In the past all the releases have been built locally on mine or my predecessors mac systems and uploaded. Past maintainers have left the company and I'm the only maintainer now for past 2 years doing it on the side when I have extra time or on my off time.

In 2023 I stopped releasing based on dated versions. The decision to switch to semantic versioning was for the bento gem itself as it better communicates changes. This project should of really been 2 separate repos. 1 for the packer templates and 1 for the bento app itself. The boxes are still published based on dated versioning scheme as that best communicates to users age of the image in use and sets an expected patch level based on date of release.

We strive for out of box support for parallels, vmware-fusion/workstation, virtualbox for arm64 and x86_64. Libvirt, qemu, hyper-v have always been best effort I'm happy to review PR submissions to fix issues you find and can help contribute fixes for. qemu i've found to be a bit more tricky due to the lack of knowledge on my part and it's plethora of configuration options for a system. Especially for EFI based systems it seems to be even harder due to packaging and file location differences in macs and linux distros for the efi var files.

[skyscooby]

@Stromweld it's much appreciated the effort you are able to put in.. having only one maintainer on an open source project also presents significant risks to it's long term existence, I guess that's a budget thing or lack of interest / expertise in the community? But this is what has happened with the "roboxes" stream of images.. the single maintainer seems to have just gone AWOL and stopped responding to the community..

It seems like vagrant ecosystem is stale in general no idea what IBM has in store for it.. maybe it's partially due to Kubernetes eating the DevOps world.. but please know that there are still plenty of use cases for these images and it would be really nice to get it setup to be maintainable long term.

---

To close off this issue we need to increase it because the 512MB default is non functional in some OS's.. I've seen some previous commits in this project increasing the build instances to 1024+.. but this doesn't seem to persist with libvirt images.

Vagrant itself sets the default to 2048 but only for 3 of the providers:
https://github.com/hashicorp/vagrant/blob/e98ee62126c3f610e4970954768a1674db0a5889/Vagrantfile#L13

The libvirt plugin is still using 512MB which is where I believe these boxes are getting their config from.
https://github.com/vagrant-libvirt/vagrant-libvirt/blob/a94ce0d7b6c90129a38435698ca97b364130313d/lib/vagrant-libvirt/config.rb#L972

One way to fix this would be to use the Vagrantfile that is packed into the box itself to set an amount of memory to a value that works for the image. I'm not sure if any of your existing images do this? That's the easiest fix if possible.. otherwise we are gonna have to try and convince the libvirt plugin maintainers to change the default to a higher value...