change default setup for multipathd

[ibrokethecloud]

change default kernel argument to disable multipath and overwrite multipathd system unit definition to allow multipathd to still be run after boot

Problem:

PR #938 changed kernel arguments to disable multipathd during boot. The original idea was to allow end users to still enable multipathd post boot for 3rd party CSI.

However the change seems to have broken boot in kvm when using ide, scsi and sata buses. The same issue has also been noticed on HP DL360's when using a raid controller.

Solution:

• revert default argument to disable multipathd back to multipath=off
• override multipathd.service file via /etc/systemd/system, where we drop the condition check for multipath=off

Related Issue:
harvester/harvester#7622

Test plan:

• Install harvester using this build to a KVM setup using sata, scsi or ide bus
• post install machine should boot successfully
• ssh to host and run systemctl enable multipathd && systemctl start multipathd
• multipath should start successfully, and can be checked using systemctl status multipathd

On a fresh v1.4.1 install user will not be able to enable and start multipath

[w13915984028]

I did not test the systemd service related part yet, put some questions for discussion:

(1) If user adds systemctl enable multipathd && systemctl start multipathd to configuration file to automate the system installation, will it work?
https://docs.harvesterhci.io/v1.4/install/harvester-configuration#osafter_install_chroot_commands

(2) Should harvester/harvester#7476 be reverted as well?

(3) After user enable multipathd service, will it affect v150->v160 upgrade?

(4) The default multipathd configuration file, we did not add, and system will use the internal defaults

defaults {
  find_multipaths "greedy"
}

blacklist {
  devnode "!^(sd[a-z]|dasd[a-z]|nvme[0-9])"

...

Seems we may add a multipath.conf to harvester os, change above 2 to:

defaults {
  find_multipaths "strict"
}

blacklist {
  devnode "!^(sd[a-z]|dasd[a-z]|vd[a-z]|nvme[0-9])"  // vd* is for kvm
}
...

[ibrokethecloud]

upgrade path PR: harvester/harvester#7678

[ibrokethecloud]

I did not test the systemd service related part yet, put some questions for discussion:

(1) If user adds systemctl enable multipathd && systemctl start multipathd to configuration file to automate the system installation, will it work? https://docs.harvesterhci.io/v1.4/install/harvester-configuration#osafter_install_chroot_commands

this is not needed as the installer already sets up the instructions https://github.com/harvester/harvester-installer/blob/master/pkg/config/cos.go#L883 and these persist across upgrades.

(2) Should harvester/harvester#7476 be reverted as well?

Already changed to reflect what the install path does: harvester/harvester#7678

(3) After user enable multipathd service, will it affect v150->v160 upgrade?

Should not affect the upgrade since the patch will already be done via the install instructions or upgrade path instructions being added via /oem/99_disable_lh_multipathd.yaml

(4) The default multipathd configuration file, we did not add, and system will use the internal defaults

defaults {
  find_multipaths "greedy"
}

blacklist {
  devnode "!^(sd[a-z]|dasd[a-z]|nvme[0-9])"

...

Seems we may add a multipath.conf to harvester os, change above 2 to:

defaults {
  find_multipaths "strict"
}

blacklist {
  devnode "!^(sd[a-z]|dasd[a-z]|vd[a-z]|nvme[0-9])"  // vd* is for kvm
}

This is not really needed. we just need to ensure dracut and kernel do not load the module during boot. I am concerned about blacklisting by device names as when performing external iscsi based installs, the external iscsi disks appear as sda,sdb...

[tserong]

As discussed, this feels like a bit of a weird hack, but I don't have any better ideas...

[w13915984028]

@ibrokethecloud Following is the systemd service on my local pc, it looks with slight differences, did you notice such?

...5.15.0-131-generic #141~20.04.1-Ubuntu SMP Thu Jan 16 18:38:51 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

[Unit]
Description=Device-Mapper Multipath Device Controller
Wants=systemd-udev-trigger.service systemd-udev-settle.service
Before=iscsi.service iscsid.service lvm2-activation-early.service
Before=local-fs-pre.target blk-availability.service
After=multipathd.socket systemd-udev-trigger.service systemd-udev-settle.service
DefaultDependencies=no
Conflicts=shutdown.target
ConditionKernelCommandLine=!nompath
ConditionKernelCommandLine=!multipath=off
ConditionVirtualization=!container

[Service]
Type=notify
NotifyAccess=main
LimitCORE=infinity
ExecStartPre=-/sbin/modprobe -a scsi_dh_alua scsi_dh_emc scsi_dh_rdac dm-multipath
ExecStart=/sbin/multipathd -d -s
ExecReload=/sbin/multipathd reconfigure
TasksMax=infinity

[Install]
WantedBy=sysinit.target
Also=multipathd.socket
Alias=multipath-tools.service

[w13915984028]

LGTM, thanks.

[ibrokethecloud]

@Mergifyio backport v1.5

[mergify]

backport v1.5

✅ Backports have been created

• #990 change default setup for multipathd (backport #988) has been created for branch v1.5 but encountered conflicts