From a63616b02468e53d388dfde0cd17cbcc48ef7219 Mon Sep 17 00:00:00 2001 From: Yann LE BRIS Date: Thu, 6 Feb 2025 15:44:19 +0100 Subject: [PATCH 01/10] doc(XCP-ng): Adding multipathing guideline The goal is to add configurations and reference guides for iSCSI and Fiber Channel multipathing. Signed-off-by: Yann LE BRIS --- docs/storage/multipathing.md | 260 +++++++++++++++++++++++++++++++++++ docs/storage/storage.md | 4 + 2 files changed, 264 insertions(+) create mode 100644 docs/storage/multipathing.md diff --git a/docs/storage/multipathing.md b/docs/storage/multipathing.md new file mode 100644 index 0000000..bbd393e --- /dev/null +++ b/docs/storage/multipathing.md @@ -0,0 +1,260 @@ +--- +sidebar_position: 1 +--- + +# Multipathing + +How to properly setup storage multipathing with Xen Orchestra and XCP-ng. + +## iSCSI + +### Requirements +* Two different network cards (It can be the same model) +* Dedicated network interfaces without VLAN tagging on XCP-ng host and storage unit +* Two different switches (not stacked) +* Two VLANs **without L3 routing** +* Two IPv4 subnets **without L3 routing** +* Multiple targets per LUN on your storage unit +* iSCSI target ports are operating in portal mode + +:::info +It is recommended to configure the network interfaces of XCP-ng host servers, switch interfaces, and storage array interfaces with Jumbo frames (MTU 9000). + +If your switch cannot handle Jumbo frames, you can leave the default values (MTU 1500). But make sure that all elements are set to MTU 1500. +::: + +:::warning +1. Make sure you do not use bond-type network interfaces on the host and on the storage unit for iSCSI interfaces. +2. Make sure you do not configure network routes on iSCSI interfaces. +::: + +### Target architecture + +#### Configuration example +| Path | Vlan | Subnet | Host PIF address | Storage Controller 1 address | Storage Controller 2 address | +| :---: | :---: | :---: | :---: | :---: | :---: | +| 🔵 | 421 | 10.42.1.0/24 | 10.42.1.11 | 10.42.1.101 | 10.42.1.102 | +| 🟢 | 422 | 10.42.2.0/24 | 10.42.2.11 | 10.42.2.101 | 10.42.2.102 | + +#### Target architecture diagram +```mermaid +--- +config: + look: handDrawn + theme: default +--- +flowchart LR + classDef blueNode fill:#A7C8F0,stroke:#2C6693,color:#000000; + classDef greenNode fill:#A8E6A2,stroke:#3E8E41,color:#000000; + classDef grayNode fill:#D6D6D6,stroke:#8C8C8C,color:#000000; + + subgraph server[XCP-ng host] + direction LR + subgraph Card2[Network card 2] + direction RL + card2pif1[pif4] + card2pif2[pif3] + end + subgraph Card1[Network card 1] + direction RL + card1pif1[pif1] + card1pif2[pif2] + end + end + subgraph Switch 2 + vlan422[vlan422]:::greenNode + end + subgraph Switch 1 + vlan421[vlan421]:::blueNode + end + subgraph storage[Storage unit] + direction LR + subgraph ctrl2[Controller 2] + direction RL + ctrl2-port1[port1] + ctrl2-port2[port2] + end + subgraph ctrl1[Controller 1] + direction RL + ctrl1-port1[port1] + ctrl1-port2[port2] + end + lun1@{ shape: lin-cyl, label: "LUN" } + + end + + card1-pif1-ip([10.42.1.11/24]):::blueNode + card1-pif2-ip([other networks]):::grayNode + card2-pif2-ip([10.42.2.11/24]):::greenNode + card2-pif1-ip([other networks]):::grayNode + ctrl1-port1-ip([10.42.1.101/24]):::blueNode + ctrl1-port2-ip([10.42.2.101/24]):::greenNode + ctrl2-port1-ip([10.42.1.102/24]):::blueNode + ctrl2-port2-ip([10.42.2.102/24]):::greenNode + + card1pif1<-->card1-pif1-ip + card1-pif1-ip<-->vlan421 + vlan421<-->ctrl1-port1-ip + ctrl1-port1-ip<-->ctrl1-port1 + vlan421<-->ctrl2-port1-ip + ctrl2-port1-ip<-->ctrl2-port1 + + card2pif2<-->card2-pif2-ip + card2-pif2-ip<-->vlan422 + vlan422<-->ctrl1-port2-ip + ctrl1-port2-ip<-->ctrl1-port2 + vlan422<-->ctrl2-port2-ip + ctrl2-port2-ip<-->ctrl2-port2 + + card1pif2<-->card1-pif2-ip + card2pif1<-->card2-pif1-ip + ctrl1 <----> lun1 + ctrl2 <----> lun1 + +linkStyle 0 stroke:#4A90E2,stroke-width:2px; +linkStyle 1 stroke:#4A90E2,stroke-width:2px; +linkStyle 2 stroke:#4A90E2,stroke-width:2px; +linkStyle 3 stroke:#4A90E2,stroke-width:2px; +linkStyle 4 stroke:#4A90E2,stroke-width:2px; +linkStyle 5 stroke:#4A90E2,stroke-width:2px; + +linkStyle 6 stroke:#5CB85C,stroke-width:2px; +linkStyle 7 stroke:#5CB85C,stroke-width:2px; +linkStyle 8 stroke:#5CB85C,stroke-width:2px; +linkStyle 9 stroke:#5CB85C,stroke-width:2px; +linkStyle 10 stroke:#5CB85C,stroke-width:2px; +linkStyle 11 stroke:#5CB85C,stroke-width:2px; + +linkStyle 12 stroke:#8C8C8C,stroke-width:2px; +linkStyle 13 stroke:#8C8C8C,stroke-width:2px; +``` + +### Operating procedure + +#### 1. Prepare XCP-ng hosts +1. On one of the host servers, make sure that the multipath.conf configuration includes your storage equipment. + + This can be found in the file `/etc/multipath.xenserver/multipath.conf` +2. If your equipment is not present, ask the manufacturer for the multipath configuration for GNU/Linux + + Add it to the file ```/etc/multipath/conf.d/custom.conf``` + + :::info + In this case, the configuration will be kept after updates. + ::: + +3. If necessary, migrate the VMs active on the XCP-ng host in question to another one +4. Reboot the current XCP-ng host on the affected pool +5. Do the same for all servers in the pool + +#### 2. Prepare the pool +Make sure that multipathing is enabled on the pool. To do this, go to the advanced configuration of the pool + +If this is not the case: +1. Make sure there are **no VMs running** on an iSCSI and/or HBA SR in the pool +2. Activate "Enable multipathing for all hosts" + +#### 3. Configure the SR +Proceed with the iSCSI SR configuration as indicated in the [storage documentation](../../storage/#iscsi). + +## Fiber Channel (HBA) +### Requirements +* Check that the Fiber Channel cards model(s) is supported via the HCL +* Two different Fiber Channel cards (It can be the same model) +* Two different SAN switches +* Multiple targets per LUN on your storage unit +* Zoning performed + +:::warning +Make sure not to mix Fiber Channel speeds. +::: + +### Target architecture +#### Target architecture diagram +```mermaid +--- +config: + look: handDrawn + theme: default +--- +flowchart LR + classDef blueNode fill:#A7C8F0,stroke:#2C6693,color:#000000; + classDef greenNode fill:#A8E6A2,stroke:#3E8E41,color:#000000; + classDef grayNode fill:#D6D6D6,stroke:#8C8C8C,color:#000000; + + subgraph server[XCP-ng host] + direction LR + subgraph Card2[Fiber Channel card 2] + direction RL + card2port1[port1] + end + subgraph Card1[Fiber Channel card 1] + direction RL + card1port1[port1] + end + end + + subgraph storage[Storage unit] + direction LR + subgraph ctrl2[Controller 2] + direction RL + ctrl2-port1[port1] + ctrl2-port2[port2] + end + subgraph ctrl1[Controller 1] + direction RL + ctrl1-port1[port1] + ctrl1-port2[port2] + end + lun1@{ shape: lin-cyl, label: "LUN" } + + end + + vlan421[SAN Switch 1]:::blueNode + vlan422[SAN Switch 2]:::greenNode + + card1port1<---->vlan421 + vlan421<---->ctrl1-port1 + vlan421<---->ctrl2-port1 + + card2port1<---->vlan422 + vlan422<---->ctrl1-port2 + vlan422<---->ctrl2-port2 + + ctrl1 <----> lun1 + ctrl2 <----> lun1 + +linkStyle 0 stroke:#4A90E2,stroke-width:2px; +linkStyle 1 stroke:#4A90E2,stroke-width:2px; +linkStyle 2 stroke:#4A90E2,stroke-width:2px; +linkStyle 3 stroke:#5CB85C,stroke-width:2px; +linkStyle 4 stroke:#5CB85C,stroke-width:2px; +linkStyle 5 stroke:#5CB85C,stroke-width:2px; +``` +### Operating procedure + +#### 1. Prepare XCP-ng hosts +1. On one of the host servers, make sure that the multipath.conf configuration includes your storage equipment. + + This can be found in the file `/etc/multipath.xenserver/multipath.conf` +2. If your equipment is not present, ask the manufacturer for the multipath configuration for GNU/Linux + + Add it to the file ```/etc/multipath/conf.d/custom.conf``` + + :::info + In this case, the configuration will be kept after updates. + ::: + +3. If necessary, migrate the VMs active on the XCP-ng host in question to another one +4. Reboot the current XCP-ng host on the affected pool +5. Do the same for all servers in the pool + +#### 2. Prepare the pool +Make sure that multipathing is enabled on the pool. To do this, go to the advanced configuration of the pool + +If this is not the case: +1. Make sure there are **no VMs running** on an iSCSI and/or HBA SR in the pool +2. Activate "Enable multipathing for all hosts" + +#### 3. Configure the SR +Proceed with the HBA SR configuration as indicated in the [storage documentation](../../storage/#hba). \ No newline at end of file diff --git a/docs/storage/storage.md b/docs/storage/storage.md index 0070a2f..9d9e97d 100644 --- a/docs/storage/storage.md +++ b/docs/storage/storage.md @@ -420,12 +420,16 @@ We also suggest to use a folder on the MooseFS cluster as a root path rather tha Shared, thick-provisioned storage. +For an iSCSI multipath configuration, please follow [these steps](../../storage/multipathing/#iscsi) first. + In Xen Orchestra, go in the "New" menu entry, then Storage, and select iSCSI. Follow instructions from there. ### HBA Shared, thick-provisioned storage. +For an Fiber Channel multipath configuration, please follow [these steps](../../storage/multipathing/#fiber-channel-hba) first. + You can add a Host Bus Adapter (HBA) storage device with `xe`: ``` From 3d016e8b3c6ae3e352a34944f627fe354113ac35 Mon Sep 17 00:00:00 2001 From: Yann LE BRIS Date: Thu, 6 Feb 2025 16:36:01 +0100 Subject: [PATCH 02/10] doc(XCP-ng): Adding multipathing guideline - iSCSI network specification Ensure that no spanning-tree is present Signed-off-by: Yann LE BRIS --- docs/storage/multipathing.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/storage/multipathing.md b/docs/storage/multipathing.md index bbd393e..fb251aa 100644 --- a/docs/storage/multipathing.md +++ b/docs/storage/multipathing.md @@ -11,7 +11,9 @@ How to properly setup storage multipathing with Xen Orchestra and XCP-ng. ### Requirements * Two different network cards (It can be the same model) * Dedicated network interfaces without VLAN tagging on XCP-ng host and storage unit -* Two different switches (not stacked) +* All network interfaces to the hosts and storage unit **must be set to "STP portEdge"** on the network equipment. +* Two different switches (not stacked) **without Spaning-Tree** +* Spanning-tree must be disabled on the switches * Two VLANs **without L3 routing** * Two IPv4 subnets **without L3 routing** * Multiple targets per LUN on your storage unit From 543d9228679e352355d8a04b00227985ae24602d Mon Sep 17 00:00:00 2001 From: Yann LE BRIS Date: Thu, 6 Feb 2025 18:20:23 +0100 Subject: [PATCH 03/10] doc(XCP-ng): Adding multipathing guideline - syntax fix Syntax fix Signed-off-by: Yann LE BRIS --- docs/storage/storage.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/storage/storage.md b/docs/storage/storage.md index 9d9e97d..d2bf872 100644 --- a/docs/storage/storage.md +++ b/docs/storage/storage.md @@ -420,7 +420,7 @@ We also suggest to use a folder on the MooseFS cluster as a root path rather tha Shared, thick-provisioned storage. -For an iSCSI multipath configuration, please follow [these steps](../../storage/multipathing/#iscsi) first. +For a iSCSI multipath configuration, please follow [these steps](../../storage/multipathing/#iscsi) first. In Xen Orchestra, go in the "New" menu entry, then Storage, and select iSCSI. Follow instructions from there. @@ -428,7 +428,7 @@ In Xen Orchestra, go in the "New" menu entry, then Storage, and select iSCSI. Fo Shared, thick-provisioned storage. -For an Fiber Channel multipath configuration, please follow [these steps](../../storage/multipathing/#fiber-channel-hba) first. +For a Fiber Channel multipath configuration, please follow [these steps](../../storage/multipathing/#fiber-channel-hba) first. You can add a Host Bus Adapter (HBA) storage device with `xe`: From c8dd710530794d12ae524e10d0cf74293c13bc5f Mon Sep 17 00:00:00 2001 From: Yann LE BRIS Date: Thu, 6 Feb 2025 18:35:51 +0100 Subject: [PATCH 04/10] doc(XCP-ng): Adding multipathing guideline - adding more details Adding more details Signed-off-by: Yann LE BRIS --- docs/storage/multipathing.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/storage/multipathing.md b/docs/storage/multipathing.md index fb251aa..41dc279 100644 --- a/docs/storage/multipathing.md +++ b/docs/storage/multipathing.md @@ -137,7 +137,7 @@ linkStyle 13 stroke:#8C8C8C,stroke-width:2px; 1. On one of the host servers, make sure that the multipath.conf configuration includes your storage equipment. This can be found in the file `/etc/multipath.xenserver/multipath.conf` -2. If your equipment is not present, ask the manufacturer for the multipath configuration for GNU/Linux +2. If your equipment is not present, ask the manufacturer for the multipath configuration for GNU/Linux otherwise move on to the [next step (Prepare the pool)](../../storage/multipathing/#2-prepare-the-pool) Add it to the file ```/etc/multipath/conf.d/custom.conf``` @@ -239,7 +239,7 @@ linkStyle 5 stroke:#5CB85C,stroke-width:2px; 1. On one of the host servers, make sure that the multipath.conf configuration includes your storage equipment. This can be found in the file `/etc/multipath.xenserver/multipath.conf` -2. If your equipment is not present, ask the manufacturer for the multipath configuration for GNU/Linux +2. If your equipment is not present, ask the manufacturer for the multipath configuration for GNU/Linux otherwise move on to the [next step (Prepare the pool)](../../storage/multipathing/#2-prepare-the-pool-1) Add it to the file ```/etc/multipath/conf.d/custom.conf``` From 4b95ea6415ddea4e366ddcb7d8f403e95e4b37a4 Mon Sep 17 00:00:00 2001 From: Yann LE BRIS Date: Thu, 6 Feb 2025 18:43:08 +0100 Subject: [PATCH 05/10] doc(XCP-ng): Adding multipathing guideline - change subtitle Change subtitle to enforce: that it's a new SR Signed-off-by: Yann LE BRIS --- docs/storage/multipathing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/storage/multipathing.md b/docs/storage/multipathing.md index 41dc279..f9b3e65 100644 --- a/docs/storage/multipathing.md +++ b/docs/storage/multipathing.md @@ -4,7 +4,7 @@ sidebar_position: 1 # Multipathing -How to properly setup storage multipathing with Xen Orchestra and XCP-ng. +How to properly setup new storage multipathing with Xen Orchestra and XCP-ng. ## iSCSI From 849f7b5f209ece29789f300fb69cbe56078c4835 Mon Sep 17 00:00:00 2001 From: Yann LE BRIS Date: Mon, 10 Feb 2025 16:12:41 +0100 Subject: [PATCH 06/10] doc(XCP-ng): Adding multipathing guideline - new sections and fix Reformulations, modifications, and additions following reviews Addition of the following sections: - Maintenance - Troubleshooting - Appendix Signed-off-by: Yann LE BRIS --- docs/storage/multipathing.md | 196 ++++++++++++++++++++++++++++++++--- 1 file changed, 181 insertions(+), 15 deletions(-) diff --git a/docs/storage/multipathing.md b/docs/storage/multipathing.md index f9b3e65..beb7e5b 100644 --- a/docs/storage/multipathing.md +++ b/docs/storage/multipathing.md @@ -4,14 +4,18 @@ sidebar_position: 1 # Multipathing -How to properly setup new storage multipathing with Xen Orchestra and XCP-ng. +How to properly setup new SR with multipathing on Xen Orchestra and XCP-ng. + +:::warning +Do not attempt to enable multipathing on a production pool with existing and active iSCSI and/or HBA and/or FC SRs. +::: ## iSCSI ### Requirements * Two different network cards (It can be the same model) * Dedicated network interfaces without VLAN tagging on XCP-ng host and storage unit -* All network interfaces to the hosts and storage unit **must be set to "STP portEdge"** on the network equipment. +* All network interfaces to the hosts and storage unit **must be set to "STP portEdge"** on the network equipment * Two different switches (not stacked) **without Spaning-Tree** * Spanning-tree must be disabled on the switches * Two VLANs **without L3 routing** @@ -20,9 +24,9 @@ How to properly setup new storage multipathing with Xen Orchestra and XCP-ng. * iSCSI target ports are operating in portal mode :::info -It is recommended to configure the network interfaces of XCP-ng host servers, switch interfaces, and storage array interfaces with Jumbo frames (MTU 9000). +If the storage vendor recommends using Jumbo Frames, you will need to implement them. -If your switch cannot handle Jumbo frames, you can leave the default values (MTU 1500). But make sure that all elements are set to MTU 1500. +Since each architecture is unique, feel free to check with the storage vendor if it’s possible to stay with an MTU of 1500 (e.g., using storage dedicated PIF at 10Gb/s or 25Gb/s). ::: :::warning @@ -141,6 +145,28 @@ linkStyle 13 stroke:#8C8C8C,stroke-width:2px; Add it to the file ```/etc/multipath/conf.d/custom.conf``` + For exemple: + ``` + devices { + + # Configuration for ACME CORP UltraSAN + # This is an example of syntax; do not use it in production. + device { + vendor "ACME" + product "UltraSAN" + path_selector "service-time 0" + path_grouping_policy group_by_prio + prio alua + features "1 queue_if_no_path" + hardware_handler "1 alua" + failback immediate + rr_weight uniform + rr_min_io 100 + no_path_retry 10 + } + } + ``` + :::info In this case, the configuration will be kept after updates. ::: @@ -159,16 +185,16 @@ If this is not the case: #### 3. Configure the SR Proceed with the iSCSI SR configuration as indicated in the [storage documentation](../../storage/#iscsi). -## Fiber Channel (HBA) +## Fibre Channel (HBA) ### Requirements -* Check that the Fiber Channel cards model(s) is supported via the HCL -* Two different Fiber Channel cards (It can be the same model) +* Check that the Fibre Channel cards model(s) is supported via the HCL +* Two different Fibre Channel ports (You can also have multiple cards if you want) * Two different SAN switches * Multiple targets per LUN on your storage unit * Zoning performed :::warning -Make sure not to mix Fiber Channel speeds. +Make sure not to mix Fibre Channel speeds. ::: ### Target architecture @@ -186,13 +212,10 @@ flowchart LR subgraph server[XCP-ng host] direction LR - subgraph Card2[Fiber Channel card 2] - direction RL - card2port1[port1] - end - subgraph Card1[Fiber Channel card 1] + subgraph Card1[Fibre Channel card] direction RL card1port1[port1] + card1port2[port2] end end @@ -219,7 +242,7 @@ flowchart LR vlan421<---->ctrl1-port1 vlan421<---->ctrl2-port1 - card2port1<---->vlan422 + card1port2<---->vlan422 vlan422<---->ctrl1-port2 vlan422<---->ctrl2-port2 @@ -243,6 +266,28 @@ linkStyle 5 stroke:#5CB85C,stroke-width:2px; Add it to the file ```/etc/multipath/conf.d/custom.conf``` + For exemple: + ``` + devices { + + # Configuration for ACME CORP UltraSAN + # This is an example of syntax; do not use it in production. + device { + vendor "ACME" + product "UltraSAN" + path_selector "service-time 0" + path_grouping_policy group_by_prio + prio alua + features "1 queue_if_no_path" + hardware_handler "1 alua" + failback immediate + rr_weight uniform + rr_min_io 100 + no_path_retry 10 + } + } + ``` + :::info In this case, the configuration will be kept after updates. ::: @@ -259,4 +304,125 @@ If this is not the case: 2. Activate "Enable multipathing for all hosts" #### 3. Configure the SR -Proceed with the HBA SR configuration as indicated in the [storage documentation](../../storage/#hba). \ No newline at end of file +Proceed with the HBA SR configuration as indicated in the [storage documentation](../../storage/#hba). + + +## Maintenance operations +### Add a new host to an existing multipathing pool + +:::warning +Do not add the new host server to the pool without completing these steps. +::: + +1. Prepare the host server as specified in this [operating procedure for iSCSI](../../storage/multipathing/#operating-procedure) or this [operating procedure for FC](../../storage/multipathing/#operating-procedure-1) +2. Ensure that the iSCSI PIF configuration is completed if you are using iSCSI +3. Add the new host server to the pool + +## Troubleshooting + +### Verify multipathing +You can use the command ```multipath -ll``` to check if multipathing is active. + +``` +3600a098765432100000123456789abcd dm-3 ACME,UltraSAN +size=500G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw +|-+- policy='service-time 0' prio=50 status=active +| |- 8:0:0:1 sdb 8:16 active ready running +| |- 8:0:1:1 sdd 8:48 active ready running +|-+- policy='service-time 0' prio=10 status=enabled + |- 8:0:2:1 sdf 8:80 active ready running + |- 8:0:3:1 sdh 8:112 active ready running +``` +:::info +In this example, we have four active paths: our multipathing is working correctly. +::: + +### iSCSI +#### Verify iSCSI sessions +You can use the command ```iscsiadm -m session``` to check if iSCSI session is active. + +``` +tcp: [1] 10.42.1.101:3260,1 iqn.2024-02.com.acme:ultrasan.lun01 (non-flash) +tcp: [2] 10.42.1.102:3260,1 iqn.2024-02.com.acme:ultrasan.lun01 (non-flash) +tcp: [3] 10.42.2.101:3260,2 iqn.2024-02.com.acme:ultrasan.lun01 (non-flash) +tcp: [4] 10.42.2.102:3260,2 iqn.2024-02.com.acme:ultrasan.lun01 (non-flash) +``` +:::info +In this example, we have four iSCSI sessions with one LUN. +::: + +#### iSCSI: Verify Jumbo Frame configuration +To check the Jumbo Frames configuration, connect to a host server and try pinging all IP addresses involved in your iSCSI storage (target and initiator) using this command. + +``` +ping -M do -s 8972 +``` +:::warning +If you get an error, usually ```ping: sendmsg: Message too long```, your MTU settings are incorrect, and you need to fix your network configuration. +::: + +:::tip +If your storage vendor allows it, feel free to use 1500 for MTU. +::: + +## Appendix +### Network configuration exemples for jumbo frames + +:::tip +We recommend keeping the standard MTU of 1500. +::: + +If you want to use jumbo frames, here are some configuration examples. We recommend following the vendor documentation, as this information is for reference only. + +#### Arista +Enabling Jumbo Frames on ```Ethernet1``` interface: +``` +conf t + interface Ethernet1 + mtu 9000 + exit +wr mem +``` + +Disabling Jumbo Frames on ```Ethernet1``` interface: +``` +conf t + interface Ethernet1 + mtu 1500 + exit +wr mem +``` +#### Cisco IOS +Enabling Jumbo Frames on ```TenGigabitEthernet1/1``` interface: +``` +conf t + interface TenGigabitEthernet1/1 + mtu 9000 + exit +wr mem +``` + +Disabling Jumbo Frames on ```TenGigabitEthernet1/1``` interface: +``` +conf t + interface Ethernet1 + mtu 1500 + exit +wr mem +``` +#### Juniper +Enabling Jumbo Frames on ```xe-0/0/1``` interface: +``` +configure +set interfaces xe-0/0/1 mtu 9216 +commit +exit +``` + +Disabling Jumbo Frames on ```xe-0/0/1``` interface: +``` +configure +delete interfaces xe-0/0/1 mtu 9216 +commit +exit +``` \ No newline at end of file From 70ff5cd22f186b20a9ba517379ceaa686ae4e3cb Mon Sep 17 00:00:00 2001 From: Yann LE BRIS Date: Mon, 10 Feb 2025 16:25:14 +0100 Subject: [PATCH 07/10] doc(XCP-ng): Adding multipathing guideline - simplification of the iSCSI requirements Simplification of the iSCSI requirements Signed-off-by: Yann LE BRIS --- docs/storage/multipathing.md | 11 ++++------- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/docs/storage/multipathing.md b/docs/storage/multipathing.md index beb7e5b..94af79a 100644 --- a/docs/storage/multipathing.md +++ b/docs/storage/multipathing.md @@ -13,7 +13,7 @@ Do not attempt to enable multipathing on a production pool with existing and act ## iSCSI ### Requirements -* Two different network cards (It can be the same model) +* Four different network interfaces (we recommend using two separate network cards) * Dedicated network interfaces without VLAN tagging on XCP-ng host and storage unit * All network interfaces to the hosts and storage unit **must be set to "STP portEdge"** on the network equipment * Two different switches (not stacked) **without Spaning-Tree** @@ -56,15 +56,12 @@ flowchart LR subgraph server[XCP-ng host] direction LR - subgraph Card2[Network card 2] - direction RL - card2pif1[pif4] - card2pif2[pif3] - end - subgraph Card1[Network card 1] + subgraph Card1[Network cards] direction RL card1pif1[pif1] card1pif2[pif2] + card2pif1[pif4] + card2pif2[pif3] end end subgraph Switch 2 From 246e973d09c9fdbdeda875e47f0f75b86cfb98d3 Mon Sep 17 00:00:00 2001 From: Yann LE BRIS Date: Mon, 10 Feb 2025 19:05:54 +0100 Subject: [PATCH 08/10] doc(XCP-ng): Adding multipathing guideline - syntax fix on Cisco IOS appendix Signed-off-by: Yann LE BRIS --- docs/storage/multipathing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/storage/multipathing.md b/docs/storage/multipathing.md index 94af79a..8f6d54c 100644 --- a/docs/storage/multipathing.md +++ b/docs/storage/multipathing.md @@ -402,7 +402,7 @@ wr mem Disabling Jumbo Frames on ```TenGigabitEthernet1/1``` interface: ``` conf t - interface Ethernet1 + interface TenGigabitEthernet1/1 mtu 1500 exit wr mem From 62e9d371ab89ad3c52a8fa8c9f21fc464038c86d Mon Sep 17 00:00:00 2001 From: Jon Sands Date: Mon, 10 Feb 2025 18:03:26 -0500 Subject: [PATCH 09/10] small spelling fixes Signed-off-by: Jon Sands --- docs/storage/multipathing.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/storage/multipathing.md b/docs/storage/multipathing.md index 8f6d54c..482b63d 100644 --- a/docs/storage/multipathing.md +++ b/docs/storage/multipathing.md @@ -4,7 +4,7 @@ sidebar_position: 1 # Multipathing -How to properly setup new SR with multipathing on Xen Orchestra and XCP-ng. +How to properly setup a new SR with multipathing on Xen Orchestra and XCP-ng. :::warning Do not attempt to enable multipathing on a production pool with existing and active iSCSI and/or HBA and/or FC SRs. @@ -16,7 +16,7 @@ Do not attempt to enable multipathing on a production pool with existing and act * Four different network interfaces (we recommend using two separate network cards) * Dedicated network interfaces without VLAN tagging on XCP-ng host and storage unit * All network interfaces to the hosts and storage unit **must be set to "STP portEdge"** on the network equipment -* Two different switches (not stacked) **without Spaning-Tree** +* Two different switches (not stacked) **without Spanning-Tree** * Spanning-tree must be disabled on the switches * Two VLANs **without L3 routing** * Two IPv4 subnets **without L3 routing** @@ -142,7 +142,7 @@ linkStyle 13 stroke:#8C8C8C,stroke-width:2px; Add it to the file ```/etc/multipath/conf.d/custom.conf``` - For exemple: + For example: ``` devices { @@ -263,7 +263,7 @@ linkStyle 5 stroke:#5CB85C,stroke-width:2px; Add it to the file ```/etc/multipath/conf.d/custom.conf``` - For exemple: + For example: ``` devices { @@ -422,4 +422,4 @@ configure delete interfaces xe-0/0/1 mtu 9216 commit exit -``` \ No newline at end of file +``` From f1f2bd45ae4f5d5c94b80c80fcb770c3732a5583 Mon Sep 17 00:00:00 2001 From: Yann LE BRIS Date: Tue, 11 Feb 2025 12:07:50 +0100 Subject: [PATCH 10/10] doc(XCP-ng): Remove appendix section The appendix section is out of scope. Signed-off-by: Yann LE BRIS --- docs/storage/multipathing.md | 64 +----------------------------------- 1 file changed, 1 insertion(+), 63 deletions(-) diff --git a/docs/storage/multipathing.md b/docs/storage/multipathing.md index 482b63d..e65e1fb 100644 --- a/docs/storage/multipathing.md +++ b/docs/storage/multipathing.md @@ -360,66 +360,4 @@ If you get an error, usually ```ping: sendmsg: Message too long```, your MTU set :::tip If your storage vendor allows it, feel free to use 1500 for MTU. -::: - -## Appendix -### Network configuration exemples for jumbo frames - -:::tip -We recommend keeping the standard MTU of 1500. -::: - -If you want to use jumbo frames, here are some configuration examples. We recommend following the vendor documentation, as this information is for reference only. - -#### Arista -Enabling Jumbo Frames on ```Ethernet1``` interface: -``` -conf t - interface Ethernet1 - mtu 9000 - exit -wr mem -``` - -Disabling Jumbo Frames on ```Ethernet1``` interface: -``` -conf t - interface Ethernet1 - mtu 1500 - exit -wr mem -``` -#### Cisco IOS -Enabling Jumbo Frames on ```TenGigabitEthernet1/1``` interface: -``` -conf t - interface TenGigabitEthernet1/1 - mtu 9000 - exit -wr mem -``` - -Disabling Jumbo Frames on ```TenGigabitEthernet1/1``` interface: -``` -conf t - interface TenGigabitEthernet1/1 - mtu 1500 - exit -wr mem -``` -#### Juniper -Enabling Jumbo Frames on ```xe-0/0/1``` interface: -``` -configure -set interfaces xe-0/0/1 mtu 9216 -commit -exit -``` - -Disabling Jumbo Frames on ```xe-0/0/1``` interface: -``` -configure -delete interfaces xe-0/0/1 mtu 9216 -commit -exit -``` +::: \ No newline at end of file