Additional information for users of multipath-tools from Debian. LVM over DM-Multipath ===================== Debian Bug #1001710 On a setup that involves both, LVM and DM-Multipath, it is important to set the right filter in lvm.conf file to ensure that LVM does not acquire the bare SCSI devices before DM-Multipath can. On such combined setups, the standard recommendation is to pvcreate devices on the enumerated multipathed devices. And also ensure that the bare SCSI devices are filtered out from LVM's list # filter = [ "a|/dev/mapper/|", "r|/block/|", "r|/disk/|", "r|/sd[b-z].*|", "a|.*|" ] In the above example snippet, it is ensured that LVM filters out any SCSI devices of name type sd[b-z].* Note: The enumeration of SCSI devices is not definite. Thus it is important to rely on persistent device names when setting up the filter. The above filter snippet is just an example and should be adapted to individual environments. I see a lot of "io_setup failed" message using the directio checker ==================================================================== Debian Bug #827322 The directio path checker makes use of the asynchronous I/O API (aio) provided by modern Linux systems. Asynchronous I/O allows an application to submit I/O requests asynchronously and be notified later of their completion status. To allow this, we must allocate an asynchronous I/O context (an object of type aio_context_t) and this task is handled by the io_setup system call. A system wide limit on the number of AIO contexts that may be active simultaneously is imposed via the aio-max-nr sysctl parameter. Once this limit has been reached further calls to io_setup will fail with the error number EAGAIN leading to the "io_setup failed" messages seen for e.g. when running "multipath -ll". To avoid this problem the number of available aio contexts should be increased by setting the aio-max-nr parameter. This can be set on a one-time basis via the /proc file system, for e.g.: # echo 131072 > /proc/sys/fs/aio-max-nr Doubles the number of available contexts from the default value of 65536. To make this setting persistent a line may be added to /etc/sysctl.conf: fs.aio-max-nr = 131072 Consult appropriate application and operating system tuning recommendations for guidance on appropriate values for this parameter. What are these weird numbers in /dev/mapper? ============================================ Modern Fibre-Channel connected storage devices are identified by World-Wide-Numbers. If multipath-tools detects one of these devices, it uses that as the default identifier in /dev/mapper. If you want multipath-tools to use easier-to-read names, set user_friendly_names = yes in the 'defaults' section of /etc/multipath.conf. Note: While widely used (including the Debian Installer, which depends on it), user_friendly_names can lead to locking issues with large numbers of LUNs resulting in some maps not having friendly_names [1]. Note: The mpathN <-> wwwn name mapping created by the user_friendly_names feature can change unexpectedly across reboots. A more stable solution for 'friendly' names is to make explicit aliases in multipath.conf, e.g. multipaths { multipath { wwwid 2226f000123deadbf alias diskbox1-lun0 } } This device should be accessible as /dev/mapper/diskbox1-lun0. If using user_friendly_names = yes, please ensure that your /etc/multipath/bindings file gets propagated to the initramfs. For that, please follow the following steps: 1) set "user_friendly_names = yes" in /etc/multipath.conf 2) Run `multipath -F` to flush your device maps. For root on multipath devices, this could still be tricky 3) Run `multipath -v3` to read your /etc/multipath.conf changes. 4) Run `update-initramfs -u -k all` to update your initramfs images. Where did my FC-connected filesystem go? ======================================== If you were previously mounting a device connected to your system by Fibre Channel and then installed multipath-tools, you need to change the way you mount the device. The device must now be accessed using the identifier by which device-mapper knows it. For example if you have in /dev/mapper a file like this: brw-rw---- 1 root disk 254, 8 2009-01-05 14:35 /dev/mapper/36000393000007d3901000000fef00a2d then you can mount the filesystem like this: mount /dev/mapper/36000393000007d3901000000fef00a2d /mnt or this mount /dev/disk/by-id/scsi-36000393000007d3901000000fef00a2d /mnt You should prefer the latter, as this will work whether or not multipath-tools is installed. Note that with multipath-tools installed you cannot use the device's node in /dev, e.g. # mount /dev/sdc1 /mnt mount: /dev/sdc1 already mounted or /mnt busy The device is 'busy' because it is part of a multipath map. See the output of 'multipath -l' to confirm this. Preventing multipath-tools mapping any devices ============================================== You may wish to prevent multipath-tools mapping any of your storage devices. To do this replace the contents of /etc/multipath.conf with: blacklist { devnode "*" } Booting from multipathed devices ================================ To enable booting from multipathed devices install the multipath-tools-boot package. Be sure to update your bootloader configuration (e.g. /boot/grub/menu.lst) to point to the multipathed root device afterwards. By default this will be /dev/disk/by-id/scsi--part If you set user_friendly_names = yes in /etc/multipath.conf, the path to the device will be /dev/mapper/mpath-part If you make changes in multipath.conf, remember to run update-initramfs so the updated version of the configuration gets included in the initramfs. -- Guido Günther Tue, 20 Feb 2008 13:49:18 0100 Development files ================= Before Debian trixie, multipath-tools shipped development files for the various multipath libraries. Given the unclear status regarding a stable ABI of these libraries, they are now considered internal, private libraries of multipath(d). Even though they are installed in "public" library paths, the intention is to prevent other packages from linking against these libraries. -- Chris Hofstaedtler Sat, 03 Jun 2023 15:53:54 +0200 -- [1] https://bugzilla.redhat.com/show_bug.cgi?id=445268