User talk:3itrekker

Definition
DM-Multipathing(DM-MPIO) provides I/O failover and load-balancing within Linux for block devices. By utilizing device-mapper, multipathd provides the host-side logic to use multiple paths of a redundant network to provide continuous availability and higher bandwidth connectivity between the host server and the block-level device. DM-MPIO handles the rerouting of block I/O to an alternate path in the event of a path failure. DM-MPIO can also balance the I/O load across all of the available paths that are typically Fibre Channel (FC) or iSCSI SAN environments. DM-MPIO is based on device-mapper which provides the basic framework that maps one block device onto another.

Considerations
When utilizing Linux DM-MPIO in a datacenter that has other operating systems and multipath solutions, key components of path management must be considered.
 * Load balancing -- The workload is distributed across the available hardware components. Goal: Reduce I/O completion time, maximize throughput, and optimize resources
 * Path failover and recover -- Utilizes redundant I/O channels to redirect application reads and writes when one or more paths are no longer available.

Components
DM-MPIO in Linux consists of kernel components and user space components.

Kernel – device-mapper – block subsystem that provides layering mechanism for block devices.

User-space – multipath-tools – provides the tools to manage multipathed devices by instructing the device-mapper multipath module what to do.


 * The tools consist of:


 * Multipath: scans the system for multipathed devices, assembles them, updates the device-mapper’s map.


 * Multipathd: daemon that waits for maps events, and then executes multipath and monitors the paths. Marks a path as failed when the path becomes faulty. Depending on the failback policy, it can reactivate the path.


 * Devmap-name: provides a meaningful device name to udev for devmaps


 * Kpartx: maps linear devmaps to device partitions to make multipath maps partionable.


 * Multipath.conf: configuration file for the multipath daemon. Used to overwrite the built-in configuration table of multipathd.


 * devmap-name: provides meaningful device names to udev for device maps (devmaps).

Output of multipath command exports (3600601603b231300e7f479939760dc11) dm-15 XIOTECH,Mag3d [size=30G][features=1 queue_if_no_path][hwhandler=1 xiotech] \_ round-robin 0 [prio=0][active] \_ 1:0:1:0 sdj 8:144  [active][ready] \_ 2:0:0:0 sdr 65:16  [active][ready] \_ round-robin 0 [prio=0][enabled] \_ 1:0:0:0 sdb 8:16   [active][ready] \_ 2:0:1:0 sdz 65:144 [active][ready]
 * 1) multipath -ll

Annotated output of the multpath command exports (3600601603b231300e7f479939760dc11) dm-15 XIOTECH Mag3d -- -     ---  --   |               |                         |       |        |--> Product |              |                         |       |---> Vendor |              |                         |---> sysfs name |              |-> WWID of the device |-- --> User defined Alias name

[size=30G][features=1 queue_if_no_path][hwhandler=1 rdac] - ---       |                 |                        |> Hardware Handler, if any |                |-> Features supported |---> Size of the DM device

Path Group 1: \_ round-robin 0 [prio=0][active] -- - --  -- |    |              |      |> Path group state |   |              |---> Path group priority |   |--> Path selector and repeat count |---> Path group level First path on Path Group 1: \_ 1:0:1:0 sdj 8:144 [active][ready] ---   --  -      |      |     |        |      |-> Physical Path state |     |     |        |> DM Path state |     |     |-> Major, minor numbers |     |---> Linux device name |--> SCSI information: host, channel, scsi_id and lun Second path on Path Group 1: \_ 2:0:0:0 sdr 65:16 [active][ready] Path Group 2: \_ round-robin 0 [prio=0][enabled] \_ 1:0:0:0 sdb 8:16   [active][ghost] \_ 2:0:1:0 sdz 65:144 [active][ghost]

Configuration file
The configuration file /etc/multipath.conf allows many of the DM-MPIO features to be user configurable. The multipath and the kernel daemon multipathd uses information found in this file. The file is only consulted during the configuration of the multipath devices. Changes must be made prior to running the multipath command. Changes to the file afterwards will require multipath to be executed again.

The multipath.conf has 5 sections : A fully annotated multipath.conffile.
 * 1) System level defaults ("defaults"): User can override system level defaults.
 * 2) Blacklisted devices ("blacklist"): User specifies the list of devices that is not be to under the control of DM-MPIO.
 * 3) Blacklist execptions ("blacklist_exceptions"): Specific devices to be treated as multipath devices even if listed in the blacklist.
 * 4) Storage controller specific settings ("devices"): User specified configuration settings will be applied to devices with specified "Vendor" and "Product" information.
 * 5) Device specific settings ("multipaths"): Fine tun the configuration settings for individual LUNs.

defaults { user_friendly_names    yes } blacklist { devnode "^(ram|raw|loop|fd|md|sr|scd|st)[0-9]*" devnode "sda$" } } multipaths { multipath { wwid	     360060480000290102841533030334534 alias	     u01 }	multipath { wwid          360060480000290102841533030334535 alias         u02 } devices { device { vendor			"XIOTECH" product			"Mag3d" path_grouping_policy	multibus } }
 * 1) This is a template multipath-tools configuration file
 * 2) Uncomment the lines relevent to your environment
 * 1) Uncomment the lines relevent to your environment
 * 1) 	udev_dir		/dev
 * 2) 	polling_interval 	10
 * 3) 	selector		"round-robin 0"
 * 4) 	path_grouping_policy	multibus
 * 5) 	getuid_callout		"/sbin/scsi_id -g -u -s /block/%n"
 * 6) 	prio_callout		/bin/true
 * 7) 	path_checker		readsector0
 * 8) 	rr_min_io		100
 * 9) 	rr_weight		priorities
 * 10) 	failback		immediate
 * 11) 	no_path_retry		fail
 * 1) 		getuid_callout         "/sbin/scsi_id -g -u -s /block/%n"
 * 2) 		path_checker		readsector0
 * 3) 		path_selector		"round-robin 0"
 * 4) 		hardware_handler	"0"
 * 5) 		failback		15
 * 6) 		rr_weight		priorities
 * 7) 		no_path_retry		queue
 * 8) 		rr_min_io		100
 * 9) 		product_blacklist	LUN_Z

Terminology

 * HBA: Host Bus Adapter provide the physical interface between the input/output (I/O) host bus of Fibre Channel devices and the underlying Fibre Channel network.
 * Path: Connection from the server through the HBA to a specific LUN.
 * DM Path States: The device mapper's view of the path condition. Only two conditions are possible:
 * Active: The last I/O operation sent through this path successfully completed. Analogous to ready path state.
 * Failed: The last I/O operation sent through this path did not successfully complete. Analogous to faulty path state.
 * Failover: When a path is determined to be in a failed state, a path that is in ready state will be made active.
 * Failback: When a failed path is determined to be active again, multipathd may choose to failback to the the path as determined by the failback policy.
 * Failback Policy: Three options as set in the multipath.conf configuration file.
 * Immediate: Immediately failback to the highest priority path.
 * Number of seconds: Wait for a specified number of seconds to allow the I/O to stabilize, then failback to the highest priority path.
 * Manual: The failed path is not monitored, requires user intervention to failback.
 * Active/Active: In a system that has two storage controllers, each controller can process I/O.
 * Active/Passive: In a system that has two storage controllers, only one controller at a time is able to process I/O, the other (passive) is in a standby mode.
 * LUN: SCSI Logical_Unit_Number
 * WWID: Worldwide Identifier is an identifier for the multipath device that is guaranteed to be globally unique and unchanging.

External Links for further reading and configuration examples
Michael, T., Kabir, R., Giles, J. & Hull, J. (2006.) Configuring Linux to Enable Multipath I/O. Retrieved from http://www.dell.com/downloads/global/power/ps3q06-20060189-Michael.pdf

Goggin, E., Kergon, A., Varoqui, C., & Olien, D. (2005) Proceedings of the Linux Symposium – Linux Multipathing. Retrieved from http://www.linuxinsight.com/files/ols2005/goggin-reprint.pdf

Red Hat Documentation. (n.d.) Red Hat Enterprise Linux 5, DM Multipath. Retrieved from http://linux.web.cern.ch/linux/scientific5/docs/rhel/DM_Multipath/index.html

Varoqui, C. (2010.) The Linux multipath implementation. Retrieved from http://christophe.varoqui.free.fr/refbook.html

---