LIO Target

From Wikipedia, the free encyclopedia
Jump to: navigation, search
LIO Target
LIO Target - targetcli screenshot.png
LIO Linux SCSI Target
Original author(s) Nicholas Bellinger
Christoph Hellwig
Jerome Martin
Developer(s) Datera, Inc.
Initial release January 14, 2011 (2011-01-14)
Stable release 4.1.0 / June 30, 2013 (2013-06-30)
Preview release 4.2.0-rc5 / June 28, 2012;
21 months ago
 (2012-06-28)
Development status Production
Written in C, Python
Operating system Linux
Available in C, Python
Type Block storage
License GNU General Public License
Website linux-iscsi.org

Linux-IO (LIO) Target is the standard open-source SCSI target included in the Linux kernel.[1][better source needed]

LIO supports common storage fabrics, including FCoE, Fibre Channel, IEEE 1394, iSCSI, iSCSI Extensions for RDMA (iSER), SCSI RDMA Protocol (SRP) and USB. It is included in most Linux distributions. Native support for LIO in QEMU/KVM, libvirt, and OpenStack makes LIO also a storage option for cloud deployments.[2]

LIO was developed by Datera, a Silicon Valley vendor of storage systems and software.

On January 15, 2011, Linus Torvalds merged the LIO SCSI target engine into Linux kernel version 2.6.38,[3][4] which was released on March 14, 2011. Additional fabric modules have been merged into subsequent Linux releases.

A competing generic SCSI target module for Linux is SCST.[5] For the narrower purpose providing a Linux iSCSI target, the older IET and STGT modules also enjoy industry support.[6]

Background[edit]

The SCSI standard provides an extensible semantic abstraction for computer data storage devices, and as such has become a "lingua franca" for data storage systems. The SCSI T10 standards[7] define the commands[8] and protocols of the SCSI command processor (sent in SCSI CDBs), and the electrical and optical interfaces for various implementations.

A SCSI initiator is the endpoint that initiates a SCSI session. A SCSI target is the endpoint that waits for initiator commands and executes the required I/O data transfers. The SCSI target usually exports one or more LUNs for initiators to operate on.

The LIO Linux SCSI Target implements a generic SCSI target that provides remote access to most data storage device types over all prevalent storage fabrics and protocols. LIO neither directly accesses data nor does it directly communicate with applications. LIO provides a highly efficient, fabric-independent and fabric-transparent abstraction for the semantics of numerous data storage device types.

Architecture[edit]

LIO Architecture

LIO implements a modular and extensible architecture around a versatile and highly efficient, parallelized SCSI command processing engine. The SCSI target engine implements the semantics of a SCSI target.

The LIO SCSI target engine is independent of specific fabric modules or backstore types. Thus, LIO supports mixing and matching any number of fabrics and backstores at the same time. The LIO SCSI target engine implements a comprehensive SPC-3/SPC-4[9] feature set with support for high-end features, including SCSI-3/SCSI-4 Persistent Reservations (PRs), SCSI-4 Asymmetric Logical Unit Assignment (ALUA), VMware vSphere APIs for Array Integration (VAAI),[10] T10 DIF, etc.

LIO is configurable via a configfs-based[11] kernel API, and can be managed via a command-line interface and API (targetcli).

SCSI target[edit]

The concept of a SCSI target isn't narrowly restricted to physical devices on a SCSI bus, but instead provides a generalized model for all receivers on a logical SCSI fabric. This includes SCSI sessions across interconnects with no physical SCSI bus at all. Conceptually, the SCSI target provides a generic block storage service or server in this scenario.

Backstores[edit]

Backstores provide the SCSI target with generalized access to data storage devices by importing them via corresponding device drivers. Backstores don't need to be physical SCSI devices.

The most important backstore media types are:

  • Block: The block driver allows using raw Linux block devices as backstores for export via LIO. This includes physical devices, such as HDDs, SSDs, CDs/DVDs, RAM disks, etc., and logical devices, such as software or hardware RAID volumes or LVM volumes.
  • File: The file driver allows using files that can reside in any Linux file system or clustered file system as backstores for export via LIO.
  • Raw: The raw driver allows using unstructured memory as backstores for export via LIO.

As a result, LIO provides a generalized model to export block storage.

Fabric modules[edit]

Fabric modules implement the frontend of the SCSI target by encapsulating and abstracting the properties of the various supported interconnect. The following fabric modules are available.

FCoE[edit]

Combined storage and local area network

The Fibre Channel over Ethernet (FCoE) fabric module allows the transport of Fibre Channel protocol (FCP) traffic across lossless Ethernet networks. The specification, supported by a large number of network and storage vendors, is part of the Technical Committee T11 FC-BB-5 standard.[12]

LIO supports all standard Ethernet NICs.

The FCoE fabric module was contributed by Cisco and Intel, and released with Linux 3.0 on July 21, 2011.[13]

Fibre Channel[edit]

Fibre Channel is a high-speed network technology primarily used for storage networking. It is standardized in the Technical Committee T11[14] of the InterNational Committee for Information Technology Standards (INCITS).

The QLogic Fibre Channel fabric module supports 4- and 8-gigabit speeds with the following HBAs:

  • QLogic 2400 Series (QLx246x), 4GFC
  • QLogic 2500 Series (QLE256x), 8GFC (fully qual'd)

The Fibre Channel fabric module[15] and low-level driver[16] (LLD) were released with Linux 3.5 on July 21, 2012.[17]

With Linux 3.9, the following QLogic HBAs and CNAs are also supported:

  • QLogic 2600 Series (QLE266x), 16GFC, SR-IOV
  • QLogic 8300 Series (QLE834x), 16GFS/10 GbE, PCIe Gen3 SR-IOV
  • QLogic 8100 Series (QLE81xx), 8GFC/10 GbE, PCIe Gen2

This makes LIO the first open source target to support 16-gigabit Fibre Channel.

IEEE 1394[edit]

LIO Firewire Target for Mac OS X

The FireWire SBP-2 fabric module enables Linux to export local storage devices via IEEE 1394, so that other systems can mount them as an ordinary IEEE 1394 storage device.

IEEE 1394 is a serial bus interface standard for high-speed communications and isochronous real-time data transfer. It was developed by Apple as "FireWire" in the late 1980s and early 1990s, and Macintosh computers have supported "FireWire target disk mode" since 1999.[18]

The FireWire SBP-2 fabric module was released with Linux 3.5 on July 21, 2012.[17][19]

iSCSI[edit]

The Internet Small Computer System Interface (iSCSI) fabric module allows the transport of SCSI traffic across standard IP networks.

By carrying SCSI sessions across IP networks, iSCSI is used to facilitate data transfers over intranets and to manage storage over long distances. iSCSI can be used to transmit data over local area networks (LANs), wide area networks (WANs), or the Internet, and can enable location-independent and location-transparent data storage and retrieval.

The LIO iSCSI fabric module also implements a number of advanced iSCSI features that increase performance and resiliency, such as Multiple Connections per Session (MC/S) and Error Recovery Levels 0-2 (ERL=0,1,2).

LIO supports all standard Ethernet NICs.

The iSCSI fabric module was released with Linux 3.1 on October 24, 2011.[20]

iSER[edit]

The iSCSI Extensions for RDMA (iSER) fabric module allows the transport of iSCSI traffic across RDMA networks.

iSER permits data to be transferred directly into and out of remote SCSI computer memory buffers without intermediate data copies (direct data placement or DDP)[21] by using RDMA. RDMA is supported on InfiniBand networks, on "enhanced" Ethernet (DCB) networks via RDMA over Converged Ethernet (RoCE), and on standard Ethernet networks with iWARP enhanced TOE NICs.

The iSER fabric module was developed together by Datera and Mellanox Technologies, and released with Linux 3.10 on June 30, 2013.[22]

SRP[edit]

The SCSI RDMA Protocol (SRP) fabric module allows the transport of SCSI traffic across RDMA (see above) networks. As of 2013, SRP is more widely used than iSER, although it is more limited, as SCSI is only a peer-to-peer protocol, whereas iSCSI is fully routable.

The SRP fabric module supports the following Mellanox host channel adapters (HCAs):

  • Mellanox ConnectX-2 VPI PCIe Gen2 HCAs (x8 lanes), single/dual-port QDR 40 Gbit/s
  • Mellanox ConnectX-3 VPI PCIe Gen3 HCAs (x8 lanes), single/dual-port FDR 56 Gbit/s
  • Mellanox ConnectX-IB PCIe Gen3 HCAs (x16 lanes), single/dual-port FDR 56 Gbit/s

The SRP fabric module was released with Linux 3.3 on March 18, 2012.[23]

In 2012, c't magazine measured almost 5000 MB/s throughput with LIO SRP Target over one Mellanox ConnectX-3 port in 56 Gbit/s FDR mode on a Sandy Bridge PCI Express 3.0 system with four Fusion-IO ioDrive PCI Express flash memory cards.

USB[edit]

The USB Gadget fabric module enables Linux to export local storage devices via the Universal Serial Bus (USB), so that other systems can mount them as an ordinary storage device.

USB was designed in the mid-1990s to standardize the connection of computer peripherals, and has also become common for data storage devices.

The USB Gadget fabric module was released with Linux 3.5 on July 21, 2012.[24]

vHost[edit]

LIO Storage Hypervisor

The LIO vHost fabric module provides KVM guests with near bare-metal local storage performance for shared SCSI LUNs. It implements highly efficient I/O processing based on the Linux 'virtio' mechanism. Currently, only Linux guest VMs are supported.[25]

The vHost fabric module enables building highly efficient storage hypervisors with Linux and KVM.

The vHost fabric module was released with Linux 3.6 on August 13, 2012.[26]

targetcli[edit]

targetcli is a user space single-node management command line interface (CLI) for LIO.[27] It supports all fabric modules and is based on a modular, extensible architecture, with plug-in modules for additional fabric modules or functionality.

targetcli provides a CLI that uses an underlying generic target library through a well-defined API. Thus the CLI can easily be replaced or complemented by a UI with other metaphors, such as a GUI.

targetcli is implemented in Python and consists of three main modules:

  • the underlying rtslib and API.[28]
  • the configshell, which encapsulates the fabric-specific attributes in corresponding 'spec' files.
  • the targetcli shell itself.

Detailed instructions on how to set up LIO targets can be found on the LIO wiki.[27]

Linux distributions[edit]

targetcli and LIO are included in most Linux distributions per default. Here is an overview over the most popular ones, together with the initial inclusion dates:

Distribution Version[a] Release Archive Installation Source git Documentation
Alpine Linux 2.5 2011-11-07 Alpine Linux mirror apk add targetcli-fb targetcli-fb.git How-to
CentOS 6.2 2011-12-20 CentOS mirror su -c 'yum install fcoe-target-utils' targetcli-fb.git Tech Notes
Debian 7.0 ("wheezy") 2013-05-04 Debian pool su -c 'apt-get install targetcli' targetcli.git LIO Wiki
Fedora 16 2011-11-08 Fedora Rawhide su -c 'yum install targetcli' targetcli-fb.git Target Wiki
openSUSE 12.1 2011-11-08 Requires manual installation from Datera targetcli.git repos.
RHEL[b] 6.2 2011-11-16 Fedora Rawhide su -c 'yum install fcoe-target-utils' targetcli-fb.git Tech Notes
Scientific Linux 6.2 2012-02-16 SL Mirror su -c 'yum install fcoe-target-utils' targetcli-fb.git Tech Notes
SLES 11 SP3 MR 2013-12 - su -c 'zypper in targetcli' targetcli.git LIO Wiki
Ubuntu PrecisePangolin v12 2012-04-26 Ubuntu universe sudo apt-get install targetcli targetcli.git LIO Wiki

Controversy[edit]

RedHat storage engineer Andy Grover accused Rising Tide Systems (RTS) of withholding some LIO-related source code for the RTS' proprietary system, RTS OS. Grover says this source code, which includes support for EXTENDED_COPY and COMPARE_AND_WRITE SCSI commands, probably is not licensed and available under GPL; RTS denied this.[30][31][32] However, support for those two commands has been later submitted by RTS for inclusion into version 3.12 of the Linux kernel.[citation needed]

See also[edit]

Notes[edit]

  1. ^ The distribution release where LIO was included first.
  2. ^ RHEL 6 included LIO, but it was the default only for FCoE targets, while STGT was used for iSCSI. In RHEL 7 beta, LIO is the default for FCoE, iSCSI, and for Mellanox InfiniBand iSER/SRP.[29]

References[edit]

  1. ^ "Target". linux-iscsi.org. 2012-10-23. Retrieved 2012-12-25. 
  2. ^ Eric Harney (2012-12-17). "Support LIO as an iSCSI backend". launchpad.net. Retrieved 2013-01-14. 
  3. ^ Linus Torvalds (2011-01-15). "Trivial merge". lkml.org. Retrieved 2011-03-07. 
  4. ^ Thorsten Leemhuis (2011-03-02). "Kernel Log: Coming in 2.6.38 (Part 4) - Storage". Heise Online. 
  5. ^ "A tale of two SCSI targets". Lwn.net. Retrieved 2014-01-20. 
  6. ^ Florian Haas, "Replicate Everything! Highly Available iSCSI Storage with DRBD and Pacemaker", (Linux iSCSI: a Tale of Four Targets section) Linux Journal Issue #217, May 2012
  7. ^ Technical Committee T10. "SCSI Storage Interfaces". t10.org. Retrieved 2012-12-24. 
  8. ^ SCSI Commands Reference Manual (PDF). 100293068, Rev. C. Scotts Valley: Seagate Technology. April 2010. Retrieved 2012-12-25. 
  9. ^ Ralph Weber (2011-01-17). "SCSI Primary Commands - 4 (SPC-4)". t10.org. Retrieved 2011-03-07. 
  10. ^ LIO Linux SCSI Target (2012-12-23). "VAAI". linux-iscsi.org. Retrieved 2012-12-25. 
  11. ^ Jonathan Corbet (2005-08-24). "Configfs - an introduction". lwn.net. Retrieved 2011-03-07. 
  12. ^ "Fibre Channel: Backbone - 5 revision 2.00" (PDF). American National Standard for Information Technology International Committee for Information Technology Standards Technical Group T11. June 4, 2009. Retrieved 2011-05-05. 
  13. ^ Linus Torvalds (2011-04-18). "[SCSI] tcm_fc: Adding FC_FC4 provider (tcm_fc) for FCoE target (TCM - target core) support". torvalds/linux-2.6.git. Retrieved 2012-12-24. 
  14. ^ "T11 Home Page". t11.org. Retrieved 2012-12-25. 
  15. ^ "Linus Torvalds" (2012-05-15). "[SCSI] tcm_qla2xxx: Add >= 24xx series fabric module for target-core". torvalds/linux-2.6.git. Retrieved 2012-05-22. 
  16. ^ "Linus Torvalds" (2012-05-15). "[SCSI] qla2xxx: Add LLD target-mode infrastructure for >= 24xx series". torvalds/linux-2.6.git. Retrieved 2012-05-22. 
  17. ^ a b Thorsten Leemhuis (2012-07-03). "Kernel Log: Coming in 3.5 (Part 2) - Filesystems and storage". Heise Online. Retrieved 2013-01-14. 
  18. ^ "How to use and troubleshoot FireWire target disk mode". apple.com. Retrieved 2012-12-24. 
  19. ^ Linus Torvalds (2012-04-15). "sbp-target: Initial merge of firewire/ieee-1394 target mode support". torvalds/linux-2.6.git. Retrieved 2012-12-24. 
  20. ^ Linus Torvalds (2011-07-27). "iSCSI merge". lkml.org. Retrieved 2012-12-14. 
  21. ^ H. Shah et al. (October 2007). "Direct Data Placement over Reliable Transports". RFC 5041. 
  22. ^ Linus Torvalds (2013-04-30). "Merge branch 'for-next-merge'". lkml.org. Retrieved 2013-07-07. 
  23. ^ Linus Torvalds (2012-01-18). "InfiniBand/SRP merge". lkml.org. Retrieved 2012-12-24. 
  24. ^ "Merge branch 'usb-target-merge'". torvalds/linux-2.6.git. Retrieved 2012-12-24. 
  25. ^ LIO Linux SCSI Target (2012-10-23). "vHost". linux-iscsi.org. Retrieved 2012-12-25. 
  26. ^ Linus Torvalds (2012-08-13). "tcm_vhost: Initial merge for vhost level target fabric driver". lkml.org. 
  27. ^ a b LIO Linux SCSI Target (2012-12-09). "Targetcli". linux-iscsi.org. Retrieved 2012-12-25. 
  28. ^ Jerome Martin (2011-08-03). "Package rtslib". daterainc.com. Retrieved 2012-12-25. 
  29. ^ "Chapter 6. Storage". Access.redhat.com. Retrieved 2014-01-20. 
  30. ^ Nick Farrell (2012-11-15). "Red Hat feuds with Rising Tide Systems - Heresy, and violation of the GPL charges | TechEye". News.techeye.net. Retrieved 2014-01-20. 
  31. ^ "RTS and the GPL". Lwn.net. Retrieved 2014-01-20. 
  32. ^ "Red Hat Developer Demands Competitor's Source Code - Slashdot". Linux.slashdot.org. Retrieved 2014-01-20. 

External links[edit]