SCST explained
SCST is a GPL licensed SCSI target software stack.The design goals of this software stack are high performance, high reliability, strict conformance to existing SCSI standards, being easy to extend and easy to use. SCST does not only support multiple SCSI protocols (iSCSI, FC, SRP, ...) but also supports multiple local storage interfaces (SCSI pass-through, block I/O and file I/O) and also storage drivers implemented in user-space via the scst_user driver.[1]
In order to reach maximum performance SCST has been implemented as a set of kernel drivers. SCST is often combined with RAID, data deduplication and/or high-availability cluster software to augment its functionality. The SCST software stack is the basis software of many SAN systems.[2] Several world records have been set with SAN systems based on SCST.[3] [4] [5] [6]
SCST competes with LIO Target for the same purpose of providing a generic SCSI target module inside the Linux kernel.[7] For the narrower purpose providing a Linux iSCSI target, the older IET and STGT modules also enjoy industry support.[8]
Between 2013 and 2017 SCST was primarily maintained by SanDisk personnel, after Fusion-io acquired the start-up ID7 in 2013, a driving force behind SCST[9] [10] [11] and after SanDisk acquired Fusion-io in 2014.[12]
Architecture
SCST consists of three groups of modules:
- The SCST core, a protocol-independent engine for processing SCSI commands.
- Target drivers which receive SCSI commands from a SCSI initiator, pass these SCSI to the SCST core and send back replies to the initiator.
- Storage drivers a.k.a. device handlers which interact with the storage medium. The supported local storage interfaces are SCSI, block device, file and scst_user. scst_user is an SCST-specific protocol that allows efficient implementation of storage drivers in user space.
Configuration of all these modules happens via a sysfs interface. Although direct configuration of SCST via its sysfs interface is convenient, the tool called scstadmin allows to control SCST via its sysfs interface and also to save and restore the SCST configuration.
The following design aspects help SCST to reach the high performance this project is known for:
- The number of threads per storage device that processes SCSI commands is configurable.
- All target driver and storage driver functions invoked by the SCST core are asynchronous, at least when the Linux kernel allows this. It is e.g. not yet possible to perform asynchronous or direct I/O from inside the Linux kernel.[13]
- One finite-state machine is associated with each SCSI command. This allows a single command thread to process multiple SCSI commands concurrently.
- Even when using multiple threads per storage device all these threads share the same I/O scheduler context. This sharing improves performance significantly when using e.g. the CFQ I/O scheduler.
- A scatter-gather vector cache. This cache avoids having to allocate memory via the kernel for each SCSI command.
- Lockless access to certain data structures needed to process SCSI commands. This also means that I/O has to be suspended before any of the data structures that are accessed without locking can be modified.[14]
- SCSI commands can be submitted by a target driver to the SCST core from softirq context. This results in a lower number of context switches compared to having to submit SCSI commands from thread context.[15]
History
In the year 2000 a SCSI target framework was developed by the University of New Hampshire InterOperability Laboratory (UNH-IOL). Although the UNH-IOL SCSI target implementation onlysupports the iSCSI protocol, it was designed such that support for other storage protocols would be easy to add.[16]
In 2004 the SCST project was announced, providing Fibre Channel and iSCSI support. The initial iSCSI target in SCST was based on the UNH-IOL iSCSI target implementation.[17] Around that time an alternative iSCSI target software stack was developed by Ardis Technologies.[18]
The iSCSI Enterprise Target software, announced in 2005, is a further evolution of the Ardis stack.[19] [20] Later in 2005 Chelsio announced an SCST iSCSI target driver for the Chelsio T2 Ethernet adapter family.[21]
In 2006 a target driver for LSI/MPT cards was added.[22] Later in this year the scstadmin tool was added. This tool allows to save and restore the SCST configuration from or to a file.[23]
In 2007 the UNH-IOL iSCSI target patch in the SCST project was replaced by a fork of IET.[24]
In 2008 an SRP target driver was donated by Mellanox.[25] Later on this driver was optimized by Bart Van Assche.[26]
In 2009 Marvell donated a target driver for their SAS cards.[27]
Also in 2009 a target driver for Emulex Fibre Channel/FCoE adapter cards was announced by Emulex.[28] [29] [30]
The same year a sysfs interface has been added to SCST next to the already existing procfs interface.
In 2010 an FCoE target driver was donated by Cisco[31] and Bart Van Assche ported the IBM System p SCSI target driver to SCST.[32]
In 2011 implicit ALUA support was added to the SCST core and to the scstadmin configuration tool.[33] The three init scripts (scst, iscsi-scst and qla2x00t) were merged into a single init script, simplifying installation and management of SCST.
In 2013 support for QLogic InfiniBand HCA's was added in the SRP target driver.[34]
In January 2014 Mellanox donated an iSER target driver implementation to the SCST project.[35]
In May 2014 QLogic made a 16 Gbit/s QLogic target driver available and also announced to take over maintainership of the SCST QLogic target driver.[36]
In 2016 support for asynchronous buffered and direct file I/O and also a more powerful latency measurement infrastructure were added.
In 2019 the SCST core was modified such that I/O keeps running at full speed if a LUN or storage device is added or removed.
See also
External links
Notes and References
- SCST, a SCSI Target Framework . Bart . Van Assche . 19 August 2015 . LinuxCon North America 2015 . Seattle.
- Web site: SCST Users . Vladislav . Bolkhovitin . December 20, 2012.
- Web site: New VMmark Cloud Computing World Record set with Cisco UCS, VMware, and Fusion-io . https://web.archive.org/web/20140702212624/http://www.fusionio.com/blog/new-cloud-computing-benchmarks-with-cisco-ucs-vmware-and-fusion-io . dead . July 2, 2014 . Fusion-io . May 10, 2013 . August 12, 2018.
- Web site: HP Achieves Two New #1 VMmark Virtualization Benchmarks with ION Data Accelerator . https://web.archive.org/web/20140616231223/https://www.fusionio.com/blog/hp-achieves-two-new-1-vmmark-virtualization-benchmarks-with-ion-data-accelerator/ . dead . June 16, 2014 . Fusion-io . September 12, 2013 . August 12, 2018.
- Web site: HP achieves world records for 2P blade and 2P rack-mount server virtualization performance on VMmark 2.5.1 benchmark . HP . October 2013.
- Web site: The K2 Tops Two Million IOPS Using a Single All-Flash Storage System . Kaminario . October 4, 2012.
- Web site: A tale of two SCSI targets [LWN.net].
- Florian Haas, "Replicate Everything! Highly Available iSCSI Storage with DRBD and Pacemaker", (Linux iSCSI: a Tale of Four Targets section) Linux Journal Issue #217, May 2012
- Web site: David . Meyer . Fusion-io picks up software-defined storage firm ID7 for SCST chops . Gigaom . 18 March 2013 . 20 July 2014.
- Web site: Kevin . O'Brien . Fusion-io Acquires ID7, Developers of SCST . 18 March 2013 . 20 July 2014.
- Web site: Fusion-io ID7 - Accelerating Shared Storage with Software Defined Solutions . https://web.archive.org/web/20140804035558/http://www.fusionio.com/id7 . dead . August 4, 2014 . Fusion-io . August 12, 2018.
- Web site: Jay . Iyer . SanDisk Completes Acquisition of Fusion-io . SanDisk . 23 July 2014.
- Web site: 2012 Linux Storage, Filesystem, and Memory Management Summit - Day 1 . Jake . Edge . April 3, 2012.
- Web site: More fine-grained locking in SCST . Bart . Van Assche . December 25, 2012.
- Web site: SCST README . Vladislav . Bolkhovitin . Bart . Van Assche . September 26, 2012.
- Palekar . Ashish . Ganapathy . Narendran . Chadda . Anshul . Russell . Robert D. . Design and implementation of a Linux SCSI target for storage area networks . Proceedings of the 5th Annual Linux Showcase & Conference . 2001 . 5 . 11–18 . USENIX Association . 10.1.1.61.2706 .
- Web site: ANNOUNCE Generic SCSI Target Middle Level for Linux (SCST) with target drivers . Vladislav . Bolkhovitin.
- Book: Tomonori . Fujita . Masanori . Ogawara . Proceedings of the international workshop on Storage network architecture and parallel I/Os - SNAPI '04 . Analysis of iSCSI target software . 2004 . 2004 . 25–32 . ACM . 10.1145/1162628.1162632 . 9781450378222 . 14730273 . https://dl.acm.org/citation.cfm?id=1162632.
- Web site: iSCSI Enterprise Target . Tomonori . Fujita . June 1, 2004.
- Web site: ANNOUNCE iSCSI enterprise target software . Tomonori . Fujita . March 1, 2005.
- Web site: Chelsio Delivers Next-Generation iSCSI Product Family Featuring Breakthroughs in Performance and Port Density . Randy . Smith . Chelsio . September 7, 2005.
- Web site: LSI/MPT target driver . Erik . Habbinga . December 13, 2006.
- Web site: Initial scstadmin commit . Mark R. . Buechler . November 14, 2006.
- Web site: Initial iSCSI-SCST commit . Vladislav . Bolkhovitin . August 15, 2007.
- Web site: Initial SRP target driver commit . Vu . Pham . May 17, 2008.
- Web site: PATCH 4/7 scst: Add SRP target driver . Bart . Van Assche . December 20, 2010.
- Web site: Target driver for Marvell 88SE64xx(3G) and 88SE94xx(6G) SAS cards . Andi . Yan . April 14, 2009.
- Web site: ANNOUNCE SCST Target driver support for Emulex lpfc FC/FCoE adapters . Vladislav . Bolkhovitin . July 8, 2009.
- Web site: SDK Brings Enhanced Value and Expanded Market Opportunity to Data Center Storage Solution Developers . https://web.archive.org/web/20091015054651/http://www.emulex.com/resources/press-releases/2009/list/sep-15-2009-emulex-releases-target-mode-driver-software-developer-kit.html . dead . October 15, 2009 . Katherine . Lane . Emulex . September 15, 2009 . August 10, 2018.
- Web site: TargetConnect Software Developer Kit: Accelerate, Simplify and Enhance Target Driver Development . https://web.archive.org/web/20091014060657/http://www.emulex.com/products/software-developer-kits/targetconnect/info/overview.html . dead . October 14, 2009 . Emulex . September 15, 2009 . August 10, 2018.
- Web site: Initial FCoE target driver commit . Joe . Eykholt . February 20, 2010.
- Web site: PATCH 5/7 ibmvstgt: Port from tgt to SCST . Bart . Van Assche . December 20, 2010.
- Web site: PATCH scst: Add implicit ALUA support . Bart . Van Assche . May 11, 2011.
- Web site: PATCH ib_srpt: Add support for QLogic InfiniBand HCA's . Bart . Van Assche . Mar 17, 2013.
- Web site: PATCH 0/9 iSER support for SCST . Yan . Burman . January 21, 2014.
- Web site: QLogic's 16G FC/FCoE HBA support . Duane . Grigsby . May 20, 2014.