Fibre Channel (FC) is a high-speed data transfer protocol providing in-order, lossless[1] delivery of raw block data.[2] Fibre Channel is primarily used to connect computer data storage to servers[3] [4] in storage area networks (SAN) in commercial data centers.
Fibre Channel networks form a switched fabric because the switches in a network operate in unison as one big switch. Fibre Channel typically runs on optical fiber cables within and between data centers, but can also run on copper cabling. Supported data rates include 1, 2, 4, 8, 16, 32, 64, and 128 gigabit per second resulting from improvements in successive technology generations. The industry now notates this as Gigabit Fibre Channel (GFC).
There are various upper-level protocols for Fibre Channel, including two for block storage. Fibre Channel Protocol (FCP) is a protocol that transports SCSI commands over Fibre Channel networks. FICON is a protocol that transports ESCON commands, used by IBM mainframe computers, over Fibre Channel. Fibre Channel can be used to transport data from storage systems that use solid-state flash memory storage medium by transporting NVMe protocol commands.
When the technology was originally devised, it ran over optical fiber cables only and, as such, was called "Fiber Channel". Later, the ability to run over copper cabling was added to the specification. In order to avoid confusion and to create a unique name, the industry decided to change the spelling and use the British English fibre for the name of the standard.[5]
Fibre Channel is standardized in the T11 Technical Committee of the International Committee for Information Technology Standards (INCITS), an American National Standards Institute (ANSI)-accredited standards committee. Fibre Channel started in 1988, with ANSI standard approval in 1994, to merge the benefits of multiple physical layer implementations including SCSI, HIPPI and ESCON.
Fibre Channel was designed as a serial interface to overcome limitations of the SCSI and HIPPI physical-layer parallel-signal copper wire interfaces. Such interfaces face the challenge of, among other things, maintaining signal timing coherence across all the data-signal wires (8, 16 and finally 32 for SCSI, 50 for HIPPI) so that a receiver can determine when all the electrical signal values are "good" (stable and valid for simultaneous reception sampling). This challenge becomes evermore difficult in a mass-manufactured technology as data signal frequencies increase, with part of the technical compensation being ever reducing the supported connecting copper-parallel cable length. See Parallel SCSI. FC was developed with leading-edge multi-mode optical fiber technologies that overcame the speed limitations of the ESCON protocol. By appealing to the large base of SCSI disk drives and leveraging mainframe technologies, Fibre Channel developed economies of scale for advanced technologies and deployments became economical and widespread.
Commercial products were released while the standard was still in draft.[6] By the time the standard was ratified lower speed versions were already growing out of use.[7] Fibre Channel was the first serial storage transport to achieve gigabit speeds[8] where it saw wide adoption, and its success grew with each successive speed. Fibre Channel has doubled in speed every few years since 1996.
In addition to a modern physical layer, Fibre Channel also added support for any number of "upper layer" protocols, including ATM, IP (IPFC) and FICON, with SCSI (FCP) being the predominant usage.
Fibre Channel has seen active development since its inception, with numerous speed improvements on a variety of underlying transport media. The following tables shows the progression of native Fibre Channel speeds:[9]
133 Mbit/s | 0.1328125 | 8b10b | 12.5 | 1993 | |
---|---|---|---|---|---|
266 Mbit/s | 0.265625 | 8b10b | 25 | 1994 | |
533 Mbit/s | 0.53125 | 8b10b | 50 | ||
1GFC (Gen 1) | 1.0625 | 8b10b | 100 | 1997 | |
2GFC (Gen 2) | 2.125 | 8b10b | 200 | 2001 | |
4GFC (Gen 3) | 4.25 | 8b10b | 400 | 2004 | |
8GFC (Gen 4) | 8.5 | 8b10b | 800 | 2008 | |
16GFC (Gen 5) | 14.025 | 64b66b | 1,600 | 2011 | |
32GFC (Gen 6) | 28.05 | 256b257b | 3,200 | 2016[11] | |
64GFC (Gen 7) | 28.9 | 256b257b (FC-FS-5) | 6,400 | 2020 | |
128GFC (Gen 8) | 56.1[12] | 256b257b | 12,800 | Planned 2024 |
10GFC | 10.51875 | 64b66b | 1,200 | 2009 | |
---|---|---|---|---|---|
128GFC (Gen 6) | 28.05 × 4 | 256b257b | 12,800 | 2016 | |
256GFC (Gen 7) | 28.9 × 4 | 256b257b | 25,600 | 2020 |
Two major characteristics of Fibre Channel networks are in-order delivery and lossless delivery of raw block data. Lossless delivery of raw data block is achieved based on a credit mechanism.[1]
There are three major Fibre Channel topologies, describing how a number of ports are connected together. A port in Fibre Channel terminology is any entity that actively communicates over the network, not necessarily a hardware port. This port is usually implemented in a device such as disk storage, a Host Bus Adapter (HBA) network connection on a server or a Fibre Channel switch.
Attribute | Point-to-point | Arbitrated loop | Switched fabric | |
---|---|---|---|---|
Max ports | 2 | 127 | ~16777216 (224) | |
Address size | 8-bit ALPA | 24-bit port ID | ||
Side effect of port failure | Link fails | Loop fails (until port bypassed) | ||
Access to medium | Dedicated | Arbitrated | Dedicated |
Fibre Channel does not follow the OSI model layering, and is split into five layers:
This diagram from FC-FS-4 defines the layers.
Layers FC-0 are defined in Fibre Channel Physical Interfaces (FC-PI-6), the physical layers of Fibre Channel.
Fibre Channel products are available at 1, 2, 4, 8, 10, 16 and 32 and 128 Gbit/s; these protocol flavors are called accordingly 1GFC, 2GFC, 4GFC, 8GFC, 10GFC, 16GFC, 32GFC or 128GFC. The 32GFC standard was approved by the INCITS T11 committee in 2013, and those products became available in 2016. The 1GFC, 2GFC, 4GFC, 8GFC designs all use 8b/10b encoding, while the 10GFC and 16GFC standard uses 64b/66b encoding. Unlike the 10GFC standards, 16GFC provides backward compatibility with 4GFC and 8GFC since it provides exactly twice the throughput of 8GFC or four times that of 4GFC.
Fibre Channel ports come in a variety of logical configurations. The most common types of ports are:
Fibre Channel Loop protocols create multiple types of Loop Ports:
If a port can support loop and non-loop functionality, the port is known as:
Ports have virtual components and physical components and are described as:
The following types of ports are also used in Fibre Channel:
The Fibre Channel physical layer is based on serial connections that use fiber optics to copper between corresponding pluggable modules. The modules may have a single lane, dual lanes or quad lanes that correspond to the SFP, SFP-DD and QSFP form factors. Fibre Channel does not use 8- or 16-lane modules (like CFP8, QSFP-DD, or COBO used in 400GbE) and there are no plans to use these expensive and complex modules.
The small form-factor pluggable transceiver (SFP) module and its enhanced version SFP+, SFP28 and SFP56 are common form factors for Fibre Channel ports. SFP modules support a variety of distances via multi-mode and single-mode optical fiber as shown in the table below. SFP modules use duplex fiber cabling with LC connectors.SFP-DD modules are used for high-density applications that need to double the throughput of an SFP Port. SFP-DD is defined by the SFP-DD MSA and enables breakout to two SFP ports. Two rows of electrical contacts enable doubling the throughput of SFP modules in a similar fashion as QSFP-DD.
The quad small form-factor pluggable (QSFP) module began being used for switch inter-connectivity and was later adopted for use in 4-lane implementations of Gen-6 Fibre Channel supporting 128GFC. QSFP uses either LC connectors for 128GFC-CWDM4 or MPO connectors for 128GFC-SW4 or 128GFC-PSM4. MPO cabling uses 8- or 12-fiber cabling infrastructure that connects to another 128GFC port or may be broken out into four duplex LC connections to 32GFC SFP+ ports. Fibre Channel switches use either SFP or QSFP modules.
Fiber type | Speed (MB/s) | Transmitter[18] | Medium variant | Distance |
---|---|---|---|---|
Single-mode Fiber (SMF) | 12,800 | 1,310 nm longwave light | 128GFC-PSM4 | 0.5m - 0.5 km |
1,270, 1,290, 1,310 and 1,330 nm longwave light | 128GFC-CWDM4 | 0.5 m – 2 km | ||
6,400 | 1,310 nm longwave light | 64GFC-LW | 0.5m - 10 km | |
3,200 | 1,310 nm longwave light | 3200-SM-LC-L | 0.5 m - 10 km | |
1,600 | 1,310 nm longwave light[19] | 1600-SM-LC-L[20] | 0.5 m – 10 km | |
1,490 nm longwave light | 1600-SM-LZ-I | 0.5 m – 2 km | ||
800 | 1,310 nm longwave light[21] | 800-SM-LC-L[22] | 2 m – 10 km | |
800-SM-LC-I | 2 m – 1.4 km | |||
400 | 1,310 nm longwave light[23] | 400-SM-LC-L[24] | 2 m – 10 km | |
400-SM-LC-M | 2 m – 4 km | |||
400-SM-LL-I[25] | 2 m – 2 km | |||
200 | 1,550 nm longwave light[26] | 200-SM-LL-V | 2 m – 50 km | |
1,310 nm longwave light | 200-SM-LC-L | 2 m – 10 km | ||
200-SM-LL-I | 2 m – 2 km | |||
100 | 1,550 nm longwave light | 100-SM-LL-V | 2 m – 50 km | |
1,310 nm longwave light[27] | 100-SM-LL-L[28] 100-SM-LC-L | 2 m – 10 km | ||
100-SM-LL-I | 2 m – 2 km | |||
Multi-mode Fiber (MMF) | 12,800 | 850 nm shortwave light[29] [30] [31] | 128GFC-SW4 | 0 – 100 m |
6,400 | 64GFC-SW | 0 - 100m | ||
3,200 | 3200-SN | 0 – 100 m | ||
1,600 | 1600-M5F-SN-I[32] | 0.5 m – 125 m | ||
1600-M5E-SN-I | 0.5–100 m | |||
1600-M5-SN-S | 0.5–35 m | |||
1600-M6-SN-S[33] | 0.5–15 m | |||
800 | 800-M5F-SN-I | 0.5–190 m | ||
800-M5E-SN-I[34] | 0.5–150 m | |||
800-M5-SN-S | 0.5–50 m | |||
800-M6-SN-S | 0.5–21 m | |||
400 | 400-M5F-SN-I | 0.5–400 m | ||
400-M5E-SN-I | 0.5–380 m | |||
400-M5-SN-I[35] | 0.5–150 m | |||
400-M6-SN-I | 0.5–70 m | |||
200 | 200-M5E-SN-I | 0.5–500 m | ||
200-M5-SN-I | 0.5–300 m | |||
200-M6-SN-I | 0.5–150 m | |||
100 | 100-M5E-SN-I[36] | 0.5–860 m | ||
100-M5-SN-I[37] | 0.5–500 m | |||
100-M6-SN-I | 0.5–300 m | |||
100-M5-SL-I | 2–500 m | |||
100-M6-SL-I[38] | 2–175 m |
Multi-mode fiber | Fiber diameter | FC media designation | |
---|---|---|---|
OM1 | 62.5 μm | M6 | |
OM2 | 50 μm | M5 | |
OM3 | 50 μm | M5E | |
OM4 | 50 μm | M5F | |
OM5 | 50 μm | N/A |
Modern Fibre Channel devices support SFP+ transceiver, mainly with LC (Lucent Connector) fiber connector. Older 1GFC devices used GBIC transceiver, mainly with SC (Subscriber Connector) fiber connector.
The goal of Fibre Channel is to create a storage area network (SAN) to connect servers to storage.
The SAN is a dedicated network that enables multiple servers to access data from one or more storage devices. Enterprise storage uses the SAN to backup to secondary storage devices including disk arrays, tape libraries, and other backup while the storage is still accessible to the server. Servers may access storage from multiple storage devices over the network as well.
SANs are often designed with dual fabrics to increase fault tolerance. Two completely separate fabrics are operational and if the primary fabric fails, then the second fabric becomes the primary.
Fibre Channel switches can be divided into two classes. These classes are not part of the standard, and the classification of every switch is a marketing decision of the manufacturer:
A fabric consisting entirely of one vendors products is considered to be homogeneous. This is often referred to as operating in its "native mode" and allows the vendor to add proprietary features which may not be compliant with the Fibre Channel standard.
If multiple switch vendors are used within the same fabric it is heterogeneous, the switches may only achieve adjacency if all switches are placed into their interoperability modes. This is called the "open fabric" mode as each vendor's switch may have to disable its proprietary features to comply with the Fibre Channel standard.
Some switch manufacturers offer a variety of interoperability modes above and beyond the "native" and "open fabric" states. These "native interoperability" modes allow switches to operate in the native mode of another vendor and still maintain some of the proprietary behaviors of both. However, running in native interoperability mode may still disable some proprietary features and can produce fabrics of questionable stability.
Fibre Channel HBAs, as well as CNAs, are available for all major open systems, computer architectures, and buses, including PCI and SBus. HBAs connect servers to the Fibre Channel network and are part of a class of devices known as translation devices. Some are OS dependent. Each HBA has a unique World Wide Name (WWN), which is similar to an Ethernet MAC address in that it uses an Organizationally Unique Identifier (OUI) assigned by the IEEE. However, WWNs are longer (8 bytes). There are two types of WWNs on an HBA; a World Wide Node Name (WWNN), which can be shared by some or all ports of a device, and a World Wide Port Name (WWPN), which is necessarily unique to each port. Adapters or routers can connect Fibre Channel networks to IP or Ethernet networks.[39]