iSCSI conformance testing is testing to determine whether an iSCSI Initiator/Target meets the iSCSI standard.
The Small Computer System Interface (SCSI) is a family of protocols for communicating with I/O devices, especially storage devices. SCSI is based on client-server model. SCSI clients, called "initiators", issue SCSI commands to request services from components, logical units of a server known as a "target". A "SCSI transport" maps the client-server SCSI protocol to a specific interconnect. An Initiator is one endpoint of a SCSI transport and a target is the other endpoint.
Internet Small Computer System Interface (iSCSI) protocol uses TCP/IP as SCSI transport. By transporting SCSI packets over TCP/IP, iSCSI provides an interoperable solution which can take advantage of existing Internet infrastructure, Internet management facilities, and address distance limitations. Unlike traditional Fibre Channel, which requires special-purpose cabling, iSCSI can run on the existing network infrastructure. It has become a popular storage area network (SAN) protocol because of its scalability and economic benefits. The protocol is defined in RFC 3720 and updated in RFC 5048.
With the moving of SCSI transport layer from the traditional reliable bus structure to the far more unreliable TCP/IP network, a fairly reasonable amount of complexity has been introduced into iSCSI:
In one word, although the economic cost of iSCSI may be lower than others, the protocol itself is not that simple.
Since the approval of the protocol, iSCSI has been steadily gaining acceptance among end users and storage vendors. Nowadays, there exist a large number of iSCSI initiator/target implementations on the market, either commercial or free/open source. These implementations vary in functionalities and protocol conformance.
Conformance testing is testing to determine whether a system meets some specified standard. It is often performed by external organizations, sometimes the standards body itself, to give greater guarantees of compliance. Products tested in such a manner are then advertised as being certified by that external organization as complying with the standard. For iSCSI protocol (or other data networking and storage protocols), The University of New Hampshire InterOperability Laboratory (UNH-IOL) is one of the premier neutral, third-party laboratory who offers test suites for iSCSI conformance testing.
As the introduction part in these test suites says:
These tests are designed to determine if an iSCSI product conforms to specifications defined in both IETF RFC 3720 iSCSI (hereafter referred to as the “iSCSI Standard”) as well as updates as contained in IETF RFC 5048 iSCSI Corrections and Clarifications RFC (hereafter referred to as “iSCSI Corrections and Clarifications”). Successful completion of all tests contained in this suite does not guarantee that the tested device will successfully operate with other iSCSI products. However, when combined with satisfactory operation in the IOL’s interoperability test bed, these tests provide a reasonable level of confidence that the Device Under Test (DUT) will function properly in many iSCSI environments.
iSCSI conformance testing mainly contains the following aspects:
Test negotiation key-value pairs; test various flags: CSG, NSG, C bit, T bit...
Test sequence numbers (CmdSN, DataSN, R2TSN); test SCSI Command/Response, Task Management Function Request/Response, Text Request/Response, NOP-In/NOP-Out, SNACK Request...
Test CHAP fields: CHAP_A, CHAP_C, CHAP_I, CHAP_N, CHAP_R...
Test multi-connection support in connection creation, termination, parameter negotiation, sequence number and task management...
Test Initiator/Target's error recovery support
Basically two types of actions could be taken to test whether the DUT meets the iSCSI protocol:
Some common scenarios can be found in iSCSI conformance testing:
To test whether an iSCSI task can be executed correctly (Task Management Function, NOP-In/NOP-Out Ping, multiple connections management...)
For example, this is an error if the W and F bit are both set to 0 in a SCSI Command PDU.
For example, this is an error if a SCSI Command PDU appears in login stage or discovery session.
Digest error can cause the Target/Initiator to send Reject/SNACK PDU.
CmdSN/StatSN/DataSN/R2T each has different scope and different rules.
Text parameters should be in key=value format, followed by one NULL ("0x00") delimiter. Each text parameter also has its own stage and negotiation rules. For example, FirstBurstLength must not exceed MaxBurstLength, SendTargets should appear only in Full Feature Phase, etc.
Too much or too little data is sent to the Target.
Drop an incoming PDU then SNACK for it, drop an outgoing PDU then retry it.
Most of the above scenarios can be emulated by modifying the content of the output PDU. For example, to emulate a data digest error, we just need to change the DataDigest field in the PDU.
An iSCSI conformance testing tool can be viewed as an iSCSI Initiator emulator (to test iSCSI Target) or an iSCSI Target emulator (to test iSCSI Initiator). A good iSCSI conformance testing tool should meet the following requirements:
An executable script is needed for each test case. It should be written according to test case description which defines the PDU sequence and verification points.
The iSCSI Engine core is in charge of sending/receiving iSCSI PDUs to/from DUT (Device Under Test). On one side, it accepts input from test script, sending PDUs or verifying check points; on the other, it should understand iSCSI protocol to a certain level so that necessary automation can be achieved, e.g., the automatic generation of iSCSI PDUs (Login/Text Request during negotiation, Nop-Out ping response,) and some PDU fields (ITT, TTT, CmdSN, data, digest...).
For received PDUs, iSCSI Engine core should do validation check to detect malformed PDU.
Malformed PDUs are often sent to DUT to simulate PDU format error or digest error.
PDU could be dropped to simulate lost PDU during transmission.
Just like Error Recovery, multiple sessions/connections support is an advanced field in iSCSI protocol.
Pass or Fail of a test case should have obvious indication. Detailed log is needed for analysis. It would be better to have a graphical interface to analyze PDU dump files.
A batch system is needed to manage test suites and test cases so that large number of test scripts can run automatically without user intervention.