Gray-box testing explained

Gray-box testing (International English spelling: grey-box testing) is a combination of white-box testing and black-box testing. The aim of this testing is to search for the defects, if any, due to improper structure or improper usage of applications.^{[1] [2]}

Overview

A black-box tester is unaware of the internal structure of the application to be tested, while a white-box tester has access to the internal structure of the application. A gray-box tester partially knows the internal structure, which includes access to the documentation of internal data structures as well as the algorithms used.^[3]

Gray-box testers require both high-level and detailed documents describing the application, which they collect in order to define test cases.^[4]

Need for gray-box testing

Gray-box testing is beneficial because it takes the straightforward technique of black-box testing and combines it with the code-targeted systems in white-box testing.

Gray-box testing is based on requirement test case generation because it presents all the conditions before the program is tested by using the assertion method. A requirement specification language is used to make it easy to understand the requirements and verify its correctness.^[5]

Gray-box testing assumptions for object-oriented software

Object-oriented software consists primarily of objects; where objects are single indivisible units having executable code and/or data. Some assumptions are stated below which are needed for the application of use gray-box testing.

• Activation of Methods^[6]
• State Reporting in Class Under Test (CUT).
• Report Testing is inherent in Class Under Test.^[5]

Examples

• Architectural model
• Unified Modeling Language - UML Design Model
• Finite-state machine - State Model.^{[7] [8]}

Techniques

Cem Kaner defines "gray-box testing as involving inputs and outputs, but test design is educated by information about the code or the program operation of a kind that would normally be out of view of the tester".^[9] Gray-box testing techniques are:

• Matrix Testing: states the status report of the project.
• Regression testing: it implies rerunning of the test cases if new changes are made.
• Pattern Testing: verify the good application for its design or architecture and patterns.
• Orthogonal array testing

used as subset of all possible combination.^[10]

Effects

Positive Effects

• Offers combined benefits: As Gray-box testing is combination of white-box and black-box testing, it serves advantages from both the testings.
• Non Intrusive: It is based on functional specification, architectural view whereas not on source code or binaries which makes it invasive too.
• Intelligent Test Authoring: Gray-box tester handles intelligent test scenario, for example, data type handling, communication protocol, exception handling.
• Unbiased Testing: In spite of all above advantages and functionalities, Gray-box testing maintains boundary for testing between tester and developer.^[11]

Negative Effects

• Partial code coverage: In gray-box testing, source code or binaries are missing because of limited access to internal or structure of the applications which results in limited access for code path traversal.
• Defect Identification: In distributed applications, it is difficult to associate defect identification. Still, Gray-box testing is a boon to find how appropriate these systems throw exceptions and how fine are these exceptions handled in distributed systems having web services environment.^{[11] [12]}

Applications

• Gray-box testing is well suited for web applications. Web applications have distributed network or systems; due to absence of source code or binaries it is not possible to use white-box testing. Black-box testing is also not used due to just contract between customer and developer, so it is more efficient to use gray-box testing as significant information is available in Web Services Description Language (WSDL).^[13]
• Gray-box testing is suited for functional or business domain testing. Functional testing is done basically a test of user interactions with may be external systems. Gray-box testing is well-suited for functional testing due to its characteristics; it also helps to confirm that software meets the requirements defined for the software.^{[14] [15] [16] [17]}

Future scope

The distributed nature of Web services allows gray-box testing to detect defects within a service-oriented architecture (SOA). As we know, white-box testing is not suitable for Web services as it deals directly with the internal structures. White-box testing can be used for state art methods; for example, message mutation which generates the automatic tests for large arrays to help exception handling states, flow without source code or binaries. Such a strategy is useful to push gray-box testing nearer to the outcomes of white-box testing.

See also

• Grey box model

Notes and References

1. Web site: Microsoft Research – Emerging Technology, Computer, and Software Research.
2. Web site: Archived copy . 17 October 2011 . https://web.archive.org/web/20120329214234/http://eastwest.inf.brad.ac.uk/document/publication/Doungsa-ard-IWS06%20.pdf . 29 March 2012 . dead .
3. Web site: Gray Box Testing . Software Testing Fundamentals . 4 November 2011 . 19 January 2012 . 16 November 2021 . https://web.archive.org/web/20211116044449/https://softwaretestingfundamentals.com/gray-box-testing/ . dead .
4. Web site: Example of grey box testing with definition . Geekinterview.com . 19 January 2012.
5. Web site: Jake Rogers . Common Questions Regarding Grey-Box Testing . cgsec.co.uk . 8 August 2016 . 8 August 2016 .
6. Web site: Object-Oriented Extensions to Pascal . Pascal-central.com . 19 January 2012 . 5 June 2021 . https://web.archive.org/web/20210605064414/http://pascal-central.com/OOE-stds.html#sect-6.5.8.2 . dead .
7. Book: Patton, Ron. Software Testing. 26 July 2005. Sams. 978-0-672-32798-8. 2. registration.
8. Web site: Archived copy . 17 October 2011 . https://web.archive.org/web/20120403002944/http://www.site.uottawa.ca/~ssome/Cours/SEG3203/gboxtesting.pdf . 3 April 2012 . dead .
9. Book: Nguyen, Hung Q . 2001 . Testing Applications on the Web: Test Planning for Internet-Based Systems . John Wiley & Sons . 9780471437642.
10. Web site: Explore the World of Gray Box Testing . Extremesoftwaretesting.com . 19 January 2012.
11. Web site: SOA Testing Tools for Black, White and Gray Box SOA Testing Techniques . Crosschecknet.com . 19 January 2012 . https://web.archive.org/web/20181001010542/http://www.crosschecknet.com/soa_testing_black_white_gray_box.php . 1 October 2018 . dead .
12. Web site: E33 Gray Box Testing.PDF.
13. Web site: Ramdeo . Anand . Gray Box Testing - Software . Testing Geek . 5 May 2011 . 19 January 2012.
14. Book: Bach, James. Lessons Learned in Software Testing. 31 December 2001 . Wiley Computer Publishing. 978-0-471-08112-8 .
15. Book: Falk, Jack. Testing Computer Software, 2nd Edition. 12 April 1999 . Wiley Computer Publishing. 978-0-471-35846-6 .
16. http://legacy.cleanscape.net/docs_lib/paper_graybox.pdf
17. IBM Systems Journal . 47 . 3 . 457–472 . 10.1147/sj.473.0457 . 6 April 2010 . Li . Z. J. . Business-process-driven gray-box SOA testing . Tan . H. F. . Liu . H. H. . Zhu . J. . Mitsumori . N. M. .