Software Testing: Lecture slides and notes

This lecture provides a general introduction to the Software Testing course.

[Slides]

Required Readings

• Textbook (Pezzè and Young): Chapter 1, Software Test and Analysis in a Nutshell SLIDES
• Textbook (Pezzè and Young): Chapter 2, A Framework for Test and Analysis SLIDES
• Whittaker, J.A., What is software testing? And why is it so hard?, IEEE Software, vol.17, no.1, pp.70-79, Jan/Feb 2000. [DOI link]

Suggested Readings

• Butler, R.W.; Finelli, G.B., The infeasibility of quantifying the reliability of life-critical real-time software, IEEE Transactions on Software Engineering, vol.19, no.1, pp.3-12, Jan 1993. [DOI link]
• Bev Littlewood and Lorenzo Strigini. 1993. Validation of ultrahigh dependability for software-based systems. Commun. ACM 36, 11 (November 1993), 69-80. [DOI link]

This lecture introduces the basic concepts of Unit Testing, and how they are supported by JUnit a framework for writing tests. It shows examples how JUnit is supported by Eclipse IDE.

[Slides]

Required Readings

• JUnit Test Infected: Programmers Love Writing Tests — an introduction to JUnit. [URL]
• Using JUnit With Eclipse IDE — an O'Reilly article. [URL]
• Unit Testing in Jazz Using JUnit — an NCSU Open Lab article on using JUnit with Eclipse. [URL]

Suggested Readings

• Michael Olan. 2003. Unit testing: test early, test often. J. Comput. Small Coll. 19, 2 (December 2003), 319-328. [URL]

Activity: to start using JUnit within Eclipe — review and try the example of defining tests for a Triangle class.

Video: this video tutorial shows how to create a new Eclipse project and start writing JUnit tests first.

This lecture discusses Software Testing with respect to different software development proceses and relevant factors (e.g. risk, cost, etc.).

[Slides]

Required Readings

• Textbook (Pezzè and Young): Chapter 4, Test and Analysis Activities within a Software Process SLIDES
• Textbook (Pezzè and Young): Chapter 20, Planning and Monitoring the Process SLIDES

Suggested Readings

• Gregory M. Kapfhammer, Software Testing, In A. Tucker (Ed.), Second Edition, The Computer Science and Engineering Handbook, Chapter 105, CRC Press, 2004. [URL]
• Mary Jean Harrold. 2000. Testing: a roadmap. In Proceedings of the Conference on The Future of Software Engineering (ICSE '00). ACM, New York, NY, USA, 61-72. [DOI link]
• Winokur, M.; Grinman, A.; Yosha, I.; Gallant, R.; Measuring the effectiveness of introducing new methods in the software development process, Euromicro Conference, 1998. Proceedings. 24th , vol.2, no., pp.800-807 vol.2, 25-27 Aug 1998. [DOI link]

Activity: to become familiar with the first Software Testing Practical, identify any relevant questions, think about what kind of information you might need to know, and to look at the code you are about to test.

This lecture introduces functional testing.

[Slides]

Required Readings

• Textbook (Pezzè and Young): Chapter 10, Functional Testing SLIDES
• T. J. Ostrand and M. J. Balcer. 1988. The category-partition method for specifying and generating fuctional tests. Commun. ACM 31, 6 (June 1988), 676-686. [DOI link]

This lecture is concerned with model-based testing, that is, coverage criteria and test cases generated from relevant software models like, e.g., control flow graphs, finate state machines and grammars.

[Slides]

Required Readings

• Textbook (Pezzè and Young): Chapter 5, Finate Models SLIDES
• Textbook (Pezzè and Young): Chapter 11, Combinatorial Testing SLIDES
• Textbook (Pezzè and Young): Chapter 14, Model-Based Testing SLIDES

Suggested Readings

• David Harel, Statecharts: a visual formalism for complex systems, Science of Computer Programming, Volume 8, Issue 3, June 1987, pp. 231-274. [DOI link]
• John Rushby, Using model checking to help discover mode confusions and other automation surprises, Reliability Engineering & System Safety, Volume 75, Issue 2, February 2002, pp. 167-177. [DOI link]

Tutorial 3 is a run-through of category-partition testing using an example from the textbook.

This lecture introduces the basic concepts of structural testing. In particular, it discusses basic structural coverage criteria: Statement Coverage, Branch Coverage, Condition Coverage and Path Coverage.

[Slides]

Required Readings

• Textbook (Pezzè and Young): Chapter 12, Structural Testing SLIDES

Suggested Readings

• Hong Zhu, Patrick A. V. Hall, and John H. R. May. 1997. Software unit test coverage and adequacy. ACM Comput. Surv. 29, 4 (December 1997), 366-427. [DOI link]

This lecture introduces the basic concepts of Data Flow Coverage.

[Slides]

Required Readings

• Textbook (Pezzè and Young): Chapter 6, Dependence and Data Flow Models SLIDES
• Textbook (Pezzè and Young): Chapter 13, Data Flow Testing SLIDES
• P. G. Frankl and E. J. Weyuker. 1988. An Applicable Family of Data Flow Testing Criteria. IEEE Trans. Softw. Eng. 14, 10 (October 1988), 1483-1498. [DOI link]

Suggested Readings

• L. A. Clarke, A. Podgurski, D. J. Richardson and Steven J. Zeil, A Formal Evaluation of Data Flow Path Selection Criteria, IEEE Transactions on Software Engineering, 15 (11), November 1989, pp. 1318-1332. [DOI link]
• Lori A. Clarke, Andy Podgurski, Debra J. Richardson, and Steven J. Zeil. 1985. A comparison of data flow path selection criteria. In Proceedings of the 8th international conference on Software engineering (ICSE '85). IEEE Computer Society Press, Los Alamitos, CA, USA, 244-251. [URL]
• S.C. Ntafos. 1988. A Comparison of Some Structural Testing Strategies. IEEE Trans. Softw. Eng. 14, 6 (June 1988), 868-874. [DOI link]
• Sandra Rapps and Elaine J. Weyuker. 1982. Data flow analysis techniques for test data selection. In Proceedings of the 6th international conference on Software engineering (ICSE '82). IEEE Computer Society Press, Los Alamitos, CA, USA, 272-278. [URL]

Tutorial 4 is concerned with Structural Testing.

This lecture discusses and compares the main coverage criteria.

[Slides]

Required Readings

• Textbook (Pezzè and Young): Chapter 9, Test Case Selection and Adequacy SLIDES

This lecture is dedicated to a question/answer session on the Assignment 1.

Tutorial 5 is concerned with Data Flow Testing.

This lecture introduces mutation testing.

[Slides]

• Textbook (Pezzè and Young): Chapter 16, Fault-Based Testing SLIDES
• Yu-Seung Ma, Jeff Offutt and Yong-Rae Kwon, MuJava: An Automated Class Mutation System. Journal of Software Testing, Verification and Reliability, 15(2):97-133, June 2005. [DOI link]
• A. Jefferson Offutt, Ammei Lee, Gregg Rothermel, Roland H. Untch, and Christian Zapf. 1996. An experimental determination of sufficient mutant operators. ACM Trans. Softw. Eng. Methodol. 5, 2 (April 1996), 99-118. [DOI link]

This lecture introduces integration testing.

[Slides]

• Textbook (Pezzè and Young): Chapter 21, Integration and Component-based Software Testing SLIDES
• Textbook (Pezzè and Young): (beginning of) Chapter 15, Testing Object-Oriented Software SLIDES
• Robyn R. Lutz, Analyzing Software Requirements Errors in Safety-Critical, Embedded Systems, In Proceedings of the IEEE International Symposium on Requirements Engineering. IEEE Computer Society Press, Jan, 1993. [DOI link]
• Jeff Offutt, Aynur Abdurazik and Roger T. Alexander (2000). An Analysis Tool for Coupling-based Integration Testing, The Sixth IEEE International Conference on Engineering of Complex Computer Systems (ICECCS '00), pp 172--178. [DOI link]

Tutorial 6 is concerned with Mutation Testing.

This lecture introduces regression testing.

[Slides]

• Textbook (Pezzè and Young): Chapter 22, System, Acceptance, and Regression Testing SLIDES

This lecture introduces GUI testing.

[Slides]

• Atif M. Memon, Mary Lou Soffa, and Martha E. Pollack. 2001. Coverage criteria for GUI testing. In Proceedings of the 8th European software engineering conference held jointly with 9th ACM SIGSOFT international symposium on Foundations of software engineering (ESEC/FSE-9). ACM, New York, NY, USA, 256-267. [DOI link]
• Joseph Bergin, XP Testing a GUI with FIT, Fitnesse, and Abbot
• The Abbot Java GUI Test Framework
• GUI Testing Checklist

Tutorial 7 is concerned with Integration Testing.

This lecture discusses different software features to be tested at the system level.

[Slides]

This lecture discusses aspects of higher-order testing. This extends/relates to System Testing.

[Slides]

This lecture reviews the Software Testing course.

[Slides]

• Antonia Bertolino (2007). Software Testing Research: Achievements, Challenges, Dreams. In 2007 Future of Software Engineering (FOSE '07). IEEE Computer Society, Washington, DC, USA, 85-103. [DOI link]