10 QUALITY  17 and 20/11/97

A use case model for Coursework 1

Slide 71

A class model for Coursework 1

Slide 72

10 Quality

\overline{SEOC97  Id : notes11.tex, v1.11997/11/2408 : 35 : 27pxsExppxs  75}

11 TESTING  17 and 20/11/97

Reading, assessment

Various papers are part of the material for this course. Some are required reading. This means they are examinable! Others are merely recommended. You should not be aiming to memorise details from the papers: what I’m after is an understanding of the main issues, questions, problems, solutions to do with quality and things that affect quality, especially human aspects of them. One way of revising them: write your own one page summary, bullet point style, of a paper. Then get together with a few other people and discuss your summaries: work towards a concensus on what’s important.

Slide 73

Required reading on quality: Two short articles: How to Avoid Software Quality, and What is a High Quality Software System.

11 Testing Testing

You should remember (from PM) the process aspects of testing. Ways of testing: black box (specification-based) and white box (structural). Different testing purposes:

• Module (or unit) testing

• Integration testing

• System testing

• Acceptance testing

• Stress testing

• Performance testing and maybe some techniques, like equivalence partitioning?

Slide 74

Why test?

Testing has three main purposes:

• to help you find the bugs

• to convince the customer that there are no bugs

• to provide information that will help with system evolution. The first one is the hard one!

A successful test is one that finds a bug.

Only once you can’t find bugs does it make sense to try to convince the customer that there aren’t any. (Honest!)

Slide 75

Test what?

From the customer’s point of view:

• Testing a system’s capabilities is more important than testing its components.

• Testing old capabilities is more important than testing new capabilities.

• Testing typical situations is more important than testing boundary value cases. Petschenik, H.N., Practical priorities in system testing, IEEE Computer, 2(5) 18-23. Often overlooked: customer documentation testing.

Slide 76

How to test

Need to test old functionality: regression test. Therefore however tedious it is, tests must be:

• repeatable

• documented (both the tests and the results)

• precise

• done on configuration controlled software Ideally, test spec should be written at the same time as the requirements spec: this helps to ensure that the requirements are highly testable. It may seem backwards to consider testability when writing requirements but it’s now standard. E.g. may write requirements in numbered sentences and keep a tally of which test(s) tests which requirement(s). Use cases may help structure tests.

Slide 77

Why is testing so often done badly?

It’s very very boring, especially the documention of the whole process. Automation may help. It’s very expensive, in time and hence money. (May exceed 50% of project time: 30% is about normal.) The alternative may be worse, but the cost may not show up until later... It’s often planned mostly for the end of the project, and gets squeezed by deadline pressure. Customers may pressure developers to deliver on time and untested rather than late and tested. (What should you do?)

Slide 78

Special problems of OO

Two main extra problems at the unit testing level:

1. The unit is the class, which is harder to test than a function. It has state, on which its behaviour may depend – so can’t test methods as though they were independent. Moral: try to avoid designing classes with complex statecharts.

2. Polymorphism. When an object of class A interacts with an object of class B, it may in fact be interacting with an object of any subclass of B – which may not even have been invented when class A is tested! (This problem is to my mind the most convincing justification for work on formal methods...) Moral: remember this problem, and keep inheritance for when its advantages outweigh the disadvantages.

Of course, because the state dependency is in objects, we hope to save effort at integration and system testing, and to win overall.

Slide 79

But that’s not the whole story

Problem: it’s very hard to describe complex tests properly. How to handle bugs that only show up if you do a complex sequence of things?

• write well-modularized and module-tested software so that this is less likely to happen in the first place!

• have someone testing software with the aim of breaking it, however sneakishly.

(E.g. particular problem of testing GUIs: if you can come up with a really good solution to this one, patent it!) That is, precise short documented tests are essential, but not sufficient.

The Black Team.

Slide 80

Reading

Software Quality, Software Process and Software Testing by Dick Hamlet. Optional – it’s revision of this course’s prerequisite, PM, so omit iff you’re confident you remember this stuff – sections 1,2,3,7 are relevant.

Required reading: Improving the Software Testing Process in NASA’s Software Engineering Laboratory by Sharon Waligora and Richard Coon

Slide 81