13 PROCEDURAL STRUCTURES 27/11/97

13 Procedural structures

So far we’ve discussed what quality is – what we’re trying to achieve – and what effects the social structures of the organisation may have on it. What about the more tangible aspects of the process the organisation follows to complete a project? And what are the interactions between process and society?

We’ve considered, briefly, how requirements analysis, analysis, design, construction and testing are done, mostly from a technical viewpoint. What about the management activities that surround this technical process?

Planning/estimation/bidding

Aim: convince the customer that the work is in safe hands with you. Must also be useful to you, so shouldn’t be fiction!

Slide 87

Here’s one possible outline of project plan, from OU M355 Unit 13. The company’s QMS (Quality Management System) defines what they actually do.

1. The Project Introduction

• project plan outline

• any assumptions made when plan was produced • project schedule

2. The Project Description

• project products

• acceptance criteria for products

• customer involvement in development tasks

3. The Phase Plan

• project phases and associated tasks

• project milestones and associated dates

4. The Organization Plan

• staff and their responsibilities 5. The Support Requirements Specification

• software development facilities required

• hardware development facilities required

• support required for the software development team

6. The Development Process Specification • development techniques and standards to be used

7. The Change Control Plan • method for handling changes during the project

8. The Documentation Plan • methods of document production

• document format

9. Appendices • test plan

• quality plan

• example development notations Notice that the project plan is a customer-readable document. This has several implic-

ations.

• It must be comprehensible by the customer. (Hence, for example, the possible inclusion of example development notations in an appendix. The developers are presumably familiar with the notations they use, but the customer will probably not be.)

• It must show the developers in a good light: any problems you highlight will come with suggested solutions, unless you can argue that none exist. It will not mention risks such as “we don’t have the expertise to do this”!

• It must not include business-sensitive information you don’t want the customer to know, e.g. cost estimate, as distinct from price estimate – you don’t tell the customer your profit margin...

Different criteria might apply if you were planning an in-house project – but increasingly IT divisions bid against outside suppliers, so the gap is not as clear as it was.

Scheduling

In PM you’ve seen such things as:

• a phase plan

• a work breakdown structure

• a PERT network

• what a critical path is and some information on how one makes time estimates for a project.

Slide 88

Project tracking

As the project goes on, estimates have to be revised in the light of progress, unforeseen problems etc. Typically a large project will replan once a week. Depending on the circumstances slippages may mean

• reallocating resource (but see The Mythical Man Month)

• cutting functionality

• asking the customer for more money

• losing profit Tell them the most important metric is what proportion of the project staff’s time is spent explaining to the customer why the project is late Anonymous software engineer. But why do projects always slip? And how do we track the project?

Slide 89

Why do projects always slip?

Required reading: The Rational Planning of (Software) Projects by Mark C. Paulk. This paper discusses the effects of three features of human nature:

• people tend to be risk-averse when there is a potential of loss

• people are unduly optimistic in their plans and forecasts

• people prefer to use intuitive judgement rather than (quantitative) models

It goes on to discuss how a framework like the CMM can help.

Slide 90

Use of metrics

You’ve encountered metrics before. But how do we decide which metrics are useful? A metric may be a:

• predictor – use one measurement (e.g. number of functions) to predict another (e.g. cost of project). The thing used is a predictor metric.

• result – if a measurement was predicted by a predictor metric, measure its actual value at the end of a project (or phase). Feed the discrepancy back into process improvement and future estimation.

Slide 91

Useful metrics must be...

• Measurable – e.g. not someone’s opinion of how complex something was

• Independent – i.e. not something that can be altered without altering the quality of the software

• Accountable – i.e. managers should be able to justify both the use of this metric and the value given to it.

• Precise – i.e. the minimal accuracy of the number must be known.

• MEANINGFUL!! – there must be some reason to believe that numbers for the metric have something to do with software quality!

Slide 92

13.0.1 Independent

You may think that there are no such things as truly independent metrics and, indeed, they are rare. The reason that you seldom encounter independent metrics has to do with Heisenberg’s uncertainty principle (slightly revised). Measuring any project parameter and attaching evident significance to it will affect the usefulness of that parameter. The keyword here is evident. If the project members know that you can count the number of smiles on Monday mornings and use that as an indication of project morale, then when they want you to believe that morale is high, they will all start smiling on Monday mornings. If on the other hand you keep to yourself the metrics you are using and the significance you attact to them, their usefulness (if any) will not degrade over time.

DeMarco, T. (1982), Controlling Software Projects, Yourdan Press, New York

13.0.2 Accountable

To justify the metric, will have to store:

• exactly what was measured

• by whom

• when

• what values were for other projects

It is essential, and can be hard, to measure the things you want to know. However software metric expert Tom Gilb propounded what Tom DeMarco dubs Gilb’s law:

14 QUALITY ASSURANCE  27/11/97

Anything you need to quantify can be measured in some way which is superior to not measuring it at all

Is there a contradiction here?

14 Quality Assurance

Recall that quality, in software, means the conjunction of any good properties a piece of software or its delivery should have in order to please the customer! For example:

• it should meet functional and non-functional requirements (stated or not!)

• it should be delivered on time and to budget

• it should continue to be useful in the face of changing situations and requirements, to the extent that this is necessary. For example, if it is still in use in the year 2000 it should not have the year 2000 problem...

Quality assurance

is the process of

(a) convincing oneself and one’s customer that any product delivered will have high quality.

(b) providing a basis for continuous improvement

This normally means examining and monitoring the process of developing software to check that the process is likely to produce high quality software.

Various standards exist (ISO9001, TickIt, CMM level n). In some contexts adherence to particular standards or particular certification is required, either legally or contractually.

Slide 93

Quality assurance is the process of convincing oneself and one’s customer that any product delivered will have high quality – or at least a guaranteed acceptable level of quality. This normally means examining and monitoring the process of developing software to check that the process is likely to produce high quality software. One way of increasing confidence that this is so is to acquire a quality certification such as BS5750/ISO9001 or TickIt. In some contexts adherence to particular standards or particular certification is required, either legally or contractually.

The most important aspect of quality assurance, however, is that by documenting and measuring what is done and how well it works, it provides a basis for continuous improvement of the process of producing software, with the aim of achieving a continuously improving level of quality in the product. If you’re aware of what was done differently in a particularly successful project, and you have a way of rolling out changes to a whole organisation, you can spread successful innovations throughout an organisation. Unfortunately, many quality systems actually work against this aim, by making it difficult and bureaucratic to change processes, even where people agree that the change is for the better.

International and US companies often classify the process they follow by claiming to be at “CMM level n” for n between 1 and 5. This refers to the Capability Maturity Model developed at CMU’s SEI. The stages of CMM are described in today’s required reading, namely The Rational Planning of Software Projects (p6). In brief:

CMM

The idea is to:

• documenting the way work is performed

• measuring its performance

• using data for controlling the performance

• planning based on historical performance

• training people

Slide 94

CMM levels are:

• Initial

• Repeatable

• Defined

• Managed

• Optimizing So quality and quality assurance are different but related things. Of course they are

essentially bound up with all sorts of other properties of an organisation and the software it produces,

14.0.3 Questions to think about Q: What difference does the method of QA (or none!) that a company uses make?

• To the customers?

• To the products?

• To the employees?

• To the shareholders?

• To the ability of the company to adapt to changing situations and markets?

Q: What are the positive effects of QA? Q: What are the negative effects of QA?