Coursework 2: Evaluate a smart refrigerator app

Following the Acme Co.'s initial design and development of their new smart refrigerator, they have now asked you and your group to evaluate another team's design. Acme wish to identify flaws and errors in the usability of the design mockups produced thus far, before the product is taken any further in the development cycle.

Acme wants you and your team to conduct an evaluation of the usability of another team's design and report on your findings. Your team may select one of the methods discussed in class from the list below, and write it up using the guidance provided. Furthermore, Acme wants you to use this study as a 'Pilot Study'.

In order to complete this assignment, you must:

Obtain ethics approval (see Learn)
Conduct an evaluation
Write a final report

Your team will be randomly assigned another team's Coursework 1 mock-up to evaluate. You will be able to find this mock-up in your group space on Learn. You will only be given the mock-up, not the tasks associated with it. It should be noted that your evaluation will not impact the marks received by the design team in the first assignment.

Ethics Approval

As with any study involving humans, you and your team must follow Informatics Ethics procedures. This will involved filling in the Ethics Form and submitting it via Learn. The Ethics sheet contains two parts. The first part is a customized questionnaire for the HCI course. The second part consists of the ethics procedure for a Level 1 study, from the School of Informatics. In addition to these two components, you will be required to also submit any materials intended to be used during your study, such as consent forms, questionnaire, etc.

You must submit via Learn:

Completed Ethics Form
Consent form (if any)
Advertisement (if any)

After you submit the form the HCI TA will look through it and either give approval to proceed or ask you to make some changes. There is no penalty for having to make changes. At this stage the TA is only looking for ethics issues, he will not be judging if your study design is a good idea or not. Just if it meets ethics requirements.

Your team must complete ethics approval before any part of your study can be conducted. This is true even if you select a methodology which does not involve outside participants.

If you fail to get ethics approval before doing your study you will automatically fail this assignment.

It is recommended that you turn in your ethics forms a week before the coursework deadline to ensure that you have enough time to run the study.

Evaluation and Final Report

Having received permission through the ethics procedure, your team will conduct your study, collect results, and report your findings. Your write up will be dependent on which methodology you have chosen, but all reports will have a similar structure. This includes:

Introduction: Provide context for the pilot study. (About 1 paragraph, 2 points)
Reasoning for method selection: Which method have you selected and why do you feel that is the most appropriate method for this coursework? (About 1 paragraph, 3 points)
Methodology: Details of the method chosen. Each of the recommended methods below have a list of the type of information that needs to be provided here. (About 1 page, 17 points)
Results: Give a general overview of the results obtained. (About 1 page, 17 points)
Recomendation: Based on your results, what should be changed about this design? (1-3 paragraphs, 7 points)
Reflection: One paragraph from each group member seperately labeled with their name answering the question: "In your opinion, when running the study you have described above, what worked, what didn't, and if you had to do it again what would you change?" - This paragraph is the only part of the coursework that is individually assessed. (One paragraph per team member, 4 points)
Appendix: Provide all materials used in your methodology, and a sample of data collected. For example, if using interviews, provide a transcription of 2 question/answer pairs for one participant. We don't need to see all the data, just enough to understand what it looks like. Roughly we are looking for the equivelent of running the "head" command on your data. (Assessed as part of Methodology and Results)

Pilot Study Methodologies

Below is a list of suggested methods you may select from. These methodologies are all reasonable approaches, but some are easier to use for this task than others. You must only choose one methodology and will receive no extra marks for conducting multiple evaluations. Each methodology has a simple guide as to how to write up, with links to examples and explanations. The methodologies include:

Frequently Asked Questions

Can we get the 3rd and 4th tasks the prior group submitted?: No. You should construct your own tasks (if needed) from the app mock-up itself. The point here is to get you to build at least some of your materials on your own. We do not expect these tasks to match what the prior group proposed.