1

CE 00901 -3 fyp !

Research & Domain Analysis

Diane Bishton – K229(d.k.bishton@staffs.ac.uk)

Part 11) Project Management, and 2) Documentation

2

Project Management

Time management - one of the commonestproblems faced at level 3 (so students’ say)

The key to success at level 3Don’t just use it for your fypHow can you hope to Control what you’re doing

without a Plan ?If you’re not in control, who (or what) is ?Planning - and, more importantly USING the plan

AS A TOOL - is indicative of a professional approach

1

3

So little time, so much to do…

How many weeks ? Float ?We’re now in the third week of your first semester

If each semester is 12 weeks long, that gives you 20 weeks left including this one (maximum) but…

You will hand your final report in BEFORE the end of the second semester

There’s always the holidays (really ? )

revision ?

actually taking a a break ?

finishing other assignments ?

further reading for your other modules ?

(yes, you are expected to do these things as well))

4

Making a Plan

Gantt chart but NOT MS ProjectTM - it’s not appropriate, takes up a lot of your time to set up, so probably won’t get altered (or even looked at ?). This is YOUR project, make the Plan WORK for you.

1 … T

Produce project specification

x H

Write up Chapter 1 EProduce rough Data Model EDo a Comparison of Methods

N

FOCUS ON PRODUCTS ! D!

5

On Having a Time Machine

Unfortunately, they don’t exist … as yet.

You can’t get back days that have gone.

What evidence will you have for the time you’ve used ?

You NEED products to : make you feel better, that you’re achieving to show to supervisors or assessors & give

them confidence in your progress to cut down your work later on

…

6

Regularly Review your Plan !

Frequent reviews & FOCUS With 4 Modules per semester, that’s a day and a

bit (in your working week) for each module.

That’s ~ 12 fyp days in total left in semester 1, and

(because fyp is 2 modules in semester 2), another

25 days in semester 2 (to hand in ??). Hmmm ! So - if you spend 2 days (not much) reading a

book, with nothing to show for it, that’s the equivalent of nearly 2 weeks work … GONE

Every time you overrun, it affects something else.

7

Closing thoughts

Although this is YOUR project, there are other people involved occasionally too, so…

Make appointments well in advance - plan ahead !

Don’t forget - maybe the people you’ll need work somewhere that may be CLOSED or VERY BUSY when you think you need them - plan ahead !

You’re bound to find something that you’d wish you’d done, seen earlier etc. etc. >>Don’t panic<<

You still have your Critical Review section(because, despite how much you plan, no-one can

predict the future)

8

Documentation

Imagine you're reading about 15 - 20 of these !

Keep your Documentation up-to-date from the beginning (remember Product Focus ?)

You need to ‘distance’ yourself from the projectif you’re going to appear ‘scientific’ in your approach

No “I did…” or “…myself…” etc. , write like an observer of the project

Make connections between the Problem & your Solution EXPLICIT through Analysis & Design

2

9

Once Upon a Time …Help your reader (s)Treat them like intelligent, interested people who have a genuine desire to find out what you did, why, how, when…Don’t forget that although you should know your project intimately, they (at best) only have an overview (and, they’re only human !)Tell a story - have ‘signposts’ to join the different parts together. These should be meaningful links between your Sections / ChaptersOrganise. Arrange. Ask yourself - Does it make sense ?

10

Writing a Report is a Process

• Understand what your writing is for• Plan• Gather appropriate content• Select and Organise the Content• Prepare a Draft Copy

• Review and Proof Read the Draft (after a break)

• Final version

11

Planning (what, more !?)

• Take the time to think about what you need to Produce

• Add extra time to what you expect, for proof reading and review (and running out of paper, ink, PC collapsing under the strain etc.)

• Have you got the binding sorted ?

Do you REALLY want to be up all night panicking before the hand-in ?

12

Content• There are so many information sources now Information Overload

• Store your References as you go, in a spreadsheet

- you won’t forget or lose them, and they’re easier to sort into order later. Include Chapter, page numbers & even paragraph - you may need to go back to the reference.

• Make Lists, ‘Spider Diagrams’, Mind Mapsof the possible content of your report

Your own preferred style will affect how you approach this

13

Mind Map

From “The Brain Book” Russell, Peter

14

Good Practice

A clear Title helps you to focusMake sure you’re using the right words - look up

words you’re not sure of. Make a Glossary for us !Use a Thesaurus to make the writing more

interesting and variedOne idea per paragraphTry and sequence and link the Main PointsShorter sentences are usually easier to read, but

longer ones, with longer words, are considered to

be more ‘academic’

15

Draft

To make a ‘rough’ copy,• Start with some possible

Headings, include at least : a form of Introduction, then Main Points with Examples (evidence) (one Main Point per paragraph) then a Summary or Conclusion

(Remember the ‘signposts’ bit earlier ?)

• Your aim is to get the ‘flow’ right, so leave ‘fiddly’ (and really difficult) bits until later

Sitting near an open window cools your brain, but that’s the wrong sort of draught

FLOW !

16

Take a Break

• You’ll find it much more effective to Review your work after you’ve had a break (maybe a couple of days)

• Because you’ve written it, you’re very close to it, so it’s harder to spot errors

• At least with a PC, it should be easier to make changes, but you need to keep this in perspective. You have got other things you need to do. Start good habits now.

17

Review

Really try and concentrate on what you’ve written

Read it out loud - does it make sense ?

Make changes (but make sure you’ve got a copy first - you might make it worse - eeek !)

Put in the fiddly bits, and illustrations

Check that the writing has a clear focus on the Main Points. Aim for Quality, not Quantity.

Go back to the requirements and make sure that you’ve covered everything

18

Proof Read

Preferably, get someone who you can rely on to do it !

The chances of you finding anyone ?

(and anyway, they’ll claim it sounds great, when they don’t want to, or can’t, understand it)

If you don’t proof read it, whoever assesses it probably will, but it’s too late then isn’t it ?

Check spelling, grammar, page numbering, references, figure and table labelling & captions etc.

19

Final VersionCheck the Contents Page

Make sure all the Appendices are in, and properly referenced

Make sure your list of References is in the Bibliography

When it’s printed, make sure that it’s right justified enough,so that when it is bound, what’s on the left can still be read !

Personally, I prefer Appendices separately bound - that way I can refer to them without having to keep turning back & forth through the main report. But that’s me.

20

A typical (general) Report Structure

A Report is more formally structured than most other forms of writing. There are Conventions for breaking reports down into manageable chunks.

• Contents Page• 1.0 Introduction / Terms of Reference• 2.0 Methods or Procedures• 3.0 Findings

– 3.1– 3.2 – etc

• 4.0 Summary or Conclusions • 5.0 Recommendations• Bibliography• Appendices

21

Part of a Technical (academic) paper

From “Semiologic From Sending Messages to Managing Meetings” Stewart, Liz Director of Semiologic Ltd

22

Some real student ‘bloopers’

“In the Olympic Games, Greeks ran races, jumped, hurled the biscuits, and threw the java”

“Joan of Arc was burnt to a steak”“In midevil times, most people were alliterate”“Sir Francis Drake circumcised the world with a

100 foot clipper”“Christopher Columbus discovered America while

cursing about the Atlantic”“Queen Victoria sat on a thorn for 63 years”“Louis Pasteur discovered a cure for rabbis”

23

Punctuation ! ! ! and scope

Virgin Rail trains will be running normally, late this afternoon

Virgin Rail trains will be running normally late, this afternoon

Serve hot or cold with cream

24

The nitty gritty

You’re given a Font style, size etc. so use it !

Reading 15-20 tiny font, poorly spaced reports makes ones eyes hurt. This may translate to ones impression of the overall quality of the product & hence to ones judgement. Hmmm.

Spell check & Proof Read.Do NOT rely on a computer to pick out all the errors“… there was a need to discus…” !

25

Summary

We’ve seen that :It is important to Plan your writing, carefully

considering the purpose of it and the target audience

Producing a Draft Copy can get your ideas organised and ultimately help you to feel better

Reviewing and Proof Reading are really important things to do to improve quality

26

CE 00901 -3 fyp !

Research & Domain Analysis

Diane Bishton – K229

(d.k.bishton@staffs.ac.uk)

Part 2

1) Literature Searching,

2) Data Gathering & Recording

27

Literature Searching

Literature searching - keep focused

Scan for likely sources, then USE the Precis of a Paper & the Contents of a Book - they summarise the main points. Don’t get sidetracked (easy for me to say !)

Use HARVARD referencing (advice leaflets in the library)

Quotes ? - well, a few are alright , but most of the writing MUST be your own. Guess what - it’s fairly easy to spot changes in writing style… so…

Plagiarism - DON'T. It’s MUCH better to acknowledge your sources - you won’t be penalised for that

1

28

Data Gathering & Recording

What do you want that data for anyway ?

What Purpose will it serve ?

Categories ? - when you include, by definition you exclude. If numbers model the world, what numbers do you choose ?

What’s so important about the different groupings you’ve chosen - what features differentiate or characterise them ?

Make sure you keep proper records of the context of your data capture - dates, units etc.

2

29

Research design - essentially a 5 stage process :

1) Problem formulation 2) Case selection (e.g. by experiment,

survey, case study) 3) Data production (e.g. by observation,

questionnaires) 4) Data analysis 5) Report writing

30

These 5 stages translate into : Set a Hypothesis Decide how to test the Hypothesis Design a Task Decide on Data Collection

o Identify Variables & Sources of Biaso Determine Independent Variables (that will be manipulated)o Determine Dependent Variables

(that will be ‘observed’ for change)o Identify a suitable Pilot & Test population

(these are the same in a Longitudinal Study)o Choose Result Analysis Method (s)o Choose Result Presentation Method (s)

Pilot the experiment Review the Pilot & Conduct the Actual experiment

31

Some definitions

Independent variables : a factor being manipulated (changed) to produce different results for comparison

The Hypothesis predicts that changing the Independent variable will affect the Dependent variable e.g. different Icon designs affecting Recognition time.

 

Dependent variables : a factor being observed & measured

examples – the number of errors made ; the Task ‘stages’ at which errors are

made 

Control Group : This group’s ‘behaviour’ is compared against the experiment group

32

Where things go wrong

In determination of the hypothesis to be tested (problem formulation),

In selection of variables, measurement instruments, test populations (case selection and data production),

In sampling and analysis of data and elicitation of findings (data analysis),

In presentation of the results (report writing)

33

Things to rememberAll data rely on theoretical presuppositions.

Results and conclusions need to be at least : Credible – capable of being believed / trustworthy, and Plausible – apparently reasonable.

It is necessary to give the recipients of research findings adequate detail about the process, the context and the decisions made.

‘Triangulation’ of data by going to different sources has the potential to ‘cross correlate’ findings for verification.

So: measure what can be measured, observe (carefully – ‘presented’ behaviour is only one possible manifestation) , and ask users for their assessment.

34

Your ‘experiments’

May reveal a number of issues, for example of : 

Lack of ‘Control’ – field based versus lab based - ‘unnatural’, ‘artificial’

Bias – practice or learning effects - ‘order’ effects

Qualitative versus Quantitative ‘scientific’ measurement

(In many ways, both quantitative and qualitative studies are tentative)

35

Your ‘experiments’ cont.

Who is doing the ‘measuring’ ? People Highly variable, adaptive and subject to context

effects. They will SELECT and PREFER, perhaps subconsciously, and can ‘over’ or ‘under’ emphasise features.

Often unable to describe or ‘rationalise’ their particular preferences / observations – professional judgement – since ‘skill’ is cognitively a highly organised and SUBCONSCIOUS behaviour.

A different experimenter, or even the same one on different days, may ‘code’ an observation under a different category. Rating scales ?

36

Some real sample questions

From a questionnaire for evaluating computer usability - NPL DITC 169/90 (Wong and Rengger, 1990)

“Please remember : there are no right or wrong answers, no “trick” questions, and your answers are completely confidential”

“To which one of the following age groups do you belong ?

Under 2525-45 Over 45” what do you think ?

“How do you rate your knowledge of computing ?”

“Low Medium High” what about this one ?

37

Some real sample questions (cont.)

“On this page you will find 10 statements of opinion… Would you please rate each statement to show your agreement or disagreement with it. You should use the following rating scale : 1 Strongly disagree… 3 Have no opinion either way… 5 Strongly agree…”

“I find EDIT unnecessarily complex 1 2 3 4 5”

and then later

“I think EDIT is easy to use 1 2 3 4 5”

Look how they’ve included statements that are ‘opposite’ one another to draw out inconsistencies in respondents’ choices

38

Some real sample questions (cont.)

“Each section consists of several statements which can be rated on a scale of 0 to 9. Your ratings will therefore indicate your impressions of certain characteristics of each aspect of EDIT use. For instance :

Organisation of information on computer screenConfusing 0 1 2 3 4 5 6 7 8 9 Very clear

…Tasks can be performed in a straightforward manner

Never 0 1 2 3 4 5 6 7 8 9 Always”Look how they’ve put all the ‘negative’ responses on the

left & phrased the statements in such a way that they all come out like that - people don’t always read what’s there !

39

Some real sample contexts

From a test context document that assesses whether users’ skills / competence may affect the judgement of a product’s usability - NPL DITC 213/92 (Bowden & Kirakowski, 1992)

“Two types of user will be tested, scientific staff… and trained task experts…Potential subjects will be questioned about their usage of…Anyone who has received any formal training…, does not use at least the basic functions on a regular basis…,is not fluent in English…, or has significant sight or hearing deficiencies…, will be rejected as a subject”

40

Some real sample contexts (cont.)

“The evaluation tasks for all the subjects will be to…”

“…evaluators …will use a script to ensure that spoken communications are kept to a minimum”

“Prior to carrying out the tasks the subjects will be asked how frequently they do these tasks in their normal job…”

“The tasks will be carried out at… between 9am and 5pm from Monday to Friday…”

“The tests will take place in the Usability Laboratory in building 93… with the test room set up as an office…”

“A telephone… and other conventional office furniture will be installed in the test room…The telephone will be located on a desk…at which the subjects will be seated…”

Some real sample contexts (cont.)

“Context of Use & Measurement of the Product”

“Context of Use Likely to Affect Usability Context of Measurement

“User Types FixU1 Scientific Staff… Yes only type 1U2 Admin Staff… tested

Skills & Knowledge Experience with the product MonitorRegular Daily Usage Yes ask subjectsTyping skill varies… No

TrainingNone Yes Select

only subjects who have received no formal training”

42

Summary

We’ve looked at

• The typical stages in the process of designing an experiment

• Definitions of terms, such as ‘Independent Variable’ etc.

• Points where things go wrong in experiment design

• Things you should keep in mind when designing your evaluation ‘experiments’, if you want the results to be considered ‘Valid’