28 09-2011 user-centred development of sakai survey tool
DESCRIPTION
TRANSCRIPT
User-centred development of the Sakai survey tool
Fawei [email protected]
Oxford University Computing Services
Software piloting: not just pressing & clicking
University of Oxford
Source: http://www.flickr.com/photos/earthworm/5889087243/
Demo: WebLearn Surveys tool
WebLearn Surveys tool pilot
Overview
What can the WebLearn Surveys tool do ?
• create an online survey • deliver to Oxford users or general public• view and export survey results • use for course experience feedback, research purposes, or general data gathering
Terminology
Survey creator
Survey respondents
Survey
WebLearn site
participants
Ad-hoc groups
General public
Template – a set of questions (items)
My Surveys
- Specifications such as dates etc
- All your surveys
Three steps in creating a survey
1Create a template
2
Assign the survey to recipients (unless public)
Configure survey settings
3
Survey dashboard
Create a new template
1
Add a question (a)
Question types
1
Add a question (b) 1
Edit template 1
prev
iew
edit
dele
te
Create a new survey (a) 2
Select your template
Create a new survey (b) 2
Assign survey to respondents3
For a public survey, just click “Next’ without choosing respondents
Recipients’ view of a survey
Scale question with a comment box
Optional text field for name
My surveys page
View partial results
View final results
Survey results
An example of a PDF summary report
A frequently requested IT application
Source: http://www.flickr.com/photos/jacobwhittaker/3388288370/
A free survey tool please
Oxford University strategic plan 2008–9 to 2012–13
“Create more effective processes for the
monitoring of student feedback”
“Review the mechanisms for ensuring that the
feedback of student views which it gathers is
systematically monitored and acted upon”
Intended outcomes:
• To test the functionality of the survey tool
• To improve the tool – identify and eliminate
WebLearn Surveys tool pilot project
WebLearn Surveys tool pilot project
Steve Jo
bs, 1997 W
WD
Ch
ttp://w
ww
.you
tub
e.co
m/w
atch
?v=
FF
-tKL
ISfP
E
Steve Jobs at WWDC 1997
Intended outcomes:
• To customise the tool – meet user needs and requests
• To help users to develop good surveys
WebLearn Surveys tool pilot project
Outline of the project
Time Activities Deliverables
11/09 Recruit participants
(40 people)
Workshop 1
Workshop 2
Workshop 3
Project survey
Focus group
Observation
§1
12/09
01/10
02/10
03/10
04/10
05/10
06/10
07/10
Phase I
Phase II
Gather U
ser Feedback
…see later
Recruit participants
• mailing lists and personal contact: expectations of the project team benefits to the participants
• 40 participants: academics researchers administrators
Mixed methods
• Workshops: demonstrate the tool observe participants creating their
own online surveys provide personalised support
• Email communication to follow up• WebLearn support site• Pilot project survey
‘walk the talk’ demonstrate good practice evaluate success of the pilot project
Three workshops
• Format: presentation demo hands-on practice Q&A
• Characteristics: similar activities for each workshop user requests shaped the hands-on and
Q&A working collaboratively with participants
Developing pilot project survey
• Survey design consult with participants design questions seek advice from experts to refine
questions test survey with a small number of
participants
• Survey delivery: July 2010, 22 responses (55%)
Survey follow-up
Focus group session:
• respond to comments in the survey results
• how to improve response rate design survey questions: useful to the survey
participants engage a small sample of survey participants
in the design process offer an incentive use reminders wisely: most people respond to
a survey either in the first couple of hours or just before the deadline
Evaluation of the project
Time Activities Deliverables
11/09 Recruit participants
(40 people)
Workshop 1
Workshop 2
Workshop 3
Questionnaire
Focus group
Observation
• an improved survey tool
• 66 completed surveys designed and delivered by participants
• project report
• training guides
• training course – WebLearn Surveys
• a set of public surveys used staff across the university, e.g. the College student experience survey
12/09
01/10
02/10
03/10
04/10
05/10
06/10
07/10
Phase I
Phase II
Gather U
ser Feedback
Contribution to Sakai community• Course evaluation + public
surveys
• Simplified for public survey
• View results of open surveys
• Fix bugs: Extension dates now work Caching problems fixed Results can be deleted Better error messages Notifications and reminders now work
Evaluation of the project
Evaluation of the project
• expanded user communities
• publicity for OUCS services, including WebLearn
• networking: pilot participants and experts in evaluation
Comparison of survey tools
Thank you, and…
More information: • http://www.weblearn.ox.ac.uk/info• [email protected]
http://ww
w.fotopedia.com
/items/
flickr-2200500024