http://cerg.csse.monash.edu.au/pedant applying agent technology to evaluation tasks in e-learning...
Post on 21-Dec-2015
218 views
TRANSCRIPT
http://cerg.csse.monash.edu.au/pedant
Applying agent technology to evaluation tasks in e-learning
environments
Applying agent technology to evaluation tasks in e-learning
environments
Selby Markham, Jason Ceddia & Judy SheardCSSE, Monash
Colin Burvill & John WeirMechanical and Manufacturing Engineering, U M
Bruce Field Mechanical Engineering, Monash
Linda Stern & Leon SterlingCSSE, UM
The PEDANT project grew out of a desire/need to understand how current students deal with electronic materials.
Aims to investigate
The relationship between the way students use
on-line and interactive educational tools and the
quality of their learning experience.
Using automated, agent-oriented software tools
Algorithms in Action : AIA
An interactive, visual tool for exploring programming
algorithms
CSSE
The educational tools
MOMUS Tutor
A graphics-based tutorial system on mechanical
engineering principles
Engineering
The educational tools
Web Industrial Experience Resource : WIER
A Web-based resource to support students in the
Industrial Experience project
CSSE
The educational tools
What is a software agent?What is a software agent?It has a purpose
It can be given intelligence
It is responsive
It can be designed to learn
It acts independently
What is a software agent?What is a software agent?It has a purpose
It is responsive
It can act independently
Software tools create issues that are not easily addressed by self-report techniques or
observational techniques.
Have you tried to analyse the log data from a Web-based tool?
The extent of the data is mind-boggling
Have you tried to trace the informational path used by the learner?
How often have you been unable to trace your own path when working on the Web?
The development of agent technology is a productive direction for achieving both short-term and long-term goals
•Agent technology is technologically compatible with the teaching/learning tasks.
•Agent technology can provide data about what the learner is doing rather than what he/she remembers that he she was doing.
•Agent technology provides a tool with a highly pervasive ability to carry out functional formative assessment.
Educator
Learner
Software
Learning Objectives
Learner Motives
Evaluation
Learning Tasks
Learner Behaviour
Inferred Behaviour
Responses
Learner Outcomes
Measures
Educator
Learner
Software
Learning Objectives
Learner Motives
Evaluation
Learning Tasks
Learner Behaviour
Inferred Behaviour
Responses
Learner Outcomes
Measures
2003
Educator
Learner
Software
Learning Objectives
Learner Motives
Evaluation
Learning Tasks
Learner Behaviour
Inferred Behaviour
Responses
Learner Outcomes
Measures
2003
We will have achieved the primary aim of developing:
•A rationale for defining pedagogical elements•Prototypical agents that can monitor the software•Have analysis models that can apply to evaluation tasks
Educator
Learner
Software
Learning Objectives
Learner Motives
Evaluation
Learning Tasks
Learner Behaviour
Inferred Behaviour
Responses
Learner Outcomes
Measures
2003
The evaluation component:
•The evaluation of the functional usability will have begun
•The format for evaluating the processes being used by students - a broader view of formative assessment
Educator
Learner
Software
Learning Objectives
Learner Motives
Evaluation
Learning Tasks
Learner Behaviour
Inferred Behaviour
Responses
Learner Outcomes
Measures
2003
20042004
20042004
20042004
20042004
There are many more tasks to be done to fill in the underlying matrix
This creates what can be called a process evaluation – as defined in the broader evaluation literature.
This is similar to formative evaluation but more as originally conceptualised by Scriven.
Objectives Application Outcomes
Specified task/goal
Operationalised in software
Learner responses
Pedagogical Agent
Data Log Raw data generated
by Application
Information Log Conversion of Data Log to usable information by
AgentKnowledge Log
Intelligent conversion of Information Log by
Agent
Agent action definition
The current process model for the project
Objectives Application Outcomes
Specified task/goal
Operationalised in software
Learner responses
Pedagogical Agent
Objectives Application Outcomes
Specified task/goal
Operationalised in software
Learner responses
Pedagogical Agent
Data Log Raw data generated
by Application
Objectives Application Outcomes
Specified task/goal
Operationalised in software
Learner responses
Pedagogical Agent
Data Log Raw data generated
by Application
Agent action definition
Objectives Application Outcomes
Specified task/goal
Operationalised in software
Learner responses
Pedagogical Agent
Data Log Raw data generated
by Application
Information Log Conversion of Data Log to usable information by
Agent
Agent action definition
Objectives Application Outcomes
Specified task/goal
Operationalised in software
Learner responses
Pedagogical Agent
Data Log Raw data generated
by Application
Information Log Conversion of Data Log to usable information by
AgentKnowledge Log
Intelligent conversion of Information Log by
Agent
Agent action definition
Action (User, Type, Time, [parameter list] )
action (user code, page, time [URLdata])
download (403, file_manager, 994867577, [doc_rep: 0 Requirements-Model.doc])
The practical meaning could be
Learning tasks that the software enables:
exploring a particular area accessing a particular resource
Learner behaviours:
passive lostwandering
directed interaction linear, orderly interaction exploration (at different levels of depth)