Design, Implementation and Design, Implementation and Evaluation of Brokerage Evaluation of Brokerage

Systems of Learning ResourcesSystems of Learning Resources

Lampros K. StergioulasLampros K. Stergioulas

Department of Information Systems & Department of Information Systems & ComputingComputing

Brunel University, UKBrunel University, UK

22

Overview of presentationOverview of presentation

Universal Brokerage PlatformUniversal Brokerage Platform Value chain and user benefitsValue chain and user benefits Platform ArchitecturePlatform Architecture Service and LR EvaluationService and LR Evaluation

The need for Brokerage:The need for Brokerage: eLearning business: far from lean eLearning business: far from lean Replication & WasteReplication & Waste Re-UseRe-Use is spectacularly absent is spectacularly absent

33

Universal Brokerage PlatformUniversal Brokerage Platform

• The Universal Brokerage Platform is a distributed, multi-lingual system supporting the exchange of learning resources

• Challenges: • Neutral business model • To interface with heterogeneous content

delivery systems such as – Web servers, – Streaming media servers, – Videoconference systems, and– Learning Management Systems

44

The UNIVERSAL Project:The UNIVERSAL Project:Universal Exchange for Pan-European Higher EducationUniversal Exchange for Pan-European Higher Education

Focus on Higher Education Institutes (HEIs)Focus on Higher Education Institutes (HEIs) Brokerage Service: Not just a catalogueBrokerage Service: Not just a catalogue Aims to Serve:Aims to Serve:

HEIsHEIs Corporate TrainingCorporate Training IndividualIndividual learners learners

Consortium: Consortium: 17 members 17 members from the EUfrom the EU Higher Education Institutions with  engineering and Higher Education Institutions with  engineering and

business studies business studies Technology supplier/integratorTechnology supplier/integrator & & Network service Network service

operatorsoperators external collaborators in Canada and Iceland external collaborators in Canada and Iceland

55

Scope of workScope of work

Key innovation:Key innovation: Flexible and open inter- Flexible and open inter-organisational information system; organisational information system; tight integration of various content and tight integration of various content and live delivery systems, such as learning live delivery systems, such as learning management systems, web server, media management systems, web server, media streaming server and video conferencing streaming server and video conferencing tools.tools.

Learning Resource (Def.):Learning Resource (Def.): a form a form (material or activity) of highly specialised (material or activity) of highly specialised academic content. A learning resource can academic content. A learning resource can be a short video or a complete course.be a short video or a complete course.

66

Service:Service: Brokerage of Learning ResourcesBrokerage of Learning Resources Approach:Approach:- - Open exchange of Open exchange of LRLRss- Interface between - Interface between bbrokerage platform and different rokerage platform and different delivery systemsdelivery systems- Compatibility with various business models- Compatibility with various business models- Collaboration catalyst between Higher Education - Collaboration catalyst between Higher Education Institutions and companiesInstitutions and companies- Standardised descriptions of pedagogical, - Standardised descriptions of pedagogical, administrative, and technical characteristics of LRsadministrative, and technical characteristics of LRs

Platform Features:Platform Features: EnquiriesEnquiries LR ProvisionLR Provision LR BookingLR Booking LR DeliveryLR Delivery

77

Contributions of UniversalContributions of Universal

The TechnologyThe Technology for building knowledge for building knowledge exchanges and educational brokers is known exchanges and educational brokers is known as the as the Universal Brokerage Platform (UBP).Universal Brokerage Platform (UBP).

The ServiceThe Service for sharing learning resources for sharing learning resources among faculty members in higher education is among faculty members in higher education is known as known as EducaNext.EducaNext.

88

99

Benefits toBenefits to LearnersLearners

Higher degree of specialisation and variety Higher degree of specialisation and variety of approachof approach

Wider choice of LRsWider choice of LRsLower cost and higher quality of teachingLower cost and higher quality of teaching

1010

Benefits toBenefits to HEIs/Corporate Training HEIs/Corporate Training ProvidersProviders

Faculties & organisations can easily re-use each other’s Faculties & organisations can easily re-use each other’s LRsLRs

enrich courses with external material, lowering the cost enrich courses with external material, lowering the cost and raising the qualityand raising the quality

Foster national and international academic Foster national and international academic alliances and exchangesalliances and exchanges

Enable/reinforce partnerships between Enable/reinforce partnerships between institutions of higher education and industry institutions of higher education and industry

Improve their teaching materials through Improve their teaching materials through feedback from usersfeedback from users

Maximise use of their LRs, minimise “waste”Maximise use of their LRs, minimise “waste” Gain reputation in a constantly growing Gain reputation in a constantly growing

communitycommunity Access new academic distribution channelsAccess new academic distribution channels Open Universities to extend range and depthOpen Universities to extend range and depth

1111

EU user-centred value EU user-centred value objectivesobjectives

Openness of UBP

(technical)

Richness of LRs

(pedagogical)

Diversity of Users

(marketing)

Scope of delivery sys.

supported

- topics- country of origin- up-to-date- existing curricula

Scope of geogr./ling. distribution

1212

Broker

Open System Architecture Open System Architecture for Materialfor Material

Content Delivery system (e.g. web server, Learning

Management System)

Metadata

Learning resource

Content Delivery system (e.g. Web server, Streaming

Server, Video Conferencing Tool) Provider PC

Provider system

System interface

1313

Open System Architecture Open System Architecture for Live Activitiesfor Live Activities

4. ConsumptionDelivery system for

live educational activities(e.g. ISABEL)

2. Provision

UNIVERSAL BrokeragePlatform

1. Announce- ment

3. Booking

4. Consumption

4. Consumption

1414

Platform ArchitecturePlatform Architecture

1515

User Interface EngineUser Interface Engine

Main FunctionsMain Functions To provide a user friendly dialogue capability between To provide a user friendly dialogue capability between

users and the platformusers and the platform To exchange information with other system To exchange information with other system

components (engines), as required.components (engines), as required.

Multilingual: English, French, German, Multilingual: English, French, German, and Slovenian and Slovenian

Quick edit mode allows instantaneous Quick edit mode allows instantaneous adaptations for web page editorsadaptations for web page editors

1616

Administration EngineAdministration EngineActs as a records Acts as a records rrepository. Thus its main operational epository. Thus its main operational objectives are to objectives are to

keep records of users and transactionskeep records of users and transactions ( (ddetailed access etailed access and transaction statisticsand transaction statistics))

provide data to other enginesprovide data to other engines Initiate, organise, and sustain actions in the platformInitiate, organise, and sustain actions in the platform Advanced user managementAdvanced user management E-mail notification management toolsE-mail notification management tools

ContracContract & Bookingt & Booking Engine EngineCE: CE: Enables formal transactions between users and Enables formal transactions between users and

platform and invokes contract formulation, acceptance platform and invokes contract formulation, acceptance and billing mechanismsand billing mechanisms

Tracks Learning Resource Usage (Research, Teaching, Tracks Learning Resource Usage (Research, Teaching, Course Preparation, Preview)Course Preparation, Preview)

Personalized List of Booked Learning ResourcesPersonalized List of Booked Learning Resources Triggers Delivery (and Assessment)Triggers Delivery (and Assessment) Notifies Learning Resource ProviderNotifies Learning Resource Provider

1717

User Profile EngineUser Profile Engineis responsible for creating and maintaining users’ is responsible for creating and maintaining users’ profiles and user authentication filesprofiles and user authentication files

LR Profile EngineLR Profile EngineThis engine keeps LR information. Its main operational This engine keeps LR information. Its main operational objectives are toobjectives are to

store LR data store LR data search LR data and provide information in response to search LR data and provide information in response to

user requests.user requests.

1818

Evaluation EngineEvaluation Engine

The The Evaluation engineEvaluation engine supports quality evaluation of supports quality evaluation of learning resources and platform activities. In doing learning resources and platform activities. In doing so, it is focused on the following operational so, it is focused on the following operational objectives: objectives: to collect evaluation data and to collect evaluation data and provide search and analysis capability on provide search and analysis capability on

evaluation and other user-related dataevaluation and other user-related data

1919

Delivery EngineDelivery EngineThe brokerage platform does not store LR contents; The brokerage platform does not store LR contents; instead it offers an interface layer which provides instead it offers an interface layer which provides communication functionality between the platform communication functionality between the platform and various delivery systems. This is the and various delivery systems. This is the responsibility of the responsibility of the delivery enginedelivery engine, which provides , which provides authentication and authorization services, delivery authentication and authorization services, delivery negotiation and delivery supervision. Presently, the negotiation and delivery supervision. Presently, the platform provides support to the following set of platform provides support to the following set of delivery systems:delivery systems:

Standard web serverStandard web server Hyperwave’s E-learning SuiteHyperwave’s E-learning Suite RealNetworks Real serverRealNetworks Real server Isabel Teleconferencing SuiteIsabel Teleconferencing Suite

2020

DeliveryDelivery

2121

Platform Data ManagementPlatform Data Management

User DatabaseUser Database Learning Resource DatabaseLearning Resource Database Evaluation Database Evaluation Database System Log DatabaseSystem Log Database Booking DatabaseBooking Database

Evaluation Data

LR Qualityevaluation

QuestionnairesLR evaluation Data

UPB Qualityevaluation

Questionnaires

UBP evaluationData

2222

Evaluation ServicesEvaluation Services

This activity is focused on:This activity is focused on: Evaluation of LR quality and collection of feedback Evaluation of LR quality and collection of feedback

that can be used to: that can be used to: improve LRsimprove LRs encourage acceptability (and even accreditation) of encourage acceptability (and even accreditation) of

LRs by professional institutions and educational LRs by professional institutions and educational organisations (HEI’s). organisations (HEI’s).

Evaluation of the effectiveness and completeness Evaluation of the effectiveness and completeness of the brokerage service, which will provide a of the brokerage service, which will provide a measured understanding of the quality of brokerage measured understanding of the quality of brokerage service and leads to future evolutionary/improved service and leads to future evolutionary/improved versions of the system.versions of the system.

2323

LR LR EEvaluation Servicevaluation Service

In LR In LR EEvaluation Service, the platform collects from users valuation Service, the platform collects from users LR evaluation data which is then stored in the LR LR evaluation data which is then stored in the LR evaluation database. This data is analyzed by evaluation database. This data is analyzed by the the evaluation engine and is transformed into meaningful LR evaluation engine and is transformed into meaningful LR evaluation information, which adds effectively a unique evaluation information, which adds effectively a unique “LR quality” dimension to the UNIVERSAL brokerage “LR quality” dimension to the UNIVERSAL brokerage operation. operation.

Fast feedback support via spot light reviews and meta Fast feedback support via spot light reviews and meta reviews, “How useful was this review?”reviews, “How useful was this review?”

Option to create Option to create customcustomised evaluation questions and ised evaluation questions and integrate them into a powerful evaluation toolintegrate them into a powerful evaluation tool

2424

LR evaluation data is collected from the following LR evaluation data is collected from the following users:users: Learners (students – consumers)Learners (students – consumers) HEI administrators and academic staff (faculty consumers)HEI administrators and academic staff (faculty consumers) LR providersLR providers

The evaluation data collection process is initiated by The evaluation data collection process is initiated by the administration engine. The administration engine the administration engine. The administration engine logs all platform usage activity, including LR delivery, logs all platform usage activity, including LR delivery, and is in a position therefore to activate the LR and is in a position therefore to activate the LR evaluation data collection process on-line. evaluation data collection process on-line.

LR evaluation: Data CollectionLR evaluation: Data Collection

2525

Platform/Service evaluationPlatform/Service evaluation

Technical evaluationTechnical evaluation Functionality evaluationFunctionality evaluation User SatisfactionUser Satisfaction

viavia Automatic logging activitiesAutomatic logging activities User feedback/comments/complaintsUser feedback/comments/complaints Automatic alert systemAutomatic alert system

User DrivenUser Driven System ActivatedSystem Activated

2626

Current Current StatusStatus

Service available on-line since April 2002 Service available on-line since April 2002 LR Usage and LR quantity growing exponentially LR Usage and LR quantity growing exponentially Continuous Evaluation and RefinementContinuous Evaluation and Refinement Peer ReviewsPeer Reviews

MISSIONMISSION STATEMENTSTATEMENT::UNIVERSAL is a world-class brokerage service UNIVERSAL is a world-class brokerage service

linking educators for exchanging/distributing linking educators for exchanging/distributing learning resources.learning resources.

2727

Universal Brokerage Platform @ EducaNextUniversal Brokerage Platform @ EducaNext

2828

Further information can be found @Further information can be found @http://www.ist-universal.orghttp://www.ist-universal.org//

Brokerage Service Brokerage Service for Learning Resourcesfor Learning Resources

available atavailable athttp://www.educanext.org/http://www.educanext.org/