improving subject access to the office for learning and teaching's resource collection

SCHOOL OF INFORMATION STUDIES

Improving subject access to the Office for Learning and Teaching’s resource collection

Philip Hider, CSU and Barbara Spiller, ACER

http://www.acer.edu.au/contact-us


OLT Resource Library

Total OLT projects: 2012-15 $57.1 million

www.olt.gov.au/resources


OLT project titles

• A threshold concepts focus to curriculum design

• Effective teaching and support of students from low socioeconomic status backgrounds

• Enabling and developing leadership in multi-campus universities

• Enhancing the student experience: transition from VET to higher education

• Inquiry-oriented learning in science

• Nature and roles of arts degrees in contemporary society

• Tracking student success: who is falling through the cracks?


Lack of control

• Poor recall • synonym problem (including plural v singular,

inconsistent hyphenation, etc.)• misspellings

• Poor precision• homonym problem (with lack of conceptual structure)

• No thesaurus to help with query formulation


Project team

Charles Sturt University Philip Hider, Barney Dalgarno, Ying-Hsang Liu, Carole Gerts, Carla Daws & Raylee Macauley

Australian Council for Educational ResearchBarbara Spiller, Pru Mitchell and Robert Parkes

University of WollongongSue Bennett


Pre-test search effectiveness

Recall and precision ratios for current system

• only able to find less than half of relevant documents in initial searches average recall = 0.45, n = 48

• only one third of retrieved items are relevantaverage precision = 0.33, n = 80


User survey and logs

• 97% rated topic or subject as a useful field by which to search the Resource Library

• author was second most useful at 74%

• vast majority of actual searches are on topics


New schemaProject title Project acronym Lead researcher Co-researcher Lead institution Partner institution Funding body Project ID Grant type Project summary Year of completion

Topic (controlled)Discipline of application (controlled) Identifier Project website URLRelated project Resource type Resource title Resource author ISBN DOI Year of publication


Thesaurus candidatesCode Thesaurus name Coverage Updated

1 ATED Australian Thesaurus of Education Descriptors AU, all 2014

2 BET British Education Thesaurus UK, all X3 EDU ÉDUthès: thésaurus de l’éducation CA , HEd 2001

4 EET European Education Thesaurus EU, all X5 ERIC ERIC Thesaurus US, all 2015

6 ETT CEDEFOP – European Training Thesaurus EU, VET 2012

7 IBE UNESCO-IBE Education Thesaurus UN, all 2007

8 ScOT Schools Online Thesaurus AU, K-12 2015

9 TESE Thesaurus for Education Systems in Europe EU, all 2009

10 UK “Education terms” from the UK Dept of Education UK, all X11 VOC VOCED Thesaurus AU, VET X

http://cunningham.acer.edu.au/multites2007/index.html

http://www.leeds.ac.uk/educol/BEID.html

http://www.cdc.qc.ca/multites_en.htm





http://www.freethesaurus.info/redined/en/index.php

http://eric.ed.gov/?ti=all

http://www.cedefop.europa.eu/en/publications-and-resources/publications/4114

http://www.ibe.unesco.org/en/services/online-materials/publications/ibe-education-thesaurus.html

http://scot.curriculum.edu.au/

http://eacea.ec.europa.eu/education/eurydice/tese_en.php

http://webarchive.nationalarchives.gov.uk/20130123124929/http:/www.education.gov.uk/vocabularies/educationtermsandtags

http://www.voced.edu.au/content/ngv45775


Thesaurus evaluation criteria

1. Concepts and terminology2. Structure3. Scope4. Depth5. Geographic coverage

6. Maintenance7. Licensing and cost8. Usability and support9. Interoperability


ATED BET EDU EET ERIC ETT IBE SCOT TESE UK VOC ED0

10

20

30

40

50

60

70

80

90

Interoperability

Support

Cost

Maintenance

Locality

Depth

Scope

Structure

Concepts and terminology

Expert analysis


Concept matching - words

ATED BET EDU EET ERIC ETT IBE SCOT TESE UK VOC

TOTAL - Y 19 14 13 10 18 10 15 19 14 15 13

ATED BET EDU EET ERIC ETT IBE SCOT TESE UK VOC 0

2

4

6

8

10

12

14

16

18

20

Total matches


Concept matching - phrases

ATED BET EDU EET ERIC ETT IBE SCOT TESE UK VOC

TOTAL - Y 18 13 8 9 13 9 11 10 7 12 12

TOTAL - P 3 7 11 10 8 11 9 10 13 9 7

ATED BET EDU EET ERIC ETT IBE SCOT TESE UK VOC 0

5

10

15

20

25

Partial Full


Enhancement of ATED


Australian Council for Educational Research

Cunningham Library



ATED

Websites

BooksTheses

Journal articles

Australian Education Index

(AEI)Australian Education

Research Theses

Cunningham Catalogue

Australian Thesaurus of

Education Descriptors

(ATED)

BOLDE

Cunningham Indexing Services

External clients

content

concepts

concepts

records

http://opac.acer.edu.au/IDP_drie/index.html

http://opac.acer.edu.au/ierd/index.html

http://opac.acer.edu.au/drde/index.html


Australian Thesaurus of Education Descriptors

• Print, digital and online• Since 1984 for Australian Education Index• 10,348 terms - 50% references

www.acer.edu.au/ated

http://www.acer.edu.au/ated



ATED concept entry

Australian Education Index

• Identifiers• New ATED terms• Warrant level for new concepts

Record display from A+ Education, informit



ATED for OLT Resources

• Mapping exercise

1,593 OLT terms examined1,160 terms remaining after exact matches removed

• Criteria for inclusion


Mapping of OLT keywords

Category Definition n

A Terms conceptually and terminologically adequately covered by ATED 934

B Terms out of scope, i.e. proper nouns 108

C Terms in scope but not conceptually covered by ATED, but with a lack of literary warrant 65

D Terms in scope but not conceptually covered by ATED, with sufficient literary warrant 30

E Terms in scope, conceptually but not terminologically covered by ATED, lacking literary warrant 2

F Terms in scope, conceptually but not terminologically covered by ATED, with literary warrant 21

1,160Categories of ATED - OLT matched terms


Dealing with candidate terms

• Proposed structure for 51 candidate terms• Fit neatly into existing structural hierarchies or

requiring structural changes?• Scope note• Broader, narrower, related terms


28 new concepts for ATED

Acupuncture Learning analyticsAlternative medicine MeasuresAssociate degrees Online assessmentAuthorship Quantity surveyingAvatars ReciprocityChiropractic Spatial sciencesCommerce StudiosDesign based research Synchronous communication Distributed leadership Teaching research relationshipDouble degrees Threshold conceptsEnvironmental engineering Traditional Chinese medicineEnvironmental science VariablesFinal year students Variation theoryHospitality education Web 2.0


24 new ATED USE references• Academic analytics USE Learning analytics• Assessment for learning USE Formative

evaluation• Assessment tools USE Measures and

Evaluation• Authoring USE Programming• Capabilities USE Ability• Cultural capital USE Cultural literacy• Cultural competence USE Cultural literacy• Digital immigrants USE Digital literacy and

Generation gap• Digital natives USE Digital literacy and

Generation gap• Dispute resolution USE Grievance

procedures• Distributive leadership USE Distributed

leadership• Educational data mining USE Learning

analytics• Emerging technologies USE Technological

change

• Exercise science USE Exercise physiology and Sports medicine

• Histology USE Anatomy• Interactive multimedia USE Multimedia

and Interactivity• Landscape architecture USE

Landscaping• Naturopathy USE Alternative medicine• Net generation USE Digital literacy and

Generation gap• Personal digital assistants USE Mobile

devices• Process systems engineering USE

Chemical engineering• Real time communication USE

Synchronous communication• Teaching research nexus USE

Teaching research relationship• Work integrated learning USE Work

based learning


Issues - Content

• New discipline areas of academic studyAlternative medicine, Chiropractic

• Higher education fieldAssociate degrees, Final year students

• General education termsCapabilities USE Skills

• Information technologyLearning analytics, Online assessment

• Style and syntaxSynchronous communication


Issues - Methodology

• Problems searching RLphrase searching, Boolean OR / AND

• Differing perspectivesacademics meet information specialists

• Sources of authority• Discipline headings

Australian Standard Classification of Education• When is the right time?• Benefits vs costs


Conclusions

Outcome 1: Improved discovery of OLT resources(post-reindexing testing now being carried out)

Outcome 2: Strengthened ATED which should lead to improved discovery of education resources through other indexes


Support for this project has been provided by the Australian Government Office for Learning and Teaching. The views in this project do not necessarily reflect the views of the Australian Government Office for Learning and Teaching.


Thank you!

Any questions?