HESA Training Seminar DLHE Introductory

10 September 2008

Introduction to HESAJane Wild, Director of Operations

Governance

HESA is a Company Limited by Guarantee …

… whose Members are Universities UK and Guild HE, who appoint Board …

… so HESA is a sector-owned agency …

… whose relationship to government departments and funding bodies (whom we call Statutory Customers) is governed by formal agreements or contracts.

Management

61 FTE staffChief Executive Professor Robin Sibson …… plus six Heads of Units …… plus 60 other staff, almost all highly-

skilled

Budget £3.9 million for 2007/08HEI core subscriptions £2.7 millionContract funding £0.7 millionCommercial and other income £0.5 million

Company based in Cheltenham

Why do we exist?

Established following Further & Higher Education Act 1992 …

… to be the central data collection and provision agency for UK higher education …

… and making first {all electronic} data collections for 1994/95.

Models for how we operate

Model 1: HESA acts on behalf of the sector to enable it to meet the statutory data requirements of funding bodies and government departments

Model 2: HESA exercises statutory powers of data collection from HEIs on behalf of its Statutory Customers

Both models are valid Minimising the accountability burden

What do we do?

Annual data collections from institutions

Creation of data bases from collections

Supply of data to Statutory Customers

What else do we do?

Regular paper and electronic publications, some National Statistics output

Supporting teaching quality assurance, equal opportunities and widening participation

Publication of Performance Indicators Information provision service DLHE web service HEIDI

Why do we collect data?

Because a Statutory Customer states a requirement for it – and nowadays has to be prepared to defend that requirement on the basis of actual usage …

… or because the HE sector needs the data for its own purposes …

… but not at the behest of anyone else … … and not on the off-chance that it might

be useful. Fit for purpose… the data quality agenda

is always central.

What data do we collect?

Student Record Collected in respect of all students registered in the

reporting institution who follow courses that lead to the award of a qualification or institutional credit

Aggregate Overseas Record Collected in respect of students studying (to date) wholly

outside the UK who are either registered with the reporting institution or who are studying for an award of the reporting institution

DLHE Record Collects destination information of the leaver 6 months

after graduating, also a 3½ year sample follow-up

What data do we collect? ITT In-Year Record Collection covering students in England on a course of Initial

Teacher Training (ITT) leading to Qualified Teacher Status - administrative data to meet statutory requirements for Provisional Registration of ITT students with the GTCE & also TDA Trainee Numbers Census

Staff Record Collects information about all staff at HEIs Finance Statistics Return Collects details of the consolidated income and expenditure

account, balance sheet and cash flow statement Campus Information System Collects identification for all campuses of the reporting

institution which are located within the UK and also the identity of campuses located outside the UK

How do we do it?

All data collections now web-based Aardvark data collection software

developed by HESA Data bases run under Oracle 10g on Intel

(PC) servers running Linux Paper and electronic publications

produced in-house, only actual printing and CD duplicating outsourced

Working with data protection Meeting standards: ISO9001 and BS7799

Who are our customers?

Statutory Customers: government departments and funding agencies in England and the devolved administrations DIUS, HEFCE, TDA, SG, SFC, WAG, HEFCW, DEL[NI],

RCs, GTCE governed by contracts, SLAs, and MoUs

The HE sector in the UK 168 institutions; representative and sector bodies

The public interest in HE students, employers, trades unions, the media,

academic researchers, …

DLHE Methodology

Overview

• DLHE is a survey of all leavers from HE who meet the criteria for inclusion

• Finds out what leavers are doing six months after completed studies

• Forms the first part of the two stage DLHE survey which investigates the career patterns of leavers over three and a half year

Timing

• Two census dates each year:

- April for leavers who completed their studies between 1 August and 31 December the previous year

- January for leavers who competed their studies between 1 January and 31 July

• Two data capture reference dates but only one return

Who uses the data?

• Institutions

• HESA

• Statutory Customers

• Ad hoc enquiries

• Those involved in production of TQI and performance indicators

DLHE and the Student Record

• Target population for DLHE is produced from the relevant Student Record

• April Target List System to assist with POPDLHE for April survey

• Read only access codes to the HESA data collection system for DLHE contacts to assist with POPDLHE for January survey

POPDLHE

• Relevant qualifiers

• All modes bar dormant

• UK and EC

• www.hesa.ac.uk/07018/coverage

Information required by HESA• Questionnaires A3 flat or A4/A5 folded

• Second mailing

• Telephone scripts

• Welsh questionnaires

• Confirm two delivery addresses

• Confirm POPDLHE or provide new figures

Delivery

• Sent direct from printers

• April survey – delivered in March

• January survey – delivered September

• Check delivery

• Any problems – contact Institutional Liaison

Data capture

• A number of data capture methods are available and are detailed in the DLHE Methodology

• When to use each method must follow the schedule for collection

Covering letter and email

• Each time a leaver is contacted certain information should be imparted

• This information is detailed in the covering letters and emails available on the website

• Again, how to use these letters and emails is detailed in the DLHE Methodology

Methods

• Postal or electronic version of questionnaire

• Telephone follow-up – leaver or third party

• Direct contact with the leaver

• Other informed source

Remember…

• Always impart the information in the covering letters and emails whenever a leaver is contacted

Response rate targets

• Currently these are set as follows:– UK domiciled students:

•Full-time 80%•Part-time 70%

– Other EU domiciled students:•All modes 50%

Audit trail

• All responses returned to HESA need to be supported by an audit trail in the institution– Questionnaire signed and dated

by the respondent– Telephone script signed and

dated by the caller

Good Practice Guide

• The Good Practice Guide features throughout the DLHE Methodology

• It will help institutions to run an effective data collection procedure

DLHE development for the 2007/08 survey

• Improved accessibility

• Reassessed core and non-core questions

• Resolved issue about multiple qualifications

• Improved guidance

• Included a Longitudinal DLHE opt-out box

And finally…

• Use the DLHE Methodology for advice and guidance

• Helps institutions work out which combination of options works for them in achieving the response rate target

• Answers a lot of questions and should be first point of reference

Developments

• Consultation about use of the HTML centrally-hosted online DLHE versions of the questionnaire

• Possibility of extending the DLHE survey system for use as a data entry tool

• Use of the April target list system

Task 1

Selecting the correct population to survey

The DLHE Return (C07018)

Objectives

• How the data collected in the survey (paper questionnaire and telephone script) is returned to HESA

• How questions link to fields– Fields that are not linked to questions

• Core questions/fields

• Common pitfalls

General guidance

• Core questions – Identified on the telephone script– Must be completed where applicable

• Identify the category of leaver and ensure the appropriate sections have been completed– Part-time– Initial Teacher Training– Research Council funded

• Contact leaver sooner rather than later for confirmation/clarification if required

General guidance

• No blank fields

• If the section does not apply to the leaver then the appropriate default ‘X’ code should be returned

• ‘Not used’ fields within the record

• Fields that are not linked to the questionnaire

Returning DLHE responses to HESA

• Input data from questionnaire/telephone script

• Extract file

• Validate file

• Submit file to HESA

• File must conform to the 2007/08 DLHE Record Specification at:

• http://www.hesa.ac.uk/index.php?option=com_collns&task=show_manuals&Itemid=233&r=07018

Fields that are not linked to the questionnaire/telephone script• Fields 1-3 Identifiers

• Not linked to questionnaire/telephone script

• Field 1 Record type indicator (RECID)– 07018 for 2007/08 DLHE

• Field 2 HESA institution identifier (INSTID)– 4-digit codes e.g. 0113 Brunel University

• Field 3 Student identifier (HUSID)– A 13-digit numeric code which is the HUSID from the student record

(matched to POPDLHE)

Fields that are not linked to the questionnaire/telephone script

• Field 4 Method of data collection (METHOD)

• This field describes the survey method used

• Used to determine which method is more successful

• Drives validation

Fields that are not linked to the questionnaire/telephone script

• NEW! Field 57 April or January survey (APRJAN)

• This field is to record whether the graduate was surveyed in April or January

• To allow graduates who obtained more than one qualification in a reporting year to be surveyed twice and their responses linked to the appropriate student record

Section A What will you be doing?

• Q1 / F5

• Q2 / F6

• Core questions for all leavers

• Need to determine what the leaver is doing with respect to BOTH work and further study

• Questions drive validation

Q1. Which of the following statements best describes your employment circumstances

• Field 5 Employment circumstances (EMPCIR)

Field 5 Employment circumstances (EMPCIR)

• Options describing type of work:– 01 Employed full-time in paid work– 02 Employed part-time in paid work– 03 Self-employed/freelance – 15 Voluntary work/other unpaid work

• Options describing reasons why leaver is not employed:– 10 Taking time out in order to travel – 11 Due to start a job within the next month – 12 Unemployed and looking for employment, further

study or training – 13 Not employed but NOT looking for employment,

further study or training – 14 Something else– 16 Permanently unable to work/retired– 17 Temporarily sick or unable to work/looking after the

home or family

Q2. Were you involved in either full-time or part-time study or training or registered as a research student?

• Field 6 Study (MODSTUDY)

• Nature of study for leavers involved in further study:– 1 Full-time study – 2 Part-time study

• Leavers not involved further study: – 3 Not in study, training or registered as a research

student

Section A What will you be doing?

• Core questions and core fields

• Default code for explicit refusals and deceased

• QUESTION:-What if the leaver does not answer these core questions?

Section A What will you be doing?• ANSWERS:-

• If the leaver does not answer Q1. or complete Section B on employment, but indicates that they are in full-time study in Q2. then return code '13‘ Not employed but NOT looking for employment, further study or training in Q1.

• A response to Q2. must be obtained from the leaver even if Section C on Further study, training or research has been completed

Section B Your employmentCore questions for

employed leavers:-

• Q3/F11, Q9/F65, Q10/F9

Non-core questions:-

• Q4/F11, Q5/F13, Q6/F14, Q7/F65, Q8/F58, Q11/F12,

Q12/F15, Q13/F16, Q14/F17-19, 21, 59-62,Q15/F25, Q16/F26, Q17/F27-32

• Questions not on telephone script:-

• Q13, Q14

Q3. What was your job title?

• Field 10 Job title (JOBTITLE)

• Core question for employed leavers

• However, returning response to HESA is currently optional

• Computer Assisted Structured Coding Tool (CASCOT) comparison will be carried out where job titles are returned

• Default code XXXX

Q3. What was your job title?Q4. Briefly describe your duties• Field 11 Standard Occupational Classification

(SOCDLHE)

• Q3. Core question for employed leavers

• Q4. Not a core question - may help in SOC coding

• Core ‘field’ for employed leavers

• CASCOT – Standard Industrial Classification (SIC) and SOC coding tool

• Current version of CASCOT will be updated shortly to include SIC 2007

• Default code XXXXX

Q6. What was your annual pay?

• Q6. What was your annual pay to the nearest thousand (£), before tax? If you were employed for less than a year or are part-time, please estimate your pay to the full-time annual equivalent

• Field 14 Salary (SALARY)

• Format – leading zeros (£15,341 returned as 015000)

• Not a core question

• Leaver can refuse to provide information

• Full-time annual equivalent

• Default code XXXXXX

Q7. What is the name of the organisation you were working for?

• New Field 64 Name of employer (EMPNAME)

• This field is compulsory if the leaver is funded by a Research Council but is optional for all other leavers

• Default code XXXX

Q8. Is this organisation part of the NHS?

• NEW! field 58 NHS organisation (NHSORG)

• This field records if the graduate was employed by the NHS

• To see if those completing healthcare courses go on to be employed by the NHS

Q7. What is the name of the organisation? Q9. What does the organisation mainly make or do? • NEW! field 65 Standard Industrial

Classification (SIC2007)

• Replaces field 8 which becomes ‘Not used’

• Q7. Not a core question

• Q9. Core question

• Core ‘field’ for employed leavers

• Default code XXXX

Q10. Where was your place of work?• Q10. Where was your place of work and, if in

the UK, what is its postcode?

• Field 9 Location of employment (LOCEMP)

• Outward part of the postcode, i.e. the characters to the left of the space

• Country code

• HESA default postcode list (where the town/city is known, but the postcode is not)

• Core question for employed leavers

• Default code XXXXXXXX

Q14. Why did you decide to take the job you will be doing?• Fields 17-19, 21, 59-62 Career related

codes 1-3, 5, 9-12– Field 17 - It fitted into my career plan/it was exactly the

type of work I wanted– Field 18 - It was the best job offer I received/only job

offer I received– Field 19 - It was an opportunity to progress in the

organisation– Field 21 - To see if I would like the type of work it

involved– New field 59 - To gain and broaden my experience in

order to get the type of job I really want– New field 60 - In order to earn a living/pay off debts– New field 61 - It was the right location– New field 62 - The job was well-paid

Q14. Why did you decide to take the job you will be doing?

Employed Employed Employed Not employed

Field 17 1 1 0 X

Field 18 1 1 0 X

Field 19 1 1 0 X

Field 21 0 1 0 X

Field 59 0 1 0 X

Field 60 1 1 0 X

Field 61 0 1 0 X

Field 62 1 1 0 X

Section C Further study, training or research

Core questions for those engaged in further study, training or research:-

• Q18/F33, Q19/F36

Non-core questions:-

• Q20/F34, Q21/F34, Q22/F35, Q23/F37-43 and 63, Q24/F46

Questions not on telephone script:-

• Q23

Q18. Which of the following best describes the basis on which you were studying, training or researching?

• Field 33 Nature of study/training (NATSTUDY)

• Core question – leavers studying

• Default code X

Q20. What is the name of the course you were registered on? Q21. What subject area will you be studying/researching?

• Field 34 Professional subject of training

• Return a SOC(DLHE) code in this field

• To find out about the nature of professional training

• Not core questions

• Default code XXXXX

Q23. Why did you decide to undertake further study, training or research?• Field 37 - Because it was a requirement of my

employment on 14 April 2008/12 January 2009 that I didField 38 - To develop a broader or more specialist range of skills or knowledgeField 39 - To change or improve my career optionsField 40 - Because I was interested in the content of the courseField 41 - Because I had enjoyed my first course and wanted to continue studyingField 42 - I wanted to go on being a student/I wanted to postpone job huntingField 43 - I had been unable to find a suitable jobField 63 - Other

Q23. Why did you decide to undertake further study, training or research?

• Field 37 - Because it was a requirement of my employment on 14 April 2008/12 January 2009 that I did

• This option implies that the leaver is in employment and undertaking further study

• Therefore the section on employment should be completed…

Section D Qualified teacher status (QTS) only

• Q25 / F47

• Q26 / F48

• Q27 / F49

• Q28 / F47

• All questions in this section are core questions for leavers who have completed a qualification (as reported in the student record) giving newly qualified teacher status

Section D Qualified teacher status (QTS) only

• Leavers who did not complete a qualification giving newly qualified teacher status are directed to tick the circle in the header - in which case code ‘9’ should be returned in Field 47 Teaching employment marker (TCHEMP):– Not Applicable (leaver did not complete an

initial teacher training course leading to QTS/a course of pre-service teacher training (as defined by the SE))

Q25. Were you employed as a teacher? Q28. Were you seeking a teaching post? • Field 47 Teaching employment

marker (TCHEMP)• If the response to Q25 is ‘yes’ the

leaver is directed to Q26 and code ‘3’ is returned in Field 47 TCHEMP

• For such leavers, the response to Q25 should not be overwritten by the response to Q28 i.e. code 3 should be returned over 1 or 2 in TCHEMP

Q25. Were you employed as a teacher? Q28. Were you seeking a teaching post?

•Field 47 Teaching employment marker (TCHEMP)

• If the response to Q25 is ‘no’ the leaver is directed to Q28. If the answer to Q28 is 'yes' return code '1‘ in TCHEMP, if the answer to Q28 is 'no', code '2‘ in TCHEMP

Q26. Were you teaching in a maintained (state) or non maintained (independent) school or college?

• Field 48 Teaching sector (TEACHSCT) Q29.

• Not applicable to leavers teaching in N.I.

Section E Did you gain your qualification in 2007 by studying part-time?Core questions for

part-time qualifiers:-

• Q29/F50

Non-core questions:-

• Q30/F51, Q31/F52-56

Question not on telephone script:-

• Q30, Q31

Q31. Did the employer you worked with before or during your course give you support during all or part of your course?

• Field 52-56 Employer sponsorship 1-5 (EMSPNS1-5)– Field 52- Yes: my tuition fees were paid

Field 53 - Yes: I was given a grant to cover my tuition fees and living expensesField 54 - Yes: I was given study leaveField 55- Yes: I was supported in other waysField 56 - No: my employer did not support me in any way

Field 52-56 Employer sponsorship 1-5

Part-time Part-time Other

Field 52 0 0 X

Field 53 1 0 X

Field 54 1 0 X

Field 55 0 0 X

Field 56 0 0 X

Section F Signature and Date• Signed and dated

• Audit trail

• Opportunity to inform leaver about Longitudinal DLHE and for them to opt out of this if they choose to

General guidance

• Ensure that the response is valid– Core questions – Consistent data

• Identify the category of leaver and ensure the appropriate sections have been completed– Part-time– Initial Teacher Training– Research Council funded

• Contact leaver sooner rather than later for confirmation/clarification if required

Common pitfalls

• Salaries – part-time, part-year

• Postcode – incomplete, invalid

• Questions with multiple options– Return each option in a separate field– Use ‘0’ rather than ‘X’ if the option is not ticked

• Part-time section not completed by part-time leavers

• Part-time section completed by full-time leavers

• QTS section completed by leavers who are not newly qualified teachers

Task 2

Completing and coding a

DLHE questionnaire

Online DLHE System

DLHE Online System

• Help to improve response rates by offering an alternative to paper or phone based questionnaires.

• Students can complete it at their own accord and at their own pace

• Intuitive and easy to register and use

DLHE Online System

• 57 Institutions used it last year – its inaugural year

• 2086 students completed questionnaires through this route.

• Free to use!

Online DLHE Questionnairefor both January and April census

Graduates complete form at www.dlhe.ac.uk

or www.dlhe.ac.uk?INST=nnnn

to bypass institution selection

Institutions download data from

submit.hesa.ac.uk

Task 3

Process and managing of the DLHE online system

Cascot

CASCOT (IER)

• CASCOT is designed to assign a code to a piece of text. e.g. a SOC code to a job title from the DLHE survey

• When CASCOT assigns a code to a piece of text it also calculates a score from 1 to 100 which represents the degree of certainty that the given code is correct

• Available to download from: https://submit.hesa.ac.uk/aardvark/CASCOT.ASP

Potential problems

• The quality of coding depends on the quality of the input text:

"advertising & marketing" 

- These are the area of work. Are they an advertising executive, an advert designer or an advertising salesman?

"answering phone enquiries“

- This is a job description, not a title.

Data Collection System

Data processing

• 4,000,000,000 data items per year

• All collection is undertaken via WWW

• Data Collection = Data Quality Assurance

The Aardvark system

• Access codes sent to record contacts

• Create new user accounts

• Add permissions to existing user accounts

Data quality model

Data Information

State1Invalid

State 2Valid

Incredible

State 3Valid

CredibleUnconfirmed

State 4Valid

CredibleConfirmed

Validation CredibilityChecking

Verification

The Aardvark system

• INSERT - a file

• DELETE - a successful insert

• COMMIT - "this is the entire institution"

Registering for the system…

Task 4

Making a DLHE submission on the data collection system

HESA Reports

HESA DLHE Record Reports

• Frequency Counts – key fields

• Check documentation – management stats using standard groupings

Check documentation

• Response rates

• Summary of employment circumstances and further study

• Publication categories

• Summary of ITT leavers

• Employed leavers by location

• Definitions

Other Reports

• Match Reports

• Computer Assisted Structured Coding Tool (CASCOT) Comparison Report

Other Reports

• Performance Indicator Preview – Table E1 Employment indicator

• TQI (Teaching Quality Information) –– Table 5 Destination Information– Table 6 Job categories information– Table 7 Common job types information

Data Collection Cycle

• Most HEIs will have at least two runs through the complete system as a result of queries raised by HESA and themselves

• There is no limit on the amount of times you replace and re-commit your data

HESA Publications and use of data

Publications...Reference volumes

Resources

Destinations

Students

HE UK

Other publications...

• Statistical First Releases (SFR) - National Statistics products:

- DLHE in the United Kingdom for the academic year 2006/07 to be released in July

• Performance Indicators

• Higher Education Information Database for Institutions

• New web-based management information tool• Developed by HESA in conjunction with

University of Leeds and BUFDG• Launched in April 2007

• HESA data plus external sources

• Further details at www.heidi.ac.uk – free trials available on request

Ad-hoc Information Provision Service

• Customised extracts of data

• Various supply formats

• Data supplied under contract

• Delivery timescales

• Charging policy

High profile releases of HESA data

• Policy

• Mechanisms– ‘DDS-Announce’ email group– Data Dissemination System (DDS)

• Feedback

And finally…..

• Useful web pages– Publications information:

www.hesa.ac.uk/products/pubs– Information Provision pages:

www.hesa.ac.uk/products/adhoc

Institutional Liaison Helpdesk

• Telephone 01242 211144

• Email Liaison@hesa.ac.uk