tac surveys of rtos and stakeholders 2015 report of ... · tac rto and stakeholder survey 2015 |...

TAC RTO and Stakeholder Survey 2015 | Produced by Australian Survey Research

TAC surveys of RTOs and stakeholders 2015 Report of overall findings JUNE 2015

Australian Survey Research Pty Ltd Level 1, 600 North Road | PO Box 340 Ormond Victoria 3204

Level 3, 22 Darly Road | PO Box 807 Manly NSW 1655 | T 03 9578 5211 | F 03 9578 5311 | E [email protected] www.aussurveys.com

TAC RTO and Stakeholder Survey 2015 | Produced by Australian Survey Research

Table of contents

Executive summary 1 Introduction 5 Methodology 5

Questionnaire 5 Data collection 5 Data analysis 6 Response and sample profile 6

RTO survey key findings 9 RTO: Regulatory interactions with TAC 9 RTO: TAC’s performance during application interactions 11 RTO: TAC’s website 14 RTO: TAC Update / Special bulletin 16 RTO: TAC general email service 16 RTO: TAC phone service 19 RTO: Registration officer / case management model 26 RTO: Accreditation or information session 26 RTO: TAC speech or presentation 27 RTO: Additional information sessions / workshops from TAC 28 RTO: Preferred method for receiving TAC information 29 RTO: Audits 30 RTO: VET sector improvements 40 RTO: Introduction of new standards 43 RTO: TAC overall 44 RTO: Overall comments 45 RTO differences analysis 48

Stakeholder survey key findings 50 Stakeholder: Interacting with TAC 50 Stakeholder: TAC website 51 Stakeholder: TAC Update / Special bulletin 52 Stakeholder: TAC general email service 52 Stakeholder: TAC phone service 53 Stakeholder: TAC speech or presentation 59 Stakeholder: VET regulation discussion 59 Stakeholder: Regulating the VET sector 60 Stakeholder: Registration officer / case management model 61 Stakeholder: Introduction of new standards in 2015 62 Stakeholder: TAC Overall 63 Stakeholder: TAC overall comments 65 RTO and stakeholder comparison overall 67

Conclusions 68 Attachment 1: Questionnaires 69

Australian Survey Research Group Pty Ltd is accredited under the AS ISO 20252 quality standard applying to market and social research. This research project was carried out in compliance with the AS ISO 20252 quality standard.

TAC RTO and Stakeholder Survey 2015 | Produced by Australian Survey Research | 1

Executive summary

In June 2015, the Training Accreditation Council of Western Australia (TAC) conducted a survey of its registered training organisations (RTOs) and stakeholders. The combined surveys formed part of its commitment to communicate effectively with stakeholders and to gather appropriate information to enhance its regulatory functions.

TAC engaged Australian Survey Research (ASR) to help design, test, deploy and analyse web surveys of 368 RTOs and 70 government, industry, employer and professional association stakeholders.

Data collection

The 2015 RTO and stakeholder questionnaires were very similar to the questionnaires used in the 2014 RTO and stakeholder surveys, with an additional topic in 2015 on the introduction of new RTO standards.

TAC provided ASR with a full listing of all RTOs across Western Australia and the lists included each RTO’s designated contact, their email address and details such as number of qualifications and units. The RTO survey was conducted as a census of TAC’s regulated training organisations. A total of 216 RTOs responded which was a statistically representative sample and which reflected the population profile across a number of attributes.

Further to this, TAC provided ASR with a list of stakeholder names, organisations and contact details. A total of 34 stakeholders responded. This was not a sufficient response to be a statistically representative sample at the 95% confidence level, and no population profiles were available, so results should be treated as indicative only for this group.

Key findings

The RTO questionnaire contained 79 rated items (excluding audit questions which only a sub-set of respondents were presented) all of which used a five-point rating scale. A number of other single and multiple choice questions and open-ended comment questions were included in the questionnaire. Of the 79 rated items, only nine scored below 4.0 (good) on the five-point rating scale—all other items were above this level. The highest scoring items focused on phone and email service while the lower scoring items focused on aspects of making applications and notifications and introduction of the new standards. While these were lower scoring, still around 70% of RTOs who responded were positive.

Apart from the overall high performance in terms of interaction with TAC staff, there were several improvement messages in the RTO feedback and commentary: greater consistency of decisions from TAC audit staff, and at times (not for all) faster responses. The website, an issue in 2014, appears to have improved, or at least attracted less negative commentary in 2015.

The stakeholder questionnaire contained 84 rated items which used a five point scale plus a number of other single and multiple choice questions and open-ended comment questions. Of the rated items, 56 items scored at or above 4.0 (good) on the five-point scale while the remainder between good and fair. As with RTOs, highest scoring items focused on personal contact with TAC, particularly interacting by phone and email. Lower scoring items focused on TAC being transparent, acting on stakeholder feedback and applying consistent regulatory decisions.

As a pattern, stakeholders tended to rate TAC more negatively on comparable items than RTOs. However, this comparison should be treated with caution as the stakeholder sample was very small when compared with the number of RTOs that responded.


Highest and lowest scoring items

The two tables below display the highest and lowest scoring items from the RTO and stakeholder surveys. The items that scored 4.5 or higher are shown in the table below for both surveys. Note that for some of these items, particularly around phone service, the sample sizes were very small and so must be treated with caution.

Mean scores for the RTO survey ranged from a high of 4.5 (between good and excellent on the rating scale) to a low of 3.7 (between good and fair). In other words, all rated items scored on average above the mid-point of the rating scale—a very positive result.

RTOs rated Management, Registration and Accreditation staff highly.

The range of mean scores for the stakeholder survey was from a high of 4.8 (close to excellent on the rating scale) to a low of 3.6 (between good and fair on the rating scale)—so a very similar result to RTOs.

Mean based on rating scale of 5= excellent, 4=good, 3=fair, 2=poor, 1=very poor. Don't know / no answer excluded.

RTO HIGHER SCORING RATED ITEMS

MEAN 2015

MEAN 2014

STAKEHOLDER HIGHER SCORING RATED ITEMS

MEAN 2015

MEAN 2014

Phone Management staff: Knowledge of staff answering 4.5 4.4 Phone Registration staff:

Courtesy 4.8 4.1

Phone Management staff: Courtesy 4.5 4.6

Phone Registration staff Respected confidentiality of org & privacy of individuals involved

4.8 4.4

Email: Respected confidentiality of org & privacy of individuals involved

4.5 4.5 Email: Courtesy 4.7 4.0

Phone Accreditation staff: Courtesy 4.5 4.4 Email: Knowledge of staff

answering 4.7 4.0

Phone Accreditation staff: Respected confidentiality of org & privacy of individuals involved

4.5 4.4

Email: Sufficient contact information so that I could contact / recontact a TAC staff member if necessary

4.7 4.0

Phone Registration staff: Courtesy 4.5 4.6 Phone Registration staff:

Impartiality 4.6 4.2

Phone Registration staff: Respected confidentiality of org & privacy of individuals involved

4.5 4.6 Phone Admin staff: Courtesy 4.6 4.3

Other phone staff: Courtesy 4.5 4.4 Phone Management staff: Courtesy 4.6 4.0

Phone Other staff: Courtesy 4.5 4.3

Phone Management staff: Respected confidentiality of organisation and privacy of individuals involved

4.5 4.2


The nine lowest scoring items are displayed below for both surveys. Note that these items still had relatively high scores but they were where more people rated TAC as poor or very poor.

For RTOs these items tended to focus on making applications and notifications using RTONet and the introduction of the new standards. For stakeholders the items focused on working with stakeholders, overall outcomes in the VET sector and introduction of the new standards.

Mean based on rating scale of 5= excellent, 4=good, 3=fair, 2=poor, 1=very poor. Don't know / no answer excluded. NA means not asked in 2014.

RTO LOWER SCORING RATED ITEMS

MEAN 2015

MEAN 2014

STAKEHOLDER LOWER SCORING RATED ITEMS

MEAN 2015

MEAN 2014

Website: Provides enough information

3.9 3.9

TAC overall: Collaborates with industry bodies, other industry regulators and peak associations

3.8 3.4

Applns and Notifications: Clarity of TAC’s application form

3.9 3.9 Website: Easy to search 3.8 3.6

New standards in 2015: Provided enough information to my organisation about the new standards

3.9 Not asked

TAC overall: Provides timely and quality advice to my organisation on its regulatory activities

3.8 3.4

New standards: Provided clear information on its website about the new standards

3.9 Not asked

New standards: Provided timely advice to my org about the new standards

3.8 Not asked

New standards: Provided timely advice to my org about the new standards

3.9 Not asked Website: Easy to navigate 3.8 3.6

Applns & Notifications: Helpfulness of information on TAC’s website regarding making applications

3.9 3.9 TAC overall: Overall as a regulator 3.8 3.6

Applns & Notifications: Ease of navigating RTONet

3.8 3.8 TAC overall: Seeks feedback from stakeholders 3.7 3.6

Applns & Notifications: Ease of completing the task required using RTONet

3.8 3.7 TAC overall: Improves the quality of VET outcomes in Western Australia

3.7 3.6

Applns & Notifications: Clarity of instructions in RTONet

3.7 3.7

New standards: Provided adequate support to my org if we had questions / needed assistance

3.7 Not asked

TAC overall: Is transparent in its regulatory decisions and activities

3.7 3.5

TAC overall: Acts on stakeholder feedback 3.7 3.3

TAC overall: Applies consistent regulatory decisions

3.6 3.5


Year comparison

2015 results were slightly better than 2014, except in relation to audits, where all scores on all items declined. However, most of these year changes were not statistically significantly different, meaning that these differences were not enough to be statistically accurate. The exception is around the TAC website where three of the six aspects rated improved significantly.

Open ended comments

Respondents offered many fewer comments in 2015 compared with 2014, and there were very few outstanding or repeated themes across questions, except for greater consistency across TAC audit staff. There was very little commentary about TAC’s website compared with the previous survey, so it appears that something has improved in that area.

RTO respondents indicated that TAC’s current method of case management was working well.

In summary

Overall TAC was seen as an effective regulator with positive client interactions and this has not changed materially in the last 12 months. In other words, overall TAC has maintained effective interactions with its stakeholders across many channels. Many RTOs appreciate the case management model. It appears that for most, the introduction of new standards went smoothly.

Activities that have room for improvement are:

RTONet Consistency across TAC auditors Auditor interactions with RTOs being audited Continued focus on engagement with stakeholders.


Introduction

In June 2015, the Training Accreditation Council of Western Australia (TAC) conducted surveys of its registered training organisations (RTOs) and stakeholders. These surveys form a key as part of TAC’s commitment to communicate effectively with stakeholders and to gather appropriate information to enhance its regulatory functions.

TAC uses feedback collected from the surveys to inform its review and continuous improvement processes and as an aid in benchmarking with other regulators. The surveys also support TAC’s ongoing commitment to transparency and to assist in communicating with internal and external stakeholders.

To help ensure objective, confidential and anonymous feedback and professional analysis, TAC engaged Australian Survey Research (ASR) to deploy and analyse web surveys of all the RTOs and key industry stakeholders within its jurisdiction including government departments, industry councils and employer representatives.

This report outlines the methodology used to conduct and analyse the two surveys as well as the key findings from both surveys. Detailed questions used in the surveys appear in an attachment to the report.

Key findings from the RTO survey are outlined first and then followed by findings from the stakeholder survey. Year comparisons are provided throughout the report.

Methodology

This section outlines how the two surveys were developed and tested, how survey participants were identified and how the survey was administered and analysed.

Questionnaire To enable a 2014/2015 year comparison, the 2015 questionnaires were closely based on those used in 2014. Some questions around the introduction of new RTO standards in 2015 were added.

Both the RTO and stakeholder web questionnaires were loaded into ASR’s proprietary web surveying tool, SurveyManager, and hosted on ASR’s internet servers located in a high security data centre in Melbourne’s CBD.

Data collection TAC provided ASR with a full listing of all RTOs (n=368) within its jurisdiction and the list included each RTO’s designated contact, their email address and details such number of complete qualifications and number of units offered. In addition, the file indicated which RTOs had been audited in 2014/15 and these respondents were presented a special set of questions around audits. Effectively, the RTO survey was conducted as a census of TAC’s regulated training organisations.

Further to this, TAC provided ASR with a list of stakeholder names, organisations and contact details (n=70). The following types of stakeholders were invited to give feedback:

Various state and federal government departments

Industry associations

Industry skills councils

Employer peak bodies


Employee associations, and

Professional associations.

Prior to going live with the full survey, TAC Chairman, Ian Hill, emailed a letter to the CEOs of all RTOs and to all stakeholders, advising them of the survey. Soon after, ASR sent an email invitation to a contact within each RTO and to each stakeholder. The email invitation contained a unique organisational hyperlink to access the web questionnaire.

ASR tracked the response rate and sent two targeted reminder emails to all non-responders in both surveys. When answering, respondents were asked to focus on TAC’s performance since the end of June 2014 to the time of completing the survey. Both questionnaires were in field from 18 May to 18 June 2015.

Data analysis Results were analysed to produce mean scores (averages) and frequency distributions. ANOVA was used to determine any statistical differences between two demographic sub-groups (RTO number of qualifications and RTO number of students) as well as year differences. All tests are reported at the p<0.05 level (95% confidence level). See the box on next page for further explanation.

Means were calculated using only the number of respondents who chose a rating point answer. In other words don’t know, not applicable and no answers (blank) were excluded from statistical calculations.

The rating scale used to assess most items is displayed in the table below. It is important to understand what the numbers represent because results subsequently described in the report are presented in numeric form only. For example, a mean (average) score of 4.0 indicates that, overall, respondents agreed that TAC was performing at a good level on a particular item. The questions in some topics were asked using a different rating scale. Where a survey item was presented with a different scale to the one shown below, it is noted and explained in the report.

RATING SCALE DESCRIPTION

ASSIGNED NUMERIC VALUE

Excellent 5

Good 4

Fair 3

Poor 2

Very poor 1

In addition to mean scores, which are a weighted summary of rating results, top 2 scores are displayed. Top 2 refers to the two most positive rating points in any scale. The percentage of the top 2 score is the sum of people answering with the two most positive rating points. In calculating the top 2 scores, don’t know, not applicable and no answers have been excluded. Means and top 2 scores are not better or worse than each other when measuring sentiment, simply different. Top 2 scores focus on the proportion of the response sample that are positive or in agreement.

Response and sample profile A total of 216 RTOs responded to the survey yielding a response rate of 59%. The sample is statistically representative of the RTO population at the 95% confidence level and the ±4% confidence interval (see box on next page explaining confidence interval and level). This is lower (more rigorous and therefore better) than an acceptable scientific confidence interval, normally ±5%, and the market research acceptable confidence interval of ±10%.


A total of 34 stakeholders answered the survey achieving a response rate of 49%. The results for the stakeholder survey have a confidence interval of ±12% and therefore should be treated as fairly indicative only.

Representativeness of a sample is often assessed at a 95% confidence level (accuracy) and a ±5% confidence interval (precision).

The confidence interval (also called margin of error) is the plus-or-minus figure usually reported in newspaper or television opinion poll results. For example, if you use a confidence interval of 4 and 47% percent of your sample picks an answer you can be "sure" that if you had asked the question of the entire relevant population between 43% (47-4) and 51% (47+4) would have picked that answer.

The confidence level tells you how sure you can be. It is expressed as a percentage and represents how often the true percentage of the population who would pick an answer lies within the confidence interval. The 95% confidence level means you can be 95% certain; the 99% confidence level means you can be 99% certain. Most researchers use the 95% confidence level.

When you put the confidence level and the confidence interval together, you can say that you are 95% sure that the true percentage of the population is between 43% and 51%. The wider the confidence interval you are willing to accept, the more certain you can be that the whole population answers would be within that range.

For example, if you asked a sample of 1000 people in a city which TV channel they preferred watching, and 60% said Channel A, you can be very certain that between 40 and 80% of all the people in the city actually do prefer that channel, but you cannot be so sure that between 59 and 61% of the people in the city prefer the channel.

Reference: www.surveysystem.com/sscalc.htm

The RTO population and sample profiles have been compared by number of qualifications to identify if any sub-groups were over/under-represented in the response set. The response profiles in the RTO survey sample have fairly similar proportions to that of the population (see table immediately below). As a result, no weighting has been applied to the RTO sample. No equivalent information was available for stakeholders.

The RTO response sample and population profiles are displayed in the following tables. Charts of the table data follow the tables.

NUMBER OF QUALIFICATIONS

RTO POPULATION

RESPONSE SAMPLE

CONFIDENCE INTERVAL

(at row level)

Freq % Freq % %

0 qualifications 116 31.5 52 24.1 10.1

1-6 qualifications 137 37.2 79 36.6 7.2

7+ qualifications 115 31.3 85 39.4 5.5

Total 368 100.0 216 100.0 4.3

The table below is for information only and shows that there is a spread of size within the survey response sample, meaning that quite different types of RTOs responded. No population information was available around this demographic (RTO attribute) so population comparisons could not be made.


NUMBER OF RTO STUDENTS RESPONSE SAMPLE

Freq %

More than 500 52 24.1

201-500 42 19.4

51-200 52 24.1

1-50 52 24.1

None 18 8.3

Total 216 100.0

31.5

37.2

31.3

24.1

36.6

39.4

0 5 10 15 20 25 30 35 40 45

0 qualifications

1‐6 qualifications

7+ qualifications

RTO: Number of qualifications% of respondents in each category

Response sample % n=216 Population % n=368

24.1

19.4

24.1

24.1

8.3

0 5 10 15 20 25 30

More than 500

201‐500

51‐200

1‐50

None

RTO: Number of students% of respondents n=216


RTO survey key findings

This section outlines the key findings from the RTO survey. For most items, mean and top 2 scores are presented along with a frequency distribution. Results are presented by topic, in the same order as presented to respondents in the web questionnaire. The differences between years and the demographic sub-groups of number of qualifications and number of students are then presented and discussed. The most common themes within free text comments are covered at the end of each topic.

In all but three of the 79 rated items presented to most respondents, 70% or more of respondents rated TAC positively choosing good /

excellent or agree / strongly agree as their answers.

RTO: Regulatory interactions with TAC

The chart below displays the types of regulatory interactions RTOs had with TAC in the last 12 months. The four most common types of interactions were using the TAC website receiving a TAC Update, having a phone conversation with TAC staff, and using RTONet. These were also the top four interactions in the previous survey.

Three-quarters of respondents had five or more interactions with TAC in the previous 12 months—refer to table below.

NUMBER OF INTERACTIONS

Number of respondents Percent

10 1 0.5

9 8 3.7

8 15 6.9

7 48 22.2

6 54 25

5 34 15.7

4 25 11.6

3 13 6

2 11 5.1

1 2 0.9

0 5 2.3

Total 216 100.0

The results from this question were used to drive the presentation of other questions within the questionnaire. Only those who indicated having a specific interaction, like using RTONet, were asked to rate it. Note that a very small percentage of RTOs (around 2%) indicated having no regulatory contact with TAC in the previous 12 months. The majority of Other responses was undertaking an audit.


2.3

8.8

25.9

34.7

47.2

47.2

55.6

80.1

80.6

84.7

88.4

0 20 40 60 80 100

No contact

Other

Used the TAC general email address / enquiry service

Attended an event, other than an RTO or accreditation information session at which a TAC

staff member spoke

Made an application to TAC

Attended a TAC RTO or accreditation information session

Notified TAC of change to organisation or scope

Used RTONet

Had a phone conversation with TAC staff

Received a TAC Update and/or TAC Special Bulletin (email newsletter)

Used the TAC website

RTO: Types of interactions with TAC in 2014/15Multiple answers allowed so total >100%; % based on n=216


Respondents who made an application or notification to TAC (n=194) were then asked to specify the type of interaction/s their application or notification related to. Typically, respondents were amending registration applications, changing RTO details or renewing their registrations. This is an identical pattern to the 2014 survey. The Other category covered varied applications or notifications including extension of some type, audits, cancelling / ceasing an RTO and paying registration fees.

RTO: TAC’s performance during application interactions This section outlines RTOs’ views of the interactions they had with TAC based on the applications and/or notifications they submitted. The following series of tables and charts display the 2015 mean and top 2 scores and the 2014 mean score, along with frequency distributions describing various aspects of these interactions. Tables with mean scores display total (n) counts for rated items. Specific questions were shown to specific groups of respondents so total counts varied throughout the survey.

Timeliness of response, ease of accessing RTONet and availability of follow up assistance are strengths for TAC. Refer to table below. Items relating to usability of RTONet were rated lowest and this was supported by open ended comments. Note that while these were relatively lower scoring items, a large majority of respondents rated the items positively as shown in the chart below.

2.1

2.6

3.6

4.6

6.2

6.2

7.2

9.3

11.9

12.4

35.6

37.1

44.8

0 20 40 60

Request for intent to transfer

Total VET activity exemption

Accreditation / application for extension to course

Change of legal entity

Initial registration application

Accreditation initial course accreditation

Accreditation / application for amendment to course

Accreditation / reaccreditation of a course

Removal of organisation's scope of registration and delivery

Other

Renewal of registration application

Change of RTO details

Amendment to registration application

RTO: Types of applications and notifications in 2014/15Multiple answers allowed so total >100%; % based on n=194


There were no statistically significant differences between 2014 and 2015 survey results for this topic.

Mean based on rating scale of 1=very poor, 2=poor, 3=fair, 4=good, 5= excellent. Don't know / no answer excluded.

RTO: APPLICATIONS AND NOTIFICATIONS n Mean

2015 Mean 2014

Top 2% 2015

Time TAC took to act on the application after you were notified that the application was received

190 4.1 4.1 80.7

Ease of accessing RTONet 192 4.0 3.9 77.7

Availability of follow up assistance 193 4.0 4.0 80.2

Helpfulness of information on TAC’s website regarding making applications

136 3.9 3.9 74.6

Clarity of TAC’s application form 141 3.9 3.9 80.6

Ease of navigating RTONet 191 3.8 3.8 69.9

Ease of completing the task required using RTONet

191 3.8 3.7 71.7

Clarity of instructions in RTONet 190 3.7 3.7 69.0

The following charts show the frequency distribution of answers (proportion of respondents choosing a particular answer) for the topic of applications and notifications.

When reading the charts it is important to note that each chart has been sorted by the proportion of positive responses and is presented in descending order. Total counts have been reported for each item if counts vary.

When reading these types of charts within this report, it is useful to look at the proportion of green (positive) and the proportion of orange/red

(negative) responses. For the following chart it is easy to see that positive ratings far outweigh negative ratings on all items but there is still room for

improvement in some areas.

Also note that the percentage of green (dark and light green) may not always be equivalent to the top 2 score as presented in tables. This is

because the chart percentages include don’t know and no answer counts in their calculation whereas the top 2 score excludes these counts.


RTO: Applications and notifications open ended comments

Respondents who had rated an aspect of applications and notifications as fair, poor or very poor were given the opportunity to explain their answer. A total of 90 respondents provided explanations. See the table below for common themes. Most comments relate to the RTONet, its navigation and functionality. Next most common were comments about the website, its functionality and information contained in it.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Clarity of instructions in RTONet n=190

Ease of navigating RTONet n=191

Ease of completing the task required using RTONet n=191

Helpfulness of information on TAC’s website regarding making applications n=136

Ease of accessing RTONet n=192

Availability of follow up assistance n=193

Clarity of TAC’s application form n=141

Time TAC took to act on the application after you were notified that the application

was received n=190

Clarity of instructions in RTONet

n=190

Ease of navigating RTONet n=191

Ease of completing

the task required

using RTONet n=191

Helpfulness of

information on TAC’s website

regarding making

applications n=136

Ease of accessing RTONet n=192

Availability of follow up assistance

n=193

Clarity of TAC’s

application form n=141

Time TAC took to act

on the application after you

were notified that the

application was

received n=190

Excellent 13.2 17.3 16.8 17.6 25.5 28.0 17.0 27.9

Good 53.7 50.8 52.4 53.7 50.5 45.6 59.6 44.7

Fair 21.6 23.0 19.4 19.1 18.8 13.5 14.9 13.7

Poor 6.3 4.2 5.2 3.7 1.6 3.1 2.1 2.6

Very poor 2.1 2.1 2.6 1.5 1.6 1.6 1.4 1.1

Don't know 3.2 2.6 3.7 4.4 2.1 8.3 5.0 10.0

RTO: TAC applications and notifications% of respondents choosing a rating point; n varies


While the above questions were about RTONet, many respondents probably do not experience any difference between RTONet and the website—they are all the same medium or channel for many.

There is also a theme about slow response time from TAC. While this appears to be contradictory to the chart results above which show a strongly positive result, it probably represents the proportion of people who have answered fair, poor or very poor to the time TAC took to act item. This may indicate that TAC’s actions are not always consistent: most of the time a process works well, but every now and then it doesn’t.

RTO: APPLICATIONS / NOTIFICATIONS COMMENT THEMES n=90 FREQ

RTONet: negative, difficult to navigate, log in, upload, not user friendly 52

Website: negative, difficult to navigate, search, information unclear or out of date 13

Timeliness of response poor 9

Staff and website do not provide specific guidance and support 8

No accountability if TAC has done the wrong thing 5

Should be provided with an update / notification of where your query / audit / request is at even if an answer has not been reached 4

One size does not fit all 3

Website: Not all information is available and follow up phone call is often necessary 3

TAC staff changes are disruptive 2

Reporting requirements should be clearer 2

Other 3

RTO: TAC’s website All items in this topic scored well and three significantly improved since 2014. Given that the website is used by more than 85% of users, it is important that this service delivery channel works well.

Mean based on rating scale of 1=very poor, 2=poor, 3=fair, 4=good, 5= excellent. Don't know / no answer excluded. Yellow highlight indicates statistically significant difference.

RTO: TAC WEBSITE n = 190 Mean 2015

Mean 2014

Top 2% 2015

Accurate information 4.2 4.1 93.1

Current/up to date information 4.1 4.0 89.9

Easy to understand information 4.1 4.0 87.3

Easy to navigate 4.0 3.8 82.6

Easy to search 4.0 3.8 81.3

Provides enough information 3.9 3.9 80.2


RTO: Website open-ended comments

Respondents who had more negative ratings of the website were asked to explain their reasons and 42 did so. Themes that were repeated more than once included the following:

Navigation difficult / not user friendly

Hard to search / no search functionality

Out of date / inaccurate information

Uploading was difficult

Information unclear / vague and not enough

Poor layout

New standards not up to date on website / little information

More FAQS / examples / templates wanted.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Provides enough information

Easy to search

Easy to navigate

Easy to understand information

Current/up to date information

Accurate information

Provides enough information Easy to search Easy to navigate




Excellent 20.2 19.6 20.5 25.4 25.3 30.5

Good 59.6 60.8 62.1 61.9 64.2 61.6

Fair 14.4 15.3 14.2 11.6 8.4 6.3

Poor 4.3 2.6 2.6 0.5 1.6 0.5

Very poor 1.1 0.5 0.5 0.5 0.0 0.0

Don't know 0.5 1.1 0.0 0.0 0.5 1.1

RTO: TAC website% of respondents choosing a rating point n=190


RTO: TAC Update / Special bulletin The TAC Update / Special bulletin received strongly positive ratings as shown in the table and chart below. No items were statistically different compared with 2014.


RTO: TAC UPDATE / SPECIAL BULLETIN n=183 Mean 2015

Mean 2014

Top 2% 2015

Helpful information 4.3 4.2 95.1

Informs on a wide range of issues 4.2 4.1 92.9

Timely information 4.2 4.1 90.6

The frequency distribution chart below shows the strong positive response for this channel. Those who rated any aspect of the channel negatively offered comments around information being vague and not timely; that information had been received through other sources.

RTO: TAC general email service Respondents were asked to indicate how many times they had used the general email service—[email protected]—in the last 12 months. Most commonly, respondents indicated using the service 1-5 times (57%). See chart below.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Timeliness of information

Informs on a wide range of issues

Helpfulness of information

Timeliness of information Informs on a wide range of issues Helpfulness of informationExcellent 26.9 24.7 32.8

Good 63.2 68.1 62.3

Fair 7.7 4.9 4.9

Poor 1.6 2.2 0.0

Very poor 0.0 0.0 0.0

Don't know 0.5 0.0 0.0

RTO: TAC Update / Special bulletin% of respondents choosing a rating point n=183


The table below shows that all items for the email service scored very well. There were no statistical differences between 2015 and 2014.


RTO: TAC GENERAL EMAIL SERVICE n=56 Mean 2015

Mean 2014

Top 2% 2015

Sufficient contact information so that I could contact / recontact a TAC staff member 4.4 4.3 98.1

Respected confidentiality of org & privacy of individuals involved 4.5 4.5 97.9

Impartiality 4.4 4.3 97.9

Courtesy 4.4 4.4 96.2

Helpfulness 4.3 4.4 94.4

Efficiency of response 4.3 4.3 94.3

Clarity of response 4.2 4.3 92.6

Promptness (speed of reply) 4.2 4.3 92.5

Knowledge of staff answering 4.2 4.3 88.7

7.1

12.5

32.1

25.0

14.3

8.9

0 20 40

No answer

Don't know / not sure

1‐2 times

3‐5 times

6‐10 times

More than 10 times

RTO: Times TAC general email service used in 2014/15% of respondents, n=56


The frequency distribution chart below shows the strong positive response for this channel. Only eight people offered comments around poorer performance and a more common theme was around responses not being timely.

There was a considerable number of don’t know answers around impartiality and respecting confidentiality. This may be because these issues did not arise in the interaction and therefore the respondent did not have sufficient information to make an assessment of this dimension.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Knowledge of staff answering

Promptness (speed of reply)

Clarity of response

Efficiency of response

Helpfulness

Courtesy

Respected confidentiality of org & privacy of individuals involved

Impartiality

Sufficient contact information to recontact TAC staff member

Knowledge of staff

answering

Promptness (speed of

reply)

Clarity of response


Helpfulness Courtesy

Respected confidentialit

y of org & privacy of individuals involved

Impartiality

Sufficient contact

information to recontact TAC staff member

Excellent 28.6 27.3 32.1 35.7 39.3 44.6 43.6 38.9 37.5

Good 55.4 61.8 57.1 53.6 51.8 46.4 40.0 46.3 57.1

Fair 10.7 5.5 3.6 3.6 1.8 1.8 1.8 1.9 1.8

Poor 0.0 1.8 3.6 1.8 3.6 1.8 0.0 0.0 0.0

Very poor 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Don't know 5.4 3.6 3.6 5.4 3.6 5.4 14.5 13.0 3.6

RTO: TAC email service% of respondents choosing a rating point, n=56


RTO: TAC phone service Most commonly (41%), RTOs used TAC’s phone service 3-5 times in last 12 months and around 36% used the service six times or more.

Respondents were asked to indicate which TAC area they had contacted and were then asked to rate each one. The table below demonstrates that the majority of respondents interacted with registration staff (69%) in the last 12 months (similar to the previous survey) and which is to be expected given the nature of the respondent group.

The table below provides a comparison of mean scores across all the different TAC areas phoned in the last 12 months. Note that a respondent could have phoned multiple areas of TAC and, if so, they were asked about each area.

There was little variation between scores. All items across all areas of TAC that were phoned rated well above 4.0 (good) on the rating scale. The only area where scores changed statistically significantly between survey periods was for Registration where there was an overall decline. However, this must be read in light of the very high scores for both surveying periods.

3.4

19.5

40.8

23.0

13.2

0 20 40 60


1‐2 times

3‐5 times

6‐10 times

More than 10 times

RTO: Number of times TAC phone service was used in 2014/15% of respondents, n=174

6.3

13.2

13.2

27.0

38.5

69.0

0 20 40 60 80


Management

Other (Complaints, Policy, etc)

Accreditation

Administration

Registration

RTO: Areas of TAC phoned in 2014/15Multiple answers allowed so total may be >100%; % based on n=174


These results are positive and show consistent behaviour across TAC for this channel.


RTO: TAC PHONE SERVICE

Accred’n n=47

Regist’n n=120

Admin n=67

Mgt n=23 Other n=23

2015 2014 2015 2014 2015 2014 2015 2014 2015 2014

Clarity of response 4.2 4.2 4.2 4.5 4.3 4.5 4.3 4.2 4.3 4.4

Courtesy 4.5 4.4 4.5 4.6 4.4 4.5 4.5 4.6 4.5 4.4

Efficiency of response 4.2 4.3 4.3 4.5 4.3 4.5 4.1 4.5 4.3 4.4

Helpfulness 4.3 4.2 4.4 4.5 4.3 4.5 4.4 4.3 4.3 4.4

Impartiality 4.4 4.3 4.4 4.5 4.3 4.5 4.4 4.6 4.4 4.4

Knowledge of staff answering 4.3 4.2 4.3 4.5 4.3 4.5 4.5 4.4 4.3 4.4


4.5 4.4 4.5 4.6 4.4 4.5 4.4 4.6 4.4 4.4

Sufficient information to recontact TAC staff member 4.3 4.3 4.4 4.5 4.4 4.5 4.4 4.4 4.4 4.4

There are very few negative responses in any area. The key themes of comments about poorer performance in each phone area were:

Accreditation: knowledge / expertise was poor and response could have been faster

Registration: need to be more helpful, respectful and faster

Administration: very small number of comments with no single theme

Management: very small number of comments and two were about audit issues

Other: only one comment about not getting a definite answer.

The following series of charts display the frequency distributions for each area of TAC’s phone service.


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Clarity of response

Helpfulness


Sufficient contact information to recontact a TAC staff member


Impartiality


Courtesy

Clarity of response

Helpfulness Efficiency of response

Sufficient contact

information to recontact a TAC staff

member

Knowledge of staff

answeringImpartiality



Courtesy

Excellent 31.9 46.8 31.9 40.4 39.1 46.8 47.8 46.8

Good 55.3 42.6 57.4 48.9 50.0 44.7 45.7 51.1

Fair 8.5 6.4 8.5 8.5 4.3 4.3 2.2 0.0

Poor 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Very poor 2.1 2.1 0.0 0.0 2.2 0.0 0.0 0.0

Don't know 2.1 2.1 2.1 2.1 4.3 4.3 4.3 2.1

RTO: Phone Accreditation staff% of respondents choosing a rating point, n=47


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Clarity of response


Helpfulness

Courtesy


Impartiality



Clarity of response

Knowledge of staff

answeringHelpfulness Courtesy Efficiency of

responseImpartiality

Sufficient contact


member



Excellent 37.8 39.2 47.0 53.3 39.2 43.3 42.9 47.5

Good 49.6 50.8 45.3 41.7 55.8 48.3 52.1 44.1

Fair 10.9 8.3 6.0 4.2 4.2 2.5 4.2 1.7

Poor 0.8 0.0 0.9 0.8 0.8 1.7 0.0 0.8

Very poor 0.8 0.8 0.9 0.0 0.0 0.0 0.0 0.0

Don't know 0.0 0.8 0.0 0.0 0.0 4.2 0.8 5.9

RTO: Phone registration staff% of respondents, n=120


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Clarity of response


Courtesy

Helpfulness



Impartiality


Clarity of response

Knowledge of staff

answeringCourtesy Helpfulness Efficiency of

response



Impartiality

Sufficient contact


member Excellent 37.3 35.8 40.3 38.8 35.8 37.3 33.3 38.8

Good 56.7 58.2 55.2 56.7 59.7 58.2 63.6 59.7

Fair 3.0 3.0 4.5 4.5 3.0 1.5 1.5 1.5

Poor 1.5 1.5 0.0 0.0 0.0 0.0 0.0 0.0

Very poor 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Don't know 1.5 1.5 0.0 0.0 1.5 3.0 1.5 0.0

RTO: Phone administration staff% of respondents, n=67


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%


Clarity of response


Impartiality

Courtesy

Helpfulness




Clarity of response



Impartiality Courtesy Helpfulness

Sufficient contact


member

Knowledge of staff

answering

Excellent 39.1 43.5 56.5 52.2 60.9 50.0 45.5 52.2

Good 39.1 39.1 30.4 34.8 30.4 45.5 50.0 47.8

Fair 13.0 17.4 4.3 4.3 8.7 0.0 4.5 0.0

Poor 0.0 0.0 0.0 0.0 0.0 4.5 0.0 0.0

Very poor 4.3 0.0 4.3 4.3 0.0 0.0 0.0 0.0

Don't know 4.3 0.0 4.3 4.3 0.0 0.0 0.0 0.0

RTO: Phone management staff% of respondents, n=23


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Clarity of response

Efficiency of response (took minimal amount of time, including waiting time)


Helpfulness

Courtesy

Respected confidentiality of organisation and privacy of individuals involved

Impartiality


Clarity of response


(took minimal

amount of time,

including waiting time)

Knowledge of staff

answeringHelpfulness Courtesy


y of organisation and privacy

of individuals involved

Impartiality

Sufficient contact


member

Excellent 30.4 30.4 30.4 30.4 39.1 30.4 30.4 30.4

Good 47.8 47.8 43.5 52.2 47.8 47.8 52.2 56.5

Fair 8.7 8.7 4.3 4.3 0.0 0.0 0.0 0.0

Poor 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Very poor 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Don't know 13.0 13.0 21.7 13.0 13.0 21.7 17.4 13.0

RTO: Phone other staff% of respondents, n=23 unlsess stated otherwise


RTO: Registration officer / case management model Respondents were asked to provide comments about TAC’s current registration / case management model and 85 respondents did so. Overwhelmingly respondents indicated their support of the model and did not want to change to another management model. The most common comment in 2015 was also the most common comment in 2014, so very little has changed in the intervening 12 months.

RTO: REGISTRATION OFFICER / CASE MANAGER COMMENT THEMES n=85 FREQ

Positive: Case manager has knowledge of individual situations / developed relationship / good relationship / don’t change 71

Good model but when CM unavailable hard to talk to anybody else 5

Needs to be consistency of decisions across RTOs and case managers 4

Higher level of service expected 4

Advice too general / FAQS would be helpful 3

Other 2

RTO: Accreditation or information session All items in this topic rated positively.


RTO: TAC RTO ACCRED / INFO SESSION n=102 Mean 2015

Mean 2014

Top 2% 2015

Timely (not outdated) information 4.4 4.2 92.2

Useful information 4.3 4.2 91.2

Insight that considered a wide range of issues 4.2 4.1 89.0

The handful of open-ended comments about poorer performance covered a number of areas included sessions being superficial or vague / unclear. The chart below displays a large majority of positive responses for this topic.


RTO: TAC speech or presentation This topic focused on speeches or presentations other than RTO accreditation or information sessions. All items in this topic rated positively.


RTO: TAC SPEECH OR PRESENTATION n=75 Mean 2015

Mean 2013

2015 Top 2%



Insight that considered a wide range of issues 4.3 4 95.6

The very small number of comments about poorer performance were about information being too generalised or not clear / vague.

The frequency distribution chart demonstrates the strongly positive response.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Insight that considered a wide range of issues

Useful information

Timely (not outdated) information


Useful information Timely (not outdated) information

Excellent 36.6 41.2 48.0

Good 51.5 50.0 44.1

Fair 8.9 4.9 5.9

Poor 2.0 3.9 2.0

Very poor 0.0 0.0 0.0

Don't know 1.0 0.0 0.0

RTO: Accreditation / information sessions% of rsepondents, n=102


RTO: Additional information sessions / workshops from TAC Respondents were asked to indicate if they would be interested in attending additional TAC information sessions or workshops on specific regulatory matters for RTOs. The chart below shows that most respondents (over 90%) were interested in the idea, with 30% of these conditionally interested.

Respondents were asked to suggest possible topics for these additional sessions / workshops and 151 did so. The table below displays the topics mentioned by more than one person. Information around compliance and audits was the overwhelming favourite.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Useful information



Useful information Insight that considered a wide range of issues


Excellent 41.9 35.1 42.7

Good 51.4 52.7 52.0

Fair 5.4 4.1 2.7

Poor 0.0 0.0 0.0

Very poor 0.0 0.0 0.0

Don't know 1.4 8.1 2.7

RTO: Other speech or presentations% of respondents, n=75

9.7

30.1

60.2

0.0 20.0 40.0 60.0 80.0

No

Depends (on topic / timing)

Yes

RTO: Interest in attending additional information sessions or workshops

% of respondents, n=216


RTO: SUGGESTED TOPIC FOR ADDITIONAL TRAINING n=151 FREQ

Compliance and audits especially validation 63

New standards 31

Updates on changes / to regulatory requirements 13

Templates of requirements / examples of best practice 13

Reporting requirements 9

Any topic 5

Vocational competencies / trainer competencies 5

Location of training, southwest 4

AQTF and cross over with other federal bodies 3

Registration / re registration 3

Tailored sessions depending on size or industry of RTO 2

Timing of training 2

Traineeship / apprenticeship data 2

Transitions 2

RTO: Preferred method for receiving TAC information

When asked about the preferred method for receiving general advice from TAC about the VET sector, the majority of respondents indicated that TAC Update was most preferred (67%), fairly closely followed by email. This is an identical result to the 2014 survey.

In the Other category, three people suggested a webinar / video.

0.5

2.8

6.5

48.1

63.6

67.3

0 20 40 60 80

None of the above

Other

Hard copy letter / sheet

Information sessions / workshops

Simple broadcast email

TAC Update

RTO: Preferred method for receiving general advice from TACMultiple answers allowed so total may be >100%; % based on n=214


RTO: Audits TAC provided ASR with a list of respondents who had been audited in the previous 12 months and only these respondents were asked a series of questions about their audit experience. The chart below shows the types of RTO audits that respondents believed they had participated in. Most had undergone a site audit (47%).

All audit ratings

The table below displays the mean scores for all audit types on six aspects. Note that the rating scale used in all audit questions was a five point scale of strongly agree, agree, part agree /part disagree, disagree and strongly disagree (as distinct from the excellent to very poor performance scale used in most other questions).

Overall, all audits achieved very similar ratings and all were slightly lower than 2014. However, only three scores were significantly lower in 2015 and these were all for Strategic Industry Audits. These are fairly positive results for any organisation performing an auditing role.

Mean based on rating scale of 1=strongly disagree, 2=disagree, 3=part agree/part disagree, 4=Agree, 5= strongly agree. Don't know / no answer excluded. Yellow highlight indicates statistically significant difference.

RTO: AUDIT ASPECT MEAN SCORES

COURSE ACCRED / REACCRED

n=27

SIA n=16 DESK n=21 SITE n=37

2015 2014 2015 2014 2015 2014 2015 2014

TAC informed my org before & during the audit about the audit purpose & process

4.3 4.5. 4.1 4.3 4.2 4.4 4.3 4.5

TAC informed my org about the scope of the audit 4.4 4.5 4.1 4.3 4.2 4.3 4.2 4.5

TAC informed us about why the audit sample was chosen

Not asked

Not asked 3.8 4.3 3.9 4.3 4.2 4.4

TAC informed my org of any changes to the audit timetable and reasons for changes

4.1 4.5 4.1 4.3 4.2 4.3 4.2 4.5

2.5

20.3

26.6

34.2

46.8

0 20 40 60

Don't know

Strategic Industry Audit (SIA)

Desk audit

Course accreditation or re‐accreditation

Site audit

RTO: Types of audits participated in during 2014/15Multiple answers allowed so total >100%; % based on n=79


RTO: AUDIT ASPECT MEAN SCORES


n=27


2015 2014 2015 2014 2015 2014 2015 2014

TAC audit staff gave my org an opportunity to discuss our particular business situation / needs

4.1 4.4 4.0 4.1 4.1 4.2 4.2 4.3

TAC audit staff were willing to discuss any issues regarding the audit findings at the time of the audit

4.0 4.5 3.9 4.3 3.9 4.3 4.1 4.4

The following table displays mean scores for rated items about audit staff. Again results were all around the good rating point and very similar. They are slightly lower than in 2014, but only one item was significantly lower—SIA audit staff being informative. However, note the small sample for SIA audits so that result should be treated with caution.


RTO: AUDIT TEAM MEAN SCORES


n=27


2015 2014 2015 2014 2015 2014 2015 2014

Objective 3.9 4.5 3.7 4.3 4.3 4.3 4.1 4.3

Knowledgeable 4.2 4.5 3.7 4.3 4.2 4.3 4.3 4.3

Organised 4.3 4.5 3.9 3.7 4.2 4.4 4.3 4.5

Informative 4.0 4.4 3.6 4.3 4.1 4.3 4.1 4.3

Constructive 3.9 4.4 3.4 4.1 4.0 4.2 4.0 4.3

Nonthreatening 4.1 4.5 3.3 4.1 4.3 4.3 4.1 4.4

The following charts display the frequency distributions of audit aspects and auditing staff ratings by audit type, starting with site audits.

Accreditation / Reaccreditation audit

A total of 72% of respondents were satisfied or very satisfied with their Accreditation / Re-accreditation audit.

After this question, respondents were asked to rate aspects of the audit. There were some negative results for some aspects and one respondent rated all staff aspects as very poor. Comments about negative aspects of this type of audit covered TAC’s response being unsatisfactory as well as a poorly written and conflicting audit report.


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

TAC informed us about audit purpose / process

TAC informed us about scope of audit

TAC informed us about audit sample

TAC informed us about changes to audit timetable & reasons

TAC audit staff gave us opportunity to discuss our situation / needs

TAC informed us about audit

purpose / process

TAC informed us about scope of

audit


sample

TAC informed us about changes to audit timetable &

reasons

TAC audit staff gave us

opportunity to discuss our

situation / needsStrongly agree 36.0 40.0 28.0 32.0 40.0

Agree 60.0 56.0 56.0 52.0 40.0

Part agree / part disagree 4.0 4.0 8.0 8.0 4.0

Disagree 0.0 0.0 4.0 8.0 16.0

Strongly disagree 0.0 0.0 0.0 0.0 0.0

Don't know 0.0 0.0 4.0 0.0 0.0

RTO: Course accreditation / reaccreditation audit aspects% of respondents n=27

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Objective

Knowledgeable

Organised

Informative

Constructive

Nonthreatening

Objective Knowledgeable Organised Informative Constructive NonthreateningExcellent 32.0 45.8 50.0 41.7 37.5 50.0

Good 40.0 37.5 37.5 29.2 33.3 20.8

Fair 20.0 12.5 8.3 20.8 16.7 20.8

Poor 4.0 0.0 0.0 4.2 8.3 4.2

Very poor 4.0 4.2 4.2 4.2 4.2 4.2

Don't know 0.0 0.0 0.0 0.0 0.0 0.0

RTO: Course accreditation / reaccreditation audit ‐ TAC audit team ratings

% of respondents n=27


Respondents were asked to rate the value the accreditation/re-accreditation audit added to their business. 68% of respondents agreed or strongly agreed that the audit added value.

Those who indicated that the audit added value were asked to indicate where. Aiding continuous improvement and demonstration of compliance were most commonly selected as seen in the chart below.

Comments about where any aspect of the audit could be improved included:

More consistency between auditors

Auditors using the most up-to-date documents when conducting audits

Clarify what is required when non-compliance occurs

Redesign the audit report form

Have auditors with more industry experience.

Strategic industry audits (SIAs)

60% of respondents who gave feedback about SIAs indicated that their organisation was satisfied or very satisfied with their audit while 13% (2 respondents) were dissatisfied or very dissatisfied.

One person offered a reason for their negative ratings and outlined that the auditor had been threatening and had lied to them and that the RTO believed there was a conflict of interest around the audit.

After the level of satisfaction question, respondents were asked to rate aspects of the audit and the results are displayed in the chart below.

4.3

8.7

52.2

65.2

73.9

78.3

0 20 40 60 80 100

Other

Benchmarked my org with other RTOs / course owners

Identified my org's strengths

Informed my org's processes & policies

Provided my org with opportunity to demonstrate compliance with AQTF

Aided in my org's continuous improvement

RTO: Accreditation / reaccreditation audit value added to business

Multiple answers allowed so total >100%; % based on n=27


The chart below shows the ratings for SIA audit staff. While there is still a majority of positive responses for most staff dimensions, the staff for SIA audits was the lowest rated compared with all other audits and has declined considerably since 2014.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%




TAC informed us of changes to audit timetable and reasons


TAC audit staff were willing to discuss any audit issues


purpose / process

TAC informed us about

scope of audit


sample

TAC informed us of changes

to audit timetable and

reasons


opportunity to discuss our situation /

needs


Strongly agree 26.7 26.7 13.3 20.0 26.7 26.7

Agree 60.0 60.0 66.7 66.7 46.7 46.7

Part agree / part disagree 13.3 13.3 13.3 6.7 26.7 20.0

Disagree 0.0 0.0 0.0 0.0 0.0 0.0

Strongly disagree 0.0 0.0 6.7 0.0 0.0 6.7

Don't know 0.0 0.0 0.0 6.7 0.0 0.0

RTO: SIA aspects% of respondents n=16


When asked to indicate whether the SIA added value to their business, 53% of respondents agreed or strongly agreed while 20% disagreed or strongly disagreed. Those who indicated that the audit added value were asked to indicate where. The chart below shows that most respondents believed that SIAs aided in continuous improvement and provided an opportunity to demonstrate compliance.

Comments about where any aspect of the audit could be improved included:

Auditors not making assumptions or relying on hearsay

Increased consistency between auditors

Have auditors with more industry experience

Auditors to be less confrontational and to show more respect

Better timing of audits (like not two in year and not during busy school periods)

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Objective

Knowledgeable

Organised

Informative

Constructive

Nonthreatening


Good 53.3 40.0 46.7 46.7 42.9 33.3

Fair 26.7 33.3 33.3 33.3 28.6 40.0

Poor 0.0 6.7 0.0 0.0 0.0 0.0

Very poor 6.7 0.0 0.0 6.7 14.3 13.3

RTO: SIA ‐ TAC audit team ratings% of respondents n=16

16.7

66.7

66.7

91.7

0 20 40 60 80 100

Benchmarked my organisation with other RTOs

Provided my organisation with an opportunity to demonstrate compliance with the AQTF

Informed my organisation's processes and policies

Aided in my organisation's continuous improvement

RTO: SIA value added to businessMultiple answers allowed so total >100%; % based on n=15


Desk audits

Overall, 85% of respondents were satisfied or very satisfied with their desk audit.

After this question, respondents were asked to rate aspects of the audit. The charts below show a fairly positive view, with the exception of two people who appear to have had issues around TAC audit staff’s willingness to discuss audit issues.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%








purpose / process


scope of audit


sample



reasons



needs


Strongly agree 20.0 20.0 15.0 20.0 21.1 26.3

Agree 75.0 80.0 60.0 80.0 68.4 57.9


Disagree 0.0 0.0 0.0 0.0 5.3 5.3


Don't know 0.0 0.0 5.0 0.0 0.0 0.0

RTO: Desk audit aspects% of respondents n=21


70% of respondents indicated that the desk audit added value to their business. Respondents who indicated that the audit added value where then asked to indicate where it added value. The opportunity to demonstrate compliance was the key benefit to respondents.

The one suggestion offered for improving desk audits was around redesigning the audit form to make it simpler.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Objective

Knowledgeable

Organised

Informative

Constructive

Nonthreatening


Good 50.0 50.0 55.0 40.0 47.4 40.0

Fair 0.0 5.0 0.0 10.0 5.3 5.0

Poor 5.0 0.0 0.0 0.0 5.3 5.0

Very poor 0.0 5.0 5.0 5.0 5.3 0.0

Don't know 10.0 5.0 10.0 10.0 5.3 10.0

RTO: Desk audit ‐ TAC audit team ratings% of respondents n=21

15.8

21.1

47.4

63.2

89.5

0 20 40 60 80 100


Identified my organisation's strengths




RTO: Desk audit value added to businessMultiple answers allowed so total >100%; % based on n=21


Site audits

Site audits were the most common type of audit for respondents and were rated positively overall. 74% of respondents were satisfied or very satisfied with the site audit conducted by TAC. It appears that two people had some negative perceptions of TAC audit staff.

After this question, respondents were asked to rate various aspects of the audit and the charts below display the responses. Most are very positive about the audit process, while a considerable portion of respondents were not so positive about audit staff.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%








purpose / process


scope of audit


sample



reasons



needs


Strongly agree 38.2 32.4 31.3 32.4 35.3 32.4

Agree 52.9 55.9 53.1 52.9 50.0 47.1


Disagree 0.0 2.9 3.1 2.9 0.0 8.8


Don't know 0.0 0.0 3.1 5.9 0.0 2.9

RTO: Site audit aspects% of respondents n=37


Just under 77% of respondents indicated that the site audit added value to their business. The group who indicated some level of value adding were asked to indicate where the audit added value. The opportunity to demonstrate compliance and aiding continuous improvement were the most common benefits—refer to chart below.

Suggestions for improving site audits included:

Auditors having more knowledge of the RTO / industry being audited

Greater consistency between auditors

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Objective

Knowledgeable

Organised

Informative

Constructive

Nonthreatening


Good 44.1 35.3 35.3 39.4 29.4 44.1

Fair 14.7 8.8 11.8 18.2 23.5 20.6

Poor 0.0 2.9 0.0 0.0 2.9 0.0

Very poor 2.9 2.9 2.9 3.0 2.9 0.0

Don't know 0.0 0.0 0.0 0.0 0.0 0.0

RTO: Site audit‐ TAC audit team ratings% of respondents n=37

10.0

26.7

63.3

66.7

80.0

90.0

0 20 40 60 80 100

Other


Identified my organisation's strengths




RTO: Site audit value added to businessMultiple answers allowed so total >100%; % based on n=37


More or more clear guidance about compliance for RTOs

Providing information on how other RTOs have implemented changes

Easier to understand audit reports

RTOs being able to pay for additional audits to help them improve.

RTO: VET sector improvements Respondents were asked to indicate which areas RTOs generally do NOT do well in order to focus TAC’s future regulatory support services. The chart below demonstrates that RTOs selected a wide range of issues with no single outstanding issue. The top four items were the same as in the 2014 survey. Note that around 10% of respondents did not know an answer.

10.4

1.9

4.2

5.7

5.7

9.0

9.4

9.4

9.4

11.3

13.2

14.6

20.8

25.9

27.8

28.3

28.3

0 20

Don't know

Complaints and appeals are addressed effectively

Adequate staff numbers, facilities & equip in relation to training provided

Learners individual needs are met

None of the above

Management systems are appropriate for the RTO

Adequate training is provided to learners

Monitoring of training conducted on behalf of an RTO

Accurate marketing of courses

Record keeping

Appropriately trained trainers and assessors

Assessments meet workplace/regulatory requirements

Employers / other parties adequately engaged in client training & assessment

Appropriate length of courses

Continuous improvement of systems

Assessments are in line with training packages

Recognition of Prior Learning (RPL)

RTO: What RTOs generally DO NOT do well% respondents choosing an answer

Multiple answers allowed so total may be >100%; % based on n=212


The results for this question have been examined further by RTO-number of qualifications and by RTO-number of students to gain further insight about differences in perspective. Organisations that offered no complete qualifications selected RPL as the top area for improvement whereas organisations that offered 1-6 complete qualifications most commonly selected continuous improvement. Those offering 7 or more qualifications selected appropriate length of courses as the top item. However, the top 5 items were the same for all RTOs irrespective of number of qualifications offered.

4.0

8.0

10.0

6.0

8.0

18.0

10.0

6.0

6.0

20.0

12.0

4.0

20.0

20.0

28.0

32.0

24.0

1.3

7.6

1.3

15.2

3.8

6.3

11.4

13.9

15.2

13.9

15.2

7.6

16.5

32.9

29.1

26.6

24.1

1.2

2.4

3.6

4.8

6.0

7.2

7.2

9.6

10.8

12.0

12.0

14.5

25.3

27.7

27.7

27.7

28.9

0 10 20 30 40

Complaints & appeals addressed effectively

Learner's individual needs are met


Management systemsappropriate for RTO

None of the above

Adequate training provided to learners


Don't know

Record keeping

Assessments meet workplace / regulatory requirements



Employers & other parties adequately engaged in client training & assessment


Assessments in line with training packages

Recognition of Prior Learning


RTO: What RTOs generally DO NOT do well by number of qualifications

% respondents choosing an answerMultiple selections allowed ‐ % based on n counts in legend below

7+ quals n=85 1‐6 quals n=79 0 quals n=52


When examined by number of students, the top five items are the same for each category, but in a slightly different order in each category.

0.0

2.0

2.0

4.0

4.0

6.0

8.0

8.0

10.0

14.0

16.0

20.0

22.0

26.0

36.0

40.0

40.0

2.4

14.6

9.8

7.3

2.4

7.3

9.8

9.8

2.4

9.8

14.6

7.3

22.0

29.3

26.8

26.8

26.8

3.8

9.6

3.8

15.4

9.6

3.8

15.4

7.7

5.8

9.6

13.5

19.2

23.1

30.8

21.2

25.0

19.2

2.0

9.8

23.5

9.8

3.9

0.0

15.7

13.7

5.9

7.8

11.8

7.8

17.6

25.5

25.5

23.5

27.5

0.0 10.0 20.0 30.0 40.0 50.0

Complaints & appeals addressed effectively

Adequate training provided to learners

Management systemsappropriate for RTO


None of the above


Record keeping

Don't know

Learner's individual needs are met


Assessments meet workplace / regulatory requirements


Employers & other parties adequately engaged in client training & assessment



Assessments in line with training packages


RTO: What RTOs generally DO NOT do well by number of students% of respondents choosing an answer

Multiple selections allowed so total may >100%; % based on n counts in legend below

1‐50 n=52 51‐200 n=52 201‐500 n=42 More than 500 n=52


RTO: Introduction of new standards This was a new topic for 2015 so no comparable year results are available. All respondents were asked to rate four aspects of TAC’s implementation of the new RTO standards. All items rated good on the rating scale.


RTO: TAC’S INTRODUCTION OF NEW STANDARDS n=216 Mean 2015

Top 2% 2015

Provided adequate support to my organisation if we had any questions or needed assistance of some kind

4.0 85.8

Provided clear information on its website about the new standards 3.9 80.6

Provided enough information to my organisation about the new standards

3.9 79.3

Provided timely advice to my organisation about the new standards 3.9 78.5

There were a small number of negative responses for each item, but a large majority rated TAC’s performance highly. Note the relatively large don’t know proportion for support / needing assistance. This is probably because these organisations did not seek assistance, which may be a good sign in itself.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Provided timely advice to my org about the new standards

Provided enough information to my org about new standards

Provided clear information on website about new standards

Provided adequate support to my org if we had questions / needed

assistance

Provided timely advice to my

org about the new standards

Provided enough information to my

org about new standards

Provided clear information on

website about new standards

Provided adequate support to my org if we had questions / needed assistance

Excellent 17.1 18.1 17.3 18.6

Good 57.4 58.3 60.3 54.4

Fair 13.9 14.4 11.7 7.0

Poor 4.2 3.2 5.1 3.3

Very poor 2.3 2.3 1.9 1.9

Don’t know / not applicable 5.1 3.7 3.7 14.9

RTO: Introduction of new RTO standards% of respondents, n=216


Those who rated any aspect of the introduction of new standards as fair, poor or very poor where asked to explain further. A total of 59 people offered a comment and the themes where two or more people offered a comment are displayed in the table below.

RTO: COMMENTS ABOUT INTRODUCTION OF NEW STANDARDS n=59 FREQ

Information not released in timely manner / no time to implement changes 25

Vague in parts / unclear 15

Unanswered questions from existing workshops 7

For RTO in remote location difficult to access information 4

Website does not have all relevant information yet 3

User guide not user friendly / hard to locate 2

RTO: TAC overall All respondents were asked to rate six aspects of TAC’s overall performance. All items rated at or slightly higher than good on the rating scale—an excellent result, particularly as the score for overall regulation is the highest in this topic.


RTO: TAC’S OVERALL PERFORMANCE Mean 2015

Mean 2014

Top 2% 2015

Overall as a regulator 4.2 4.2 88.1

Provides timely and quality advice to my organisation on its regulatory activities

4.1 4.2 87.8

Promotes and encourages continuous improvement of RTOs 4.1 4.1 87.1

Improves the quality of VET outcomes in Western Australia 4.1 4.1 86.3

Applies consistent regulatory decisions 4.0 4.1 86.7

Is transparent in its regulatory decisions and activities 4.0 4.1 85.6

There were very few negative responses as shown in the chart below and 88% of respondents rated TAC at the good or excellent level as a regulator, an identical result to 2014. This percentage of 88% excludes don’t know and no answers, whereas the percentages in the chart below include don’t know / no answers.


Note the relatively large number of don’t know responses for several items. It is reasonable that many organisations don’t know about the consistency of regulatory decisions, but it may be of concern that so many organisations were not prepared or did not have enough information to rate TAC on improving VET quality in WA.

RTO: Overall comments

Explanation of negative comments

When asked why respondents rated TAC negatively on any aspect of its overall performance 40 respondents offered comments which were quite varied. Common themes and frequency counts are displayed below.

RTO: REASON FOR NEGATIVE COMMENTS TAC OVERALL n=40 FREQ

Inconsistency of decisions, particularly auditors 19

No support for continuous improvement / no focus on helping RTOs 7

Rarely hear from them / not good communicators 6

Be more accessible / transparent / how are decisions made 5

Not easy to deal with 5

Other RTOs offering poor service impact badly on the sector / TAC does not improve outcomes for the entire sector 3

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Is transparent in its regulatory decisions and activities

Improves the quality of VET outcomes in Western Australia

Applies consistent regulatory decisions

Promotes and encourages continuous improvement of RTOs


Overall as a regulator

Is transparent in its regulatory decisions and

activities

Improves the quality of VET outcomes in

Western Australia


Promotes and encourages continuous

improvement of RTOs

Provides timely and quality advice to my organisation on its regulatory

activities


Excellent 25.5 24.1 20.8 24.5 27.3 30.6

Good 51.9 49.1 51.9 53.7 56.0 51.9

Fair 7.4 8.3 6.0 9.7 9.7 9.3

Poor 2.8 1.4 2.8 1.9 0.9 0.9

Very Poor 2.8 1.9 2.3 0.0 0.9 0.9

Don’t know 9.7 15.3 16.2 10.2 5.1 6.5

RTO: TAC's overall performance% of respondents n=216


RTO: REASON FOR NEGATIVE COMMENTS TAC OVERALL n=40 FREQ

Does not add value 3

Communications are not timely 3


RTO: Areas that create unnecessary regulatory burden

Respondents were given the opportunity to discuss any areas that created unnecessary regulatory burden and around half did. Many respondents indicated they were unsure about areas of heavy burden or that there were no areas of unnecessary burden—a positive result. The table below displays the themes were more than one respondent offered a suggestion to this question.

RTO: AREAS OF UNECESSARY BURDEN n=123 FREQ

Unsure / nothing 29


Streamline audit process, especially for high performers 15

Too many forms / reports / duplication of reports fed and state 15

Minor changes to scope should incur minor burden (not the case at present) 13

Surveys, ACER, learner employer surveys, AQTF surveys 7

Too many changes / not notified soon enough about changes 6

Should be evidence/ example based and not written 6

Ensure consistency of staff especially auditors 5

TAC to be more timely 5

Validation / cross industry validation 4

Poor operators are still out there 3

TAC lacks understanding of our RTO / industry 3

More submissions online needed 3


RTO: What TAC does well

When asked to offer suggestions about what TAC does well and that it should continue doing, respondents most commonly indicated that they were happy with the case manager and state based approach of TAC. TAC’s more personal focus is an over-arching theme in these comments.

RTO: WHAT TAC DOES WELL n=155 FREQ

Case managers 44

Informs RTO about changes / keeps us up to date 19

Feedback and support given to RTOs especially about compliance 17

Communication 12

Good / OK overall 11

Staff helpful / pleasant / nice / available 9

Timely response 7

Human element / approachable / open 7

Easy to deal with 7

Good auditors / audit process 6

State approach 5

Improve / monitor quality of training 4

Website / online applications 3

Newsletter 3

Unsure / nothing 2

Include regional areas 2

RTO: What TAC could improve

The most common suggestions for improvement related to consistency of decisions from audit staff—a similar common theme from the same question in the 2014 survey. Balancing this is that next most commonly, respondents indicated that nothing needed to change. This year, the website has received much less comment, indicating that TAC has done something to address this area.

RTO: WHAT TAC COULD IMPROVE n=135 FREQ

Inconsistency, especially auditors’ knowledge and decision making 27

Positive / nothing / all good / unsure 25

Use simple clear communication / unambiguous / correct information 17

More information / info sessions / training 11


Provide general advice / be a sounding board / help us stay up to date and be compliant before we a reaudited 7

More help with changes / advise of changes earlier 7

Provide templates / case studies / best practice examples 6

Website / online transactions / reporting functions 5

Timeliness of decisions / responses / interactions 5

Fees too high / lower costs 5

Automate some procedures, eg reaccreditation if no changes made 5


RTO: WHAT TAC COULD IMPROVE n=135 FREQ

Dealing with poorly behaving RTOs 4

Update / respond to queries even if they cannot be solved immediately 3

Have specialist staff, like industry experts 3

Simplify processes / reduce red tape 3

Better prepared guidelines / new standards 3

Be more decisive 2

RTO differences analysis All rated questions were analysed by two RTO attributes: number of qualifications and number of students. Analysis of variance (ANOVA) was used to determine if there were any statistically significant differences between sub-groups within the two attributes of number of qualifications and number of students. Only items with significant differences have been included in the report.

Differences have been calculated at the 95% confidence level, that is, p<0.05.

Number of qualifications

The table below displays the six items that were statistically significantly different between the three sub-groups of number of qualifications. Blue highlighting indicates that means scores were significantly lower than other groups while yellow highlighting indicates significantly higher. Organisations offering no qualifications tended to rate TAC lower than other sub-groups (1-6 and 7+ quals) for these items. However, for all but the item relating to introduction of new RTO standards, there are very small cell counts (as in 4 or 5 responses) and so the results should be treated as very indicative only.


RTO: STATISTICALLY DIFFERENT ITEMS x NUMBER OF QUALS 0 quals 1-6

quals 7+

quals

Phone accreditation staff: Efficiency of response 4.6 3.9 4.4

Phone administration staff: Clarity of response 3.8 4.3 4.5

Phone management staff: Clarity of response 3.7 5.0 4.2

Phone management staff: Sufficient contact information to recontact TAC staff member 3.7 5.0 4.4

Accreditation session: Timely (not outdated) information 4.0 4.6 4.4

Introduction of new RTO standards: Provided adequate support to my organisation if we had questions / needed assistance

3.7 4.1 4.0

Number of students

There was only one item that was statistically significantly different when the sub-groups within number of students were compared. This was the score for efficiency of response when phoning Accreditation staff. Organisations with 1-50 students rated significantly lower (mean 3.8) when compared with all other groups.


Overall

That there are so few differences indicates that TAC’s behaviour is consistent, and consistently good and has remained consistent over time across all areas and with all types of organisations it interacts with—an excellent result.


Stakeholder survey key findings

The following section presents results from the stakeholder survey. Results are presented in the same order as presented to respondents in the questionnaire. The response sample for this survey was relatively small so results should be treated as indicative only.

Stakeholder: Interacting with TAC Respondents were asked to indicate the different types of interactions they had with TAC in the previous 12 months. Most common interactions were website usage and phone conversations. No stakeholder indicated having no interactions in the time period and none undertook moderation activities with TAC. The Other responses included discussing cases, requesting an audit, giving audit feedback, briefings, council meetings and exchange of information.

It is likely in this survey that many stakeholders who had little or no interaction did not participate, so this level of no interaction may be slightly misleading if interpreted to represent all stakeholders.

Only stakeholders who had used a particular channel or method of interaction were asked questions about that channel / method.

0.0

26.5

0.0

2.9

8.8

47.1

55.9

58.8

73.5

82.4

0 20 40 60 80 100

No contact

Other

Undertook moderation activities with TAC

Attended a TAC audit as an observer or technical advisor

Used the TAC general email address / enquiry service

Received a TAC Update / Special Bulletin

Attended an event at which a TAC staff member spoke

Met with TAC to discuss issues of VET regulation

Had a phone conversation with TAC staff

Used the TAC website

Stakeholder: Types of interactions with TAC in 2014/15multiple selections allowed, n=37


Stakeholder: TAC website All items below were rated at or just below good on the rating scale. While there were slight improvements in scores between years, there were no statistically significant differences.


RTO: TAC WEBSITE n=28 Mean 2015

Mean 2014

Top 2% 2015

Accurate information 4.0 3.9 92.0

Current/up to date information 3.9 3.8 88.5

Provides enough information 3.9 3.6 88.5

Easy to understand information 3.9 3.8 84.6

Easy to search 3.8 3.6 76.9

Easy to navigate 3.8 3.6 73.1

The frequency distribution below demonstrates that scores were fairly positive for this topic. Few aspects had excellent ratings while two items in the topic had a small proportion of poor ratings. These lower rated items related to searching and navigation.

Eight respondents provided poorer performance comments about this topic: searching and navigation were commonly mentioned as areas for improvement along with documents being buried, language was very bureaucratic and information was not always up-to-date.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Easy to navigate

Easy to search





Easy to navigate Easy to searchEasy to

understand information




Excellent 7.1 10.7 7.1 3.6 0.0 7.1

Good 60.7 60.7 71.4 78.6 82.1 75.0

Fair 21.4 14.3 14.3 10.7 10.7 7.1

Poor 3.6 7.1 0.0 0.0 0.0 0.0

Very poor 0.0 0.0 0.0 0.0 0.0 0.0

Don't know 3.6 3.6 3.6 3.6 3.6 3.6

No answer 3.6 3.6 3.6 3.6 3.6 7.1

Stakeholder: Website% of respondents choosing an answer, n=28


Stakeholder: TAC Update / Special bulletin Sixteen respondents were shown the question about TAC Update / Special bulletin. Mean scores fell around good on the rating scale.


RTO: TAC UPDATE n=16 Mean 2015

Mean 2014

Top 2% 2015

Helpfulness of information 4.1 3.9 100.0

Timeliness of information 3.9 3.8 86.7

Informs on a wide range of issues 3.9 3.8 85.7

The chart below displays positive results. Helpfulness of information has one of the highest positive proportions (sum of excellent and good percentages) in the stakeholder survey. One comment for the fair ratings was that not much new information was provided regularly.

Stakeholder: TAC general email service Only three stakeholders indicated that they had used the TAC email service. All service dimensions were rated either excellent or good, with the exception that one person indicated that promptness was fair: sometimes there were delays in receiving answers.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%




Informs on a wide range of issues Timeliness of information Helpfulness of informationExcellent 6.3 6.3 6.3

Good 68.75 75 87.5

Fair 12.5 12.5 0

Poor 0 0 0

Very poor 0 0 0

Don't know 0 0 0

No answer 12.5 6.25 6.25

Stakeholder: TAC Update / Special bulletin% of respondents choosing answer, n=16


Stakeholder: TAC phone service Twenty-five respondents indicated phoning TAC in the last 12 months and, most commonly, respondents phoned 3 to 5 times (40%). Four stakeholders phoned more than 10 times. Stakeholders were most likely to phone TAC management (56%) compared with only 13% of RTOs that phoned TAC management in the same period. Management was the same area that stakeholders most commonly phoned in the previous survey.

The table below compares the mean scores across all the different areas that stakeholders phoned in the last 12 months. All scores are around the good rating point, on average, with some even higher—an overall excellent result. Only two areas improved significantly since 2014: courtesy in Registration and Management. Note the very small cell counts with some phone areas, so all phone results for stakeholders should be treated with caution.


STAKEHOLDER: TAC PHONE SERVICE

Accred’n n=9

Regist’n n=10

Admin n=5

Mgt n=14

Other n=12

2015 2014 2015 2014 2015 2014 2015 2014 2015 2014

Clarity of response 4.0 4.0 4.3 4.0 4.2 4.0 4.1 4.1 4.4 4.1

Courtesy 4.3 4.4 4.8 4.1 4.6 4.3 4.6 4.0 4.5 4.3

Efficiency of response 3.9 4.1 4.4 3.9 4.0 4.3 4.2 3.9 4.2 4.0

Helpfulness 4.1 4.0 4.4 4.0 4.4 4.0 4.4 4.1 4.3 4.4

Impartiality 4.2 4.1 4.6 4.2 4.2 4.3 4.2 4.1 4.2 4.3


3.9 4.1 4.3 4.0 4.2 4.0 4.4 4.1 4.4 4.0


4.4 4.4 4.8 4.4 4.3 4.7 4.5 4.2 4.1 4.1


4.0 4.2 4.3 4.1 4.0 4.7 4.2 4.3 4.3 4.1

20.0

36.0

40.0

48.0

56.0

0 20 40 60

Administration

Accreditation

Registration

Other (Complaints, Policy, etc)

Management

Stakeholder: Areas of TAC phoned in 2014/15% respondents choosing an answer

Multiple selections allowed so total >100%; % based on n=25


The following series of charts display the frequency distributions for the phone service topic.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Courtesy


Impartiality

Helpfulness

Clarity of response




Courtesy




Impartiality Helpfulness Clarity of response


(took minimal

amount of time,


Knowledge of staff

answering

Sufficient contact


Excellent 33.3 33.3 22.2 11.1 11.1 0.0 22.2 11.1

Good 66.7 55.6 77.8 88.9 77.8 88.9 55.6 77.8

Fair 0.0 0.0 0.0 0.0 11.1 11.1 11.1 11.1

Poor 0.0 0.0 0.0 0.0 0.0 0.0 11.1 0.0

Very poor 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Don't know 0.0 11.1 0.0 0.0 0.0 0.0 0.0 0.0

No answer 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Stakeholder: Phone accreditation staff% of respondents choosing answer, n=9


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Courtesy


Impartiality

Helpfulness

Clarity of response




Courtesy






(took minimal

amount of time,


Knowledge of staff

answering

Sufficient contact


Excellent 70.0 60.0 50.0 40.0 40.0 50.0 40.0 40.0

Good 20.0 20.0 30.0 50.0 40.0 30.0 40.0 40.0

Fair 0.0 0.0 0.0 0.0 10.0 10.0 10.0 10.0

Poor 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Very poor 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Don't know 10.0 20.0 20.0 10.0 10.0 10.0 10.0 10.0

No answer 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Stakeholder: Phone registration staff% of respondents choosing an answer, n=10


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Courtesy


Impartiality

Helpfulness

Clarity of response




Courtesy






(took minimal

amount of time,


Knowledge of staff

answering

Sufficient contact


Excellent 60.0 20.0 20.0 40.0 20.0 20.0 20.0 20.0

Good 40.0 60.0 80.0 60.0 80.0 60.0 80.0 60.0

Fair 0.0 0.0 0.0 0.0 0.0 20.0 0.0 20.0

Poor 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Very poor 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Don't know 0.0 20.0 0.0 0.0 0.0 0.0 0.0 0.0

No answer 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Stakeholder: Phone administration staff% of respondents choosing an answer, n=5


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Courtesy


Impartiality

Helpfulness

Clarity of response



Sufficient contact information so that I could contact / recontact a TAC staff …

Courtesy




Impartiality HelpfulnessClarity of response


(took minimal

amount of time,


Knowledge of staff

answering

Sufficient contact

information so that I

could contact /

recontact a TAC staff member if necessary

Excellent 64.3 57.1 35.7 50.0 35.7 35.7 42.9 28.6

Good 28.6 28.6 42.9 42.9 42.9 50.0 50.0 50.0

Fair 7.1 7.1 14.3 7.1 21.4 14.3 7.1 14.3

Poor 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Very poor 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Don't know 0.0 7.1 7.1 0.0 0.0 0.0 0.0 7.1

No answer 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0

Stakeholder: Phone management staff% of respondents choosing an answer, n=14


Stakeholder: Phone service comments

There were only two comments for the phone service areas: inconsistencies when staff were on leave and rude reception staff.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Courtesy


Impartiality

Helpfulness

Clarity of response




Courtesy






(took minimal

amount of time,


Knowledge of staff

answering

Sufficient contact


Excellent 50.0 33.3 16.7 33.3 33.3 33.3 33.3 25.0

Good 41.7 41.7 58.3 41.7 50.0 50.0 58.3 66.7

Fair 0.0 0.0 0.0 8.3 0.0 0.0 0.0 0.0

Poor 0.0 0.0 0.0 0.0 0.0 8.3 0.0 0.0

Very poor 0.0 8.3 0.0 0.0 0.0 0.0 0.0 0.0

Don't know 8.3 16.7 25.0 8.3 8.3 8.3 8.3 8.3

No answer 0.0 0.0 0.0 8.3 8.3 0.0 0.0 0.0

Stakeholder: Other phone staff% of respondents choosing an answer, n=12


Stakeholder: TAC speech or presentation Mean scores for this topic were good or above. The rating for useful information in speeches improved significantly since 2014.


STAKEHOLDER: TAC SPEECH / PRESENTATION n=19 Mean 2015

Mean 2014

Top 2% 2015




The frequency distribution chart shows no negative results for this topic, but two fair responses for one item. One comment offered was about TAC’s limited world view which is affected by legislation and ASQA being in the same space.

Stakeholder: VET regulation discussion All items below are rated at or above good on the rating scale. No items were significantly different from the previous survey.


STAKEHOLDER: VET REGULATION DISCUSSION n=20 Mean 2015

Mean 2014

Top 2% 2015




0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%



Useful information


Timely (not outdated) information Useful information

Excellent 21.1 36.8 31.6

Good 63.2 63.2 63.2

Fair 10.5 0.0 0.0

Poor 0.0 0.0 0.0

Very poor 0.0 0.0 0.0

Don't know 0.0 0.0 0.0

No answer 5.3 0.0 5.3

Stakeholder: Speech or presentation% of respondents choosing an answer, n=19


The chart below displays the frequency distribution for this topic. Five respondents provided poorer performance comments for this topic and they focused on having a broader and deeper understanding of stakeholders or organisations being audited.

Stakeholder: Regulating the VET sector Respondents were asked to demonstrate their awareness of the TAC’s recent strategic industry audits. The vast majority (97%) of respondents had some level of awareness of SIAs. When asked to rate their level of support for continual SIAs about 79% of respondents indicated a strong level of support (a lot or great deal of support).

Stakeholders were also asked to rate areas of regulatory support services that RTOs generally do not do well. The top items for stakeholders as shown in the chart below were quite different to RTOs’ top items but appropriate length of courses was common as a top item between the two surveys.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%



Useful information


Timely (not outdated) information Useful information

Excellent 25.0 35.0 40.0

Good 50.0 60.0 55.0

Fair 25.0 5.0 5.0

Poor 0.0 0.0 0.0

Very poor 0.0 0.0 0.0

Don't know 0.0 0.0 0.0

Stakeholder: VET regulation discussion% of repsondents choosing an answer, n=20

3.0

0.0

6.1

12.1

24.2

54.5

0 20 40 60

Don’t have a view / unsure

No support at all

A little support

A moderate amount of support

A lot of support

A great deal of support

Stakeholder: Level of support for continual SIAs% of respondents choosing an answer, n=33


Stakeholder: Registration officer / case management model Respondents were asked to provide comments about the current registration / case management model in place at TAC. Only 11 respondents provided a comment and most of these did not address the question. Four indicated that the model worked well.

2.9

2.9

2.9

5.9

8.8

8.8

11.8

14.7

17.6

23.5

23.5

29.4

35.3

41.2

41.2

0 20 40 60

Learners individual needs are met

Complaints and appeals are addressed effectively

Management systems appropriate for RTO

Record keeping

Adequate staff numbers, facilities & equip for training provided




Adequate training is provided to learners




Assessments meet workplace/regulatory requirements

Assessments are in line with training packages

Employers / others adequately engaged in client training & assessment

Stakeholders: Areas that RTOs do not do well% respondents choosing an answer

Multiple selections allowed so total may be >100%; % based on n=34


Stakeholder: Introduction of new standards in 2015 All items below are rated on average around good on the rating scale.


STAKEHOLDER: INTRODUCTION OF NEW STNDS n=34 Mean 2015

Top 2% 2015


Provided enough information to my organisation about the new standards

3.9 85.7



3.7 81.5

The frequency distribution chart shows mostly positive responses where respondents had an opinion. A considerable number of people answered don’t know to each item in the topic, and this might be reasonable given the backgrounds of some stakeholders.

Comments from those who rated the topic from fair to very poor indicated that TAC did not offer clarity about nuances of the standard, there was a lack of briefings to some sectors / organisations, guidelines were released very late and there was little explanation about why the changes were being made.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Provided enough information to my org about the new standards

Provided timely advice to my org about the new standards

Provided clear information on its website about the new standards

Provided adequate support to my org if we had questions / needed

assistance

Provided enough information to my org about the new

standards

Provided timely advice to my org about the

new standards

Provided clear information on its

website about the new standards

Provided adequate support to my org if we had questions / needed assistance

Excellent 8.8 5.9 8.8 5.9

Good 61.8 55.9 64.7 58.8

Fair 5.9 11.8 2.9 5.9

Poor 5.9 5.9 2.9 5.9

Very poor 0.0 0.0 2.9 2.9

Don’t know / not applicable 14.7 20.6 14.7 20.6

No answer 2.9 0.0 2.9 0.0

Stakeholder: Introduction of new RTO standards in 2015% of respondents choosing an answer, n=34


Stakeholder: TAC Overall All respondents were asked to rate TAC on 16 overall performance items and the table below shows mean scores for these items. All scores rounded to the good rating point and many stakeholder mean scores increased since 2014, including the overall rating as a regulator. Two of these increases were statistically significant: providing advice to and collaborating with industry bodies, other regulators and peak associations.

The top 2 scores indicate some areas for improvement from a stakeholder perspective, particularly those items that scored below 70%. This improvement might come through more or more detailed information from TAC.


STAKEHOLDER: TAC OVERALL n=34 Mean 2015

Mean 2014

Top 2% 2015

Provides timely information to the VET sector in general about changes to regulations / general directions

4.0 3.8 93.3

Engages with stakeholders such as my organisation 3.9 3.6 73.5

Acts on complaints received about its own performance 3.9 3.7 73.7

Provides timely, quality advice about the VET sector to my organisation

3.9 3.7 76.7

Acts on complaints received about training providers 3.9 3.6 81.5

Open to hearing concerns about the quality of VET outcomes 3.9 3.9 84.8

Provides advice to industry bodies such as industry skills councils, other regulators and peak associations

3.9 3.5 86.2

Promotes and encourages continuous improvement of RTOs 3.9 3.6 86.7

Overall as a regulator 3.8 3.6 73.5


3.8 3.4 82.1

Collaborates with industry bodies, other industry regulators and peak associations

3.8 3.4 83.3

Acts on stakeholder feedback 3.7 3.3 67.9

Is transparent in its regulatory decisions and activities 3.7 3.5 70.4

Seeks feedback from stakeholders 3.7 3.6 71.0

Improves the quality of VET outcomes in Western Australia 3.7 3.6 80.6

Applies consistent regulatory decisions 3.6 3.5 65.4

The frequency distribution charts (chart A and B below) show a relatively high proportion of don’t know responses from stakeholders for many items in this topic and it was highest in this topic compared with all other topics in both surveys. The reasons may include:

Stakeholders genuinely have no or limited experience, understanding or feedback about particular areas making it difficult to choose a valid response, particularly around complaints and consistency of decisions.

Lack of willingness to supply an answer as it might incriminate the respondent in some way (maybe not trusting the confidentiality of the conduct of the survey).

The areas with high don’t know responses could be messages that TAC includes when communicating with stakeholders in the future.


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%


Acts on stakeholder feedback


Seeks feedback from stakeholders

Engages with stakeholders such as my organisation


Acts on complaints received about its own performance



Acts on complaints received about training providers


Acts on stakehold

er feedback

Is transparent in its

regulatory decisions

and activities

Seeks feedback

from stakehold

ers

Engages with

stakeholders such as my

organisation

Overall as a

regulator

Acts on complaints received about its

own performan

ce

Provides timely, quality advice

about the VET

sector to my

organisation

Improves the quality

of VET outcomes

in Western Australia

Acts on complaints received

about training

providers

Excellent 8.8 8.8 11.8 5.9 17.6 5.9 11.8 14.7 2.9 11.8

Good 41.2 47.1 44.1 58.8 55.9 67.6 29.4 52.9 70.6 52.9

Fair 17.6 23.5 17.6 23.5 20.6 23.5 11.8 14.7 11.8 8.8

Poor 5.9 0.0 2.9 2.9 5.9 2.9 2.9 5.9 2.9 2.9

Very Poor 2.9 2.9 2.9 0.0 0.0 0.0 0.0 0.0 2.9 2.9

Don’t know 23.5 17.6 20.6 8.8 0.0 0.0 44.1 11.8 8.8 20.6

Stakeholders: Overall ‐ chart A% of respondents, n=37


Stakeholder: TAC overall comments

Explanation of negative comments

Respondents who rated TAC negatively in the overall section were given the opportunity to explain their responses. A total of 15 respondents provided comments. Themes included:

One or more aspects of audits are unsatisfactory, like inconsistencies and the need to work more with industry experts

TAC’s communication style—being more responsive and willing to listen.

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%



Open to hearing concerns about the quality of VET outcomes

Provides advice to industry bodies such as industry skills councils, other regulators and

peak associations



Provides timely and quality

advice to my organisation on its regulatory

activities

Collaborates with industry bodies, other industry regulators and

peak associations

Open to hearing concerns about

the quality of VET outcomes

Provides advice to industry

bodies such as industry skills councils, other regulators and

peak associations

Promotes and encourages continuous

improvement of RTOs

Provides timely information to

the VET sector in general about

changes to regulations /

general directions

Excellent 2.9 2.9 8.8 5.9 11.8 11.8

Good 64.7 70.6 73.5 67.6 64.7 70.6

Fair 11.8 11.8 14.7 8.8 5.9 2.9

Poor 0.0 2.9 0.0 2.9 5.9 0.0

Very Poor 2.9 0.0 0.0 0.0 0.0 2.9

Don’t know 17.6 11.8 2.9 14.7 11.8 11.8

Stakeholders: Overall ‐ chart B% of respondents choosing an answer, n=34


Stakeholder: Areas that created unnecessary regulatory burden

A total of 8 respondents provided comments unnecessary regulatory burdens for high performing providers. Ideas offered covered were quite specific and there were no common themes.

Stakeholder: What TAC does well

When asked what TAC does well that it should continue doing, 13 respondents provided comments. Areas that TAC did well included:

SIAs

Providing information

Engaging with stakeholders

Customer service.

Stakeholder: What TAC could improve

A total of 16 respondents suggested improvements. Common themes were:

Transparency of decision making

Issues around audits, particularly consistency

Greater outcome focus

More information to stakeholders and greater understanding of their contexts.


RTO and stakeholder comparison overall Identical items in two topics in the RTO and stakeholder surveys were compared: the introduction of new standards in 2015 and the overall topic. No other similar items were compared due to differences in sample sizes in specific questions (many more RTOs than stakeholders answered and this large difference in sample size reduces the reliability of statistical difference tests).

The table below shows the mean scores for the 10 comparable. RTOs rated items at the same level or higher than stakeholders—exactly the opposite of the 2014 survey. Three of the differences below were statistically significantly different at the 95% confidence level.


RTO AND STAKEHOLDER: COMPARABLE ITEMS RTO MEAN n=216

STAKEHOLDER MEAN n=34

Provided enough information to my organisation about the new standards 3.9 3.9



Provided adequate support to my organisation if we had any questions or needed assistance of some kind 4.0 3.7

Provides timely and quality advice to my organisation on its regulatory activities 4.1 3.8

Is transparent in its regulatory decisions and activities 4.0 3.7

Applies consistent regulatory decisions 4.0 3.6

Promotes and encourages continuous improvement of RTOs 4.1 3.9

Improves the quality of VET outcomes in Western Australia 4.1 3.7

Overall as a regulator 4.2 3.8


Conclusions

General

Overall, the results are positive for TAC. Most RTOs perceive TAC as an effective regulator and that it is improving the quality of VET outcomes in Western Australia. Stakeholders are not quite as generous in their ratings in this area.

Nearly all aspects of client interactions are working well, and TAC has not only maintained but slightly improved this overall experience since 2014.

Case management is the preferred model

RTOs like the current case management approach that TAC has in place. It is the main area that RTOs indicated TAC does well and should continue doing.

Audits

Audit was the only area where results decreased across the board since 2014. There appear to be issues with some auditors and their interactions with RTOs. Consistency of auditor decisions and approach is an area where TAC could look to improve its audits and subsequently the relationship it has with RTOs.

Stakeholders

Stakeholders rated TAC fairly positively overall. Areas where TAC could improve with stakeholders focused on transparency in decisions, audits, greater outcome focus and more engagement with stakeholders.


Attachment 1: Questionnaires

Produced by Australian Survey Research Page | 1

TAC RTO survey 2015

Introduction You are answering about #%LegalName%# As part of its improvement focus, the Training Accreditation Council (TAC) is seeking feedback from all TAC regulated training providers on your recent interactions with TAC. Feedback will be used to improve how TAC interacts with the industry as a whole. TAC has engaged Australian Survey Research (ASR) as an independent research agency to conduct the survey on its behalf. The survey will take around 10 – 20 minutes to complete, depending on the types of interactions you may have had with TAC. Please complete by 9am, 9 June 2015. When answering all questions, please refer to the period from June 2014 until now. Your answers are confidential. At no time will TAC see your individual answers. ASR will only present summary results to TAC. Because the questions cover a range of topics and interactions, you may need to involve other people from your organisation in answering. Think of the link to the questionnaire as an organisational link which can be passed from person to person. However, keep in mind that once the questionnaire is finalised (submitted), the link will de-activate. ASR can re-activate the link at your request. Printing answers If you would like to print the questions and your answers, you can do this in two ways:

1. Page by page - use the print icon to the right of the top bar of the questionnaire.

2. All pages together - this is ONLY available once you have finalised (submitted) the questionnaire. A link will be available on the finalise page. Just click on it to commence the printing process.

Hints for answering

If you get interrupted while answering, please click on the return later link at the bottom right of a page. This will save you answers so far and close down the questionnaire.

When you return using your unique link in your invitation email, you will be returned to the last page you saved.

Clicking on the word Next at the bottom of a screen will save your answers and move you to the next page. If you cannot see this word on your screen, use your scroll bar to move down to it.

Clicking on the word Previous at the bottom of a page will return you to the previous page. Please do not hit the back button on your browser as it may take you out of the questionnaire.

If you have any technical questions about the survey please contact Australian Survey Research on (03) 9578 5211 or 1800 068 489 or [email protected]. If you have any questions about the survey content or rationale, please contact Kieran Tynan at TAC on 08 9441 1917 or [email protected]. About your organisation Approximately, how many student enrolments did your organisation have across all the nationally recognised training programs you offered from June 2014 to now?

More than 500 201-500 51-200 1-50 None

What types of interactions did your organisation have with TAC from June 2014 until now? Select all that apply.

Made an application to TAC Notified TAC of change to organisation or scope Used the TAC website


Received a TAC Update and/or TAC Special Bulletin (email newsletter) Used RTONet Used the TAC general email address / enquiry service - [email protected] Had a phone conversation with TAC staff Attended a TAC RTO or accreditation information session Attended an event, other than an RTO or accreditation information session, at which a TAC staff

member spoke Other Please specify ______________________________________________________ No contact

Applications and notifications What type/s of application or notification has your organisation made to TAC since June 2014? Select all that apply.

Initial registration application Renewal of registration application Amendment to registration application Change of RTO details Removal of organisation’s scope of registration and delivery Change of legal entity Request for intent to transfer Accreditation - initial course accreditation Accreditation – reaccreditation of a course Accreditation – application for amendment to course Accreditation – application for extension to course Total VET activity exemption Other Please specify______________________________________________________

Rating applications and notifications Based on the applications and/or notifications that you have submitted to TAC, please rate the following items. If you have made several applications / notifications since June 2014, refer to your overall experience and not a specific interaction.

Excellent

Good

Fair

Poor

Very poor

Don't know

Ease of accessing RTONet

Ease of navigating RTONet

Ease of completing the task required using RTONet

Clarity of instructions in RTONet

Clarity of TAC’s application form

Helpfulness of information on TAC’s website regarding making applications

Availability of follow up assistance

Time TAC took to act on the application after you were notified that the application was received


Application / notification comments You indicated in one or more of your answers to the previous question that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ TAC website Please rate the TAC website on the following:

Excellent Good Fair Poor Very poor Don't know





Easy to navigate

Easy to search

TAC website comments You indicated in one or more of your answers about the TAC website that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ TAC Update and/or Special Bulletin Please rate the TAC Update and/or TAC Special Bulletins (both email newsletters) on the following:





TAC Update and/or Special Bulletins comments You indicated in one or more of your answers about TAC's Update and Special Bulletins that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ TAC general email service Approximately, how many times have you used the TAC general email service since June 2014?

1-2 times 3-5 times 6-10 times More than 10 times Don't know / not sure


Overall, please rate TAC's general email service on the following:

Excellent

Good

Fair

Poor

Very poor

Don't know

Courtesy


Impartiality

Helpfulness

Clarity of response

Efficiency of response (took minimal amount of words to respond)

Promptness (speed of reply)


Sufficient contact information so that I could contact / recontact a TAC staff member if necessary

TAC email service comments You indicated in one or more of your answers about TAC's email service that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ TAC phone service Approximately, how many times have you phoned TAC since June 2014?

1-2 times 3-5 times 6-10 times More than 10 times Don't know / not sure

Which area/s of TAC have you phoned since June 2014? Select all that apply.

Accreditation Registration (where Registration Officers / Case Managers are located) Administration Management Other (Complaints, Policy, etc) Don't know / not sure

Phone service ratings Please rate the phone conversations you had with TAC Accreditation staff on the following:

Excellent

Good

Fair

Poor

Very poor

Don't know

Courtesy



Impartiality

Helpfulness

Clarity of response




Please rate the phone conversations you had with TAC Registration staff on the following:

Excellent

Good

Fair

Poor

Very poor

Don't know

Courtesy


Impartiality

Helpfulness

Clarity of response




Please rate the phone conversations you had with TAC Administration staff on the following:

Excellent

Good Fair Poor Very poor

Don't know

Courtesy


Impartiality

Helpfulness

Clarity of response



Sufficient contact information so that I could contact / recontact a TAC staff member if


necessary

Please rate the phone conversations you had with TAC Management on the following:

Excellent


Don't know

Courtesy


Impartiality

Helpfulness

Clarity of response




Please rate the phone conversations you had with other TAC staff (not Accreditation, Registration, Administration or Management) on the following:

Excellent


Don't know

Courtesy


Impartiality

Helpfulness

Clarity of response




TAC phone service comments - Accreditation You indicated in one or more of your answers about TAC's Accreditation phone service that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________


TAC phone service comments - Registration You indicated in one or more of your answers about TAC's Registration phone service that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ TAC phone service comments - Administration You indicated in one or more of your answers about TAC's Administration phone service that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ TAC phone service comments - Management You indicated in one or more of your answers about TAC's Management phone service that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ TAC phone service comments - other areas You indicated in one or more of your answers about TAC's phone service in areas other than Accreditation, Registration, Administration and Management that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ Registration officer What comments would you like to offer about the current registration officer / case management model of handling RTO enquiries? You may wish to comment about how well or not so well the service model works or alternative service models that would work better for your organisation.

__________________________________________________________________ TAC RTO / Accreditation session Please rate the information you received from attending an RTO or Accreditation information session on the following:



Useful information


TAC information session comments You indicated in one or more of your answers about an RTO / accreditation session that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ TAC speech or presentation Please rate the information you received from a TAC staff member's presentation (other than an RTO / accreditation session) on the following:



Useful information



TAC speech / presentation comments You indicated in one or more of your answers about a TAC staff member's speech or presentation (other than an RTO / accreditation session) that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ Interacting with TAC If TAC was to provide additional information sessions or workshops on specific regulatory matters for RTOs, would you be interested in attending?

Yes No Depends (on topic / timing)

What topics would you like TAC to address?

__________________________________________________________________ What is your most preferred method for receiving general advice about the VET sector from TAC? Ideally, please select one method, but if you cannot decide, choose more.

Simple broadcast email Hard copy letter / sheet TAC Update (which is sent as a formatted email newsletter) Information sessions / workshops OtherPlease specify__________________________________________________________ None of the above

Type of audit TAC has indicated that your organisation was audited during the last 12 months. What type of audit/s did TAC conduct of your RTO in that period?Select all that apply.

Course accreditation or re-accreditation (NOT RTO Registration) Strategic Industry Audit (SIA) Desk audit Site audit Don't know

Course accreditation / re-accreditation audit Please rate TAC's performance when conducting the course accreditation / re-accreditation audit on the following:

Strongly agree

Agree Part agree /

part disagree

Disagree

Strongly disagree

Don't know

TAC informed my organisation before and during the audit about the audit purpose and process

TAC informed my organisation about the scope of the audit

TAC informed my organisation of any changes to the audit timetable and reasons for changes

TAC audit staff gave my organisation an opportunity to discuss our particular business situation /


needs


How much do you agree/disagree that the Accreditation / Re-accreditation audit added value to your business? Strongly agree Agree Part agree / part disagree Disagree Strongly disagree Don't know

You indicated in the previous question that the audit added some degree of value to your business. Which types of value did it add? Select all that apply.

Provided my organisation with an opportunity to demonstrate compliance with the AQTF Informed my organisation's processes and policies Benchmarked my organisation with other RTOs / course owners Identified my organisation's strengths Aided in my organisation's continuous improvement Other Please specify______________________________________________________

Please rate the TAC audit team on the following:


Objective

Knowledgeable

Organised

Informative

Constructive

Non-threatening

Overall, rate your organisation's satisfaction with the TAC Accreditation / Re-accreditation audit? Very satisfied Satisfied Part satisfied / part dissatisfied Dissatisfied Very dissatisfied

You indicated in the previous question that you were dissatisfied or very dissatisfied with the Accreditation / Re-accreditation audit. Why have you chosen this answer?

__________________________________________________________________ What could TAC do to improve any aspect of the Accreditation / Re-accreditation audits that it conducts?

__________________________________________________________________


Strategic Industry Audit (SIA) Please rate TAC's performance when conducting the Strategic Industry Audit (SIA) on the following:

Strongly agree

Agree Part agree /

part disagree

Disagree

Strongly

disagree

Don't know

TAC informed us before and during the audit about the audit purpose and process

TAC informed us about the scope of the audit


TAC informed us of any changes to the audit timetable and reasons for changes

TAC audit staff gave my organisation an opportunity to discuss our particular business situation / needs


How much do you agree/disagree that the Strategic Industry Audit (SIA) added value to your business? Strongly agree Agree Part agree / part disagree Disagree Strongly disagree Don't know

You indicated in the previous question that the Strategic Industry Audit (SIA) audit added some degree of value to your business. Which types of value did it add? Select all that apply.

Provided my organisation with an opportunity to demonstrate compliance with the AQTF Informed my organisation's processes and policies Benchmarked my organisation with other RTOs Aided in my organisation's continuous improvement Other Please specify __________________________________________________________

Please rate the TAC audit team on the following :


Objective

Knowledgeable

Organised

Informative

Constructive

Non-threatening


Overall, rate your organisation's satisfaction with the TAC Strategic Industry Audit (SIA)? Very satisfied Satisfied Part satisfied / part dissatisfied Dissatisfied Very dissatisfied

You indicated in the previous question that you were dissatisfied or very dissatisfied with the Strategic Industry Audit (SIA). Why have you chosen this answer?

__________________________________________________________________ What could TAC do to improve any aspect of the Strategic Industry Audits (SIA) that it conducts?

__________________________________________________________________ Desk audit Please rate TAC's performance when conducting the desk audit on the following:

Strongly agree

Agree Part agree /

part disagree

Disagree Strongly disagree

Don't know







How much do you agree/disagree that the desk audit added value to your business? Strongly agree Agree Part agree / part disagree Disagree Strongly disagree Don't know

You indicated in the previous question that the desk audit audit added some degree of value to your business. Which types of value did it add?Select all that apply.

Provided my organisation with an opportunity to demonstrate compliance with the AQTF Informed my organisation's processes and policies Benchmarked my organisation with other RTOs


Identified my organisation's strengths Aided in my organisation's continuous improvement Other Please specify___________________________________________________________



Objective

Knowledgeable

Organised

Informative

Constructive

Non-threatening

Overall, rate your organisation's satisfaction with the TAC desk audit? Very satisfied Satisfied Part satisfied / part dissatisfied Dissatisfied Very dissatisfied

You indicated in the previous question that you were dissatisfied or very dissatisfied with the desk audit. Why have you chosen this answer?

__________________________________________________________________ What could TAC do to improve any aspect of the desk audits that it conducts?

__________________________________________________________________ Site audit Please rate TAC's performance when conducting the site audit on the following:

Strongly agree

Agree Part agree /

part disagree

Disagree Strongly disagree

Don't know








How much do you agree/disagree that the site audit added value to your business?

Strongly agree Agree Part agree / part disagree Disagree Strongly disagree Don't know

You indicated in the previous question that the site audit added some degree of value to your business. Which types of value did it add? Select all that apply.

Provided my organisation with an opportunity to demonstrate compliance with the AQTF Informed my organisation's processes and policies Benchmarked my organisation with other RTOs Identified my organisation's strengths Aided in my organisation's continuous improvement Other Please specify ___________________________________________________________



Objective

Knowledgeable

Organised

Informative

Constructive

Non-threatening

Overall, rate your organisation's satisfaction with the TAC site audit?

Very satisfied Satisfied Part satisfied / part dissatisfied Dissatisfied Very dissatisfied

You indicated in the previous question that you were dissatisfied or very dissatisfied with the site audit. Why have you chosen this answer?

__________________________________________________________________ What could TAC do to improve any aspect of the site audits that it conducts?

__________________________________________________________________ VET Sector To assist TAC plan future regulatory support services, from the list below which are the areas that you consider RTOs generally do NOT do well.You can select up to 3 items.

Continuous improvement of systems Adequate training is provided to learners


Assessments are in line with training packages Assessments meet workplace/regulatory requirements Recognition of Prior Learning (RPL) Appropriate length of courses Adequate staff numbers, facilities and equipment in relation to the training being provided Appropriately trained trainers and assessors Learner’s individual needs are met Employers and other parties are adequately engaged in client training and assessment Record keeping Complaints and appeals are addressed effectively Management systems are appropriate for the RTO Monitoring of training conducted on behalf of an RTO Accurate marketing of courses None of the above Don't know

Introduction of new RTO Standards in 2015 From your perspective as an RTO, please rate TAC‘s performance when introducing the new RTO standards earlier this year on the following aspects:

Excellent


Don’t know / not applica

ble

Provided enough information to my organisation about the new standards (through workshops, bulletins, emails, TAC Updates, etc)

Provided timely advice to my organisation about the new standards



Introduction of new RTO Standards in 2015 comments You indicated in one or more of your answers about the introduction of new RTO Standards in 2015 that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ TAC overall From your perspective, please rate TAC‘s performance on the following:

Excellent

Good

Fair

Poor

Very Poor

Don’t know








TAC overall comments You indicated in one or more of your answers that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s.

__________________________________________________________________ TAC is committed to reducing regulatory burden for high performing providers. What areas, if any, do you consider create unnecessary regulatory burden?

__________________________________________________________________ What does TAC do well that it should continue doing?

__________________________________________________________________ What does TAC need to improve?

__________________________________________________________________


TAC Stakeholder Survey 2015

Introduction As part of its improvement focus, the Training Accreditation Council (TAC) is seeking feedback from stakeholders within the Western Australia VET sector on your recent interactions with TAC. Feedback will be used to improve how TAC interacts with the industry as a whole.

TAC has engaged Australian Survey Research (ASR) as an independent research agency to conduct the survey on its behalf. The survey will take around 10 – 20 minutes to complete, depending on the types of interactions you may have had with TAC. Please complete by 9am, 9 June 2015. When answering all questions, please refer to the period from June 2014 until now. Your answers are confidential. At no time will TAC see your individual answers. ASR will only present summary results to TAC. Because the questions cover a range of topics and interactions, you may need to involve other people from your organisation in answering. Think of the link to the questionnaire as an organisational link which can be passed from person to person. However, keep in mind that once the questionnaire is finalised (submitted), the link will de-activate. ASR can re-activate the link at your request. Printing answers If you would like to print the questions and your answers, you can do this in two ways:

1. Page by page - use the print icon to the right of the top bar of the questionnaire.

2. All pages together - this is ONLY available once you have finalised (submitted) the questionnaire. A link will be available on the finalise page. Just click on it to commence the printing process.

Hints for answering If you get interrupted while answering, please click on the return later link at the bottom right of a page.

This will save you answers so far and close down the questionnaire. When you return using your unique link in your invitation email, you will be returned to the last page you

saved. Clicking on the word Next at the bottom of a screen will save your answers and move you to the next

page. If you cannot see this word on your screen, use your scroll bar to move down to it. Clicking on the word Previous at the bottom of a page will return you to the previous page. Please do not hit the back button on your browser as it may take you out of the questionnaire.

If you have any technical questions about the survey please contact Australian Survey Research on (03) 9578 5211 or 1800 068 489 or [email protected]. If you have any questions about the survey content or rationale, please contact Kieran Tynan at TAC on 08 9441 1917 or [email protected]. Interacting with TAC What types of interactions has your organisation had with TAC from June 2014 until now? Select all that apply.

Used the TAC website Received a TAC Update and/or TAC Special Bulletin (email newsletter) Used the TAC general email address / enquiry service - [email protected] Had a phone conversation with TAC staff Attended an event at which a TAC staff member spoke Attended a TAC audit as an observer or technical advisor Undertook moderation activities with TAC Met with TAC to discuss issues of VET regulation Other Please specify __________________________________________________________ No contact


TAC website Please rate the TAC website on the following:


Current/up to date information ™ ™ ™ ™ ™ ™

Accurate information ™ ™ ™ ™ ™ ™

Easy to understand information ™ ™ ™ ™ ™ ™

Provides enough information ™ ™ ™ ™ ™ ™

Easy to navigate ™ ™ ™ ™ ™ ™

Easy to search ™ ™ ™ ™ ™ ™

TAC website comments You indicated in one or more of your answers about the TAC website that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ TAC Update and/or Special Bulletin Please rate the TAC Update and/or Special Bulletin (both email newsletters) on the following:


Helpfulness of information ™ ™ ™ ™ ™ ™

Timeliness of information ™ ™ ™ ™ ™ ™

Informs on a wide range of issues ™ ™ ™ ™ ™ ™

TAC Update and/or Special Bulletin comments You indicated in one or more of your answers about TAC's Update that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ TAC general email service Approximately, how many times have you used the TAC general email service since June 2014?

™ 1-2 times ™ 3-5 times ™ 6-10 times ™ More than 10 times ™ Unsure / don't know

Overall, please rate TAC's general email service on the following:

Excellent

Good

Fair

Poor

Very poor

Don't know

Courtesy ™ ™ ™ ™ ™ ™

Respected confidentiality of organisation and privacy of ™ ™ ™ ™ ™ ™


Excellent

Good

Fair

Poor

Very poor

Don't know

individuals involved

Impartiality ™ ™ ™ ™ ™ ™

Helpfulness ™ ™ ™ ™ ™ ™

Clarity of response ™ ™ ™ ™ ™ ™

Efficiency of response (took minimal amount of words to respond) ™ ™ ™ ™ ™ ™

Promptness (speed of reply) ™ ™ ™ ™ ™ ™

Knowledge of staff answering ™ ™ ™ ™ ™ ™


™ ™ ™ ™ ™ ™

TAC email service comments You indicated in one or more of your answers about TAC's email service that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ TAC phone service Approximately, how many times have you phoned TAC since June 2014?

™ 1-2 times ™ 3-5 times ™ 6-10 times ™ More than 10 times ™ Don't know / not sure

Which area/s of TAC have you phoned since June 2014? Select all that apply.

Accreditation Registration (where Registration Officers / Case Managers are located) Administration Management Other (Complaints, Policy, etc) Don't know / not sure

Phone service ratings Please rate the phone conversations you had with TAC Accreditation staff on the following:

Excellent

Good

Fair

Poor

Very poor

Don't know



™ ™ ™ ™ ™ ™


Excellent

Good

Fair

Poor

Very poor

Don't know





™ ™ ™ ™ ™ ™



™ ™ ™ ™ ™ ™

Please rate the phone conversations you had with TAC Registration staff on the following:

Excellent

Good

Fair

Poor

Very poor

Don't know



™ ™ ™ ™ ™ ™





™ ™ ™ ™ ™ ™



™ ™ ™ ™ ™ ™

Please rate the phone conversations you had with TAC Administration staff on the following:

Excellent

Good

Fair

Poor

Very poor

Don't know



™ ™ ™ ™ ™ ™





Excellent

Good

Fair

Poor

Very poor

Don't know


™ ™ ™ ™ ™ ™



™ ™ ™ ™ ™ ™

Please rate the phone conversations you had with TAC Management on the following:

Excellent

Good

Fair

Poor

Very poor

Don't know



™ ™ ™ ™ ™ ™





™ ™ ™ ™ ™ ™



™ ™ ™ ™ ™ ™

Please rate the phone conversations you had with other TAC staff (not Accreditation, Registration, Administration or Management) on the following:

Excellent

Good

Fair

Poor

Very poor

Don't know



™ ™ ™ ™ ™ ™





™ ™ ™ ™ ™ ™



Excellent

Good

Fair

Poor

Very poor

Don't know


™ ™ ™ ™ ™ ™

TAC phone service comments - Accreditation You indicated in one or more of your answers about TAC's Accreditation phone service that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ TAC phone service comments - Registration You indicated in one or more of your answers about TAC's Registration phone service that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ TAC phone service comments - Administration You indicated in one or more of your answers about TAC's Administration phone service that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ TAC phone service comments - Management You indicated in one or more of your answers about TAC's Management phone service that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ TAC phone service comments - Other staff You indicated in one or more of your answers about TAC's other staff (not Accreditation, Registration, Administration or Management) phone service that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ TAC speech or presentation Please rate the information you received from a TAC staff member's speech or presentation on the following:


Timely (not outdated) information ™ ™ ™ ™ ™ ™

Useful information ™ ™ ™ ™ ™ ™

Insight that considered a wide range of issues ™ ™ ™ ™ ™ ™

TAC speech or presentation comments You indicated in one or more of your answers about a TAC staff member's speech or presentation that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________


TAC moderation activities Please rate the information you received from the TAC moderation activity/ies you were involved with.





TAC moderation activities comments You indicated in one or more of your answers about a TAC moderation activity that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ TAC VET regulation discussion Please rate the information you received when you met with TAC to discuss issues of VET regulation on the following:





TAC regulation comments You indicated in one or more of your answers about VET regulation issues that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ Technical advisor / observer Please rate your experience as a technical advisor and/or observer

Excellent

Good

Fair

Poor

Very poor

Don't know

TAC briefed me appropriately ™ ™ ™ ™ ™ ™

My presence added value / made a difference to the process

™ ™ ™ ™ ™ ™

I found the exercise useful ™ ™ ™ ™ ™ ™

If you have any comments to make about the technical advisor / observer role, please enter them here

__________________________________________________________________


Regulating the VET sector What is your awareness of TAC’s recent strategic industry audits?

Very aware (knew lots of detail) Aware (knew they were happening but that’s all) Not aware (knew nothing about them)

Please indicate your level of support for TAC’s continual auditing of strategic industry areas.

A great deal of support A lot of support A moderate amount of support A little support No support at all Don’t have a view / unsure

To assist TAC plan future regulatory support services, from the list below which are the areas that you consider RTOs generally do NOT do well. You can select up to 3 items.

Continuous improvement of systems Adequate training is provided to learners Assessments are in line with training packages Assessments meet workplace/regulatory requirements Recognition of Prior Learning (RPL) Appropriate length of courses Adequate staff numbers, facilities and equipment in relation to the training being provided Appropriately trained trainers and assessors Learner’s individual needs are met Employers and other parties are adequately engaged in client training and assessment Record keeping Complaints and appeals are addressed effectively Management systems are appropriate for the RTO Monitoring of training conducted on behalf of an RTO Accurate marketing of courses None of the above Don't know

What comments would you like to offer about the current registration officer / case management model of handling RTO enquiries? You may wish to comment about how well or not so well the service model works or alternative service models that would work better for RTOs.

__________________________________________________________________ Introduction of new RTO Standards in 2015 From your perspective as a stakeholder, please rate TAC‘s performance when introducing the new RTO standards earlier this year, on the following aspects:

Excellent

Good

Fair

Poor

Very

poor

Don’t know / not

applicable

Provided enough information to my organisation about the new standards (through workshops, bulletins, emails, TAC Updates, etc)

™ ™ ™ ™ ™ ™

Provided timely advice to my organisation about the new standards

™ ™ ™ ™ ™ ™



™ ™ ™ ™ ™ ™


™ ™ ™ ™ ™ ™

Introduction of new RTO Standards in 2015 comments You indicated in one or more of your answers about the introduction of new RTO Standards in 2015 that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s and which aspect in particular it applies to.

__________________________________________________________________ TAC overall From your perspective, please rate TAC‘s performance on the following:

Excellent

Good

Fair

Poor

Very Poor

Don’t know

Open to hearing concerns about the quality of VET outcomes ™ ™ ™ ™ ™ ™


™ ™ ™ ™ ™ ™


™ ™ ™ ™ ™ ™

Engages with stakeholders such as my organisation ™ ™ ™ ™ ™ ™

Seeks feedback from stakeholders ™ ™ ™ ™ ™ ™


™ ™ ™ ™ ™ ™

Acts on stakeholder feedback ™ ™ ™ ™ ™ ™

Acts on complaints received about training providers ™ ™ ™ ™ ™ ™

Acts on complaints received about its own performance ™ ™ ™ ™ ™ ™

Is transparent in its regulatory decisions and activities ™ ™ ™ ™ ™ ™

Applies consistent regulatory decisions ™ ™ ™ ™ ™ ™

Promotes and encourages continuous improvement of RTOs ™ ™ ™ ™ ™ ™

Improves the quality of VET outcomes in Western Australia ™ ™ ™ ™ ™ ™

Provides advice to industry bodies such as industry skills councils, other regulators and peak associations

™ ™ ™ ™ ™ ™


™ ™ ™ ™ ™ ™

Overall as a regulator ™ ™ ™ ™ ™ ™


TAC overall comments You indicated in one or more of your answers that TAC's performance was fair, poor or very poor. Please indicate why you have chosen this answer/s.

__________________________________________________________________ TAC is committed to reducing regulatory burden for high performing providers. What areas, if any, do you consider create unnecessary regulatory burden?

__________________________________________________________________ What does TAC do well that it should continue doing?

__________________________________________________________________ What does TAC need to improve?

__________________________________________________________________

tac surveys of rtos and stakeholders 2015 report of ... · tac rto and stakeholder survey 2015 |...

Documents