airs certification program

25
AIRS Certification Program Computer-Based Testing (CBT) AIRS Webinar: October 2013

Upload: kyria

Post on 25-Feb-2016

68 views

Category:

Documents


3 download

DESCRIPTION

AIRS Certification Program. Computer-Based Testing (CBT) AIRS Webinar: October 2013. The Original Way. AIRS Certification has been around for over 15 years Eligible candidates take a 100 question exam and acquire either CIRS, CIRS-A or CRS Certification if they pass the exam - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: AIRS Certification Program

AIRS Certification Program

Computer-Based Testing (CBT)

AIRS Webinar: October 2013

Page 2: AIRS Certification Program

The Original Way

• AIRS Certification has been around for over 15 years

• Eligible candidates take a 100 question exam and acquire either CIRS, CIRS-A or CRS Certification if they pass the exam

• The exams have always been proctored with testing taking place in classroom style settings

Page 3: AIRS Certification Program

The Original Way

• And the exam answers have always been entered and marked using Scantron forms

Page 4: AIRS Certification Program

Problems with the Original Way

• Scantron forms are not the friendliest pieces of paper. It is a strain on the eyes and requires a reasonably intense focus

• Although machine readable, AIRS Certification staff often hand-scored exams if the score was within 5 of a passing mark or it looked like a candidate had made an honest mistake in alignment

• The exam questions themselves were always at risk as boxes of question papers were sent around the US and Canada, then returned

Page 5: AIRS Certification Program

Problems with the Original Way• Providing timely results to candidates was also a

constant challenge – we had to wait for the boxes to be returned, retrieve the forms, feed them through the machine, verify the results – and during busy times, there would be boxes going in and out every day, and if one person was off for a couple of days, then the backlog would build up, etc. etc. …

• Candidates wanted their results within a few days. Our target was 30 days and sometimes, we were not even able to meet that …

Page 6: AIRS Certification Program

First Look at Computer-Based Testing

• CBT has been on the AIRS radar for about 6 years• Original concerns were:– Expense to members (costs of up to $100 per person)– Lack of location choices (most operated through commercial

testing centers – less options, even more expense)– Concern of impact on Affiliate conferences– Sense that not everyone was ready for the switch– AIRS Certification had other priority (from 2008 to 2012, all

AIRS Certifications underwent stringent psychometric review with new questions and new exams)

Page 7: AIRS Certification Program

Second Look at Computer-Based Testing

• More software options with lower pricing and more flexible options

• More unease about Scantron as suitable for persons with visual impairments

• Survey of AIRS members in 2012 showed that 40% of members had a training room with computers/Internet

Page 8: AIRS Certification Program
Page 9: AIRS Certification Program

Second Look at Computer-Based Testing

• Do you know of a 'training-type room' within your community (for example, an employment center, a library, community college) that has a number of Internet-capable computers that could be easily accessed for a CBT Certification exam?

• Yes, and I am aware that either my own or other agencies have used this space for various reasons = 39%

• Yes, I am aware of such a place but I have never had cause to contact them = 33%

• No, but I imagine such a space exists and obtaining it should not be an obstacle = 15%

• No, this may be a problem = 13%

Page 10: AIRS Certification Program

Second Look at Computer-Based Testing

• Which of the following best describes your feelings towards Computer-based testing?

• I fully welcome it = 69%• I welcome it but have some concerns= 27%• I am concerned about it but appreciate there are some

benefits= 10%• I prefer the existing system = 4%

Page 11: AIRS Certification Program

Conclusion: Let’s Make the Move to Computer-Based Testing

Page 12: AIRS Certification Program

Software Selection

• After testing several CBT softwares, AIRS chose an application called Questionmark

• What we liked about it:– Admin end of it was relatively easy to use– Existing clients included US military, major

technology companies, financial services, government

– Price … honestly, it was the only one that would not have driven up the cost of Certification for AIRS members

Page 13: AIRS Certification Program

Software Selection

• Over past 12 months:– Loading of 800+ questions each with 4 option

choices– Testing of software by AIRS Certification

Commission members, volunteer agencies, Affiliates

– Framing and refining of documentation– More testing– Bit more testing

Page 14: AIRS Certification Program

Operating for past few months on Stealth Mode

Page 15: AIRS Certification Program

But now CBT is the only option for AIRS Certification exams …

BUT THERE ARE OPTIONS WITHIN THAT OPTION …

Page 16: AIRS Certification Program

CBT: What stays the same

• This is not a low stakes Certification exam. There is still a requirement for a proctor

• The application process to be approved as a candidate to take an AIRS Certification exam remains the same

• AIRS Affiliates still need to approve tests taking place in their state/regions

• The ‘atmosphere’ and ‘general expectations’ of an exam environment remain the same

Page 17: AIRS Certification Program

CBT: What is different

• An organization wanting to host an exam must fulfill more detailed criteria (basically, a set number of seats with reliable computers and Internet access with technical backup on hand for start of exam)

• The proctor’s environment changes and proctor must enter a password before each candidate can start their exam

Page 18: AIRS Certification Program

CBT: What is different

• The candidate experience is much improved:– There is one question per page. It is much easier

to read the question and answer options– Easy to leave questions unanswered and go back

to them later or to answer them and ‘flag’ them for later review

– You get the results within 5 business days

Page 19: AIRS Certification Program

Three Exam Options

– An exam hosted by an AIRS member (either at their own facility or at a community-based one)

– An exam proctored by a college or university

– An exam conducted by an online proctor via a webcam at home or at work

Page 20: AIRS Certification Program

Hosted by AIRS member

• Host must complete an application form (found via www.airs.org/certification)

• Requirements include:– Facility with computers/laptops and Internet– In a quiet setting– Securing a proctor– Having some technical expertise on hand on the day of

the exam just in case• No additional cost unless required by the host site to

cover specific costs

Page 21: AIRS Certification Program

Hosted by College/University

• Exam can be taken at certain colleges in the United States (along with a few in Canada) that are members of the National College Testing Association (NCTA at www.ncta-testing.org

• Find an institution near you and go through the required steps

• Additional cost usually of around $25 to $30 per person that is paid directly to the college

Page 22: AIRS Certification Program

Hosted by Webcam-based Proctor

• AIRS working with company called ProctorU• Allows exam to be taken at home or at work (but in

quiet/private setting) as long as you have webcam and secure Internet

• Cost is $25 but provides flexibility and no travel costs

Page 23: AIRS Certification Program

So What Does The Exam Look Like??

• https://ondemand.questionmark.com/delivery/perception.php?customerid=399354

Page 24: AIRS Certification Program

AIRS CBT: First Reports

• The work involved in hosting an exam is a bit more involved for the AIRS member organization but (as with all other things), it will seem a lot simpler the second and third times around

• The feedback from candidate has been very positive. It makes the exam process so much easier

• However, we are working with technology and although it has not happened yet, something somewhere will go wrong – be patient and we will try our best to fix it

Page 25: AIRS Certification Program

Questions

• How come … ?• What about … ?• So what if … ?

And when you are ready, go to www.airs.org/certification

If stuck or confused, email [email protected]