introduction to questionmark’s assessment management ... · introduction to questionmark’s...
TRANSCRIPT
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Introduction to Questionmark’sAssessment Management Technologies
Brian McNamara
Marketing Director, Questionmark
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Assessment Solutions enabled by Questionmark
Assessment Management System Collaborative Authoring
Demonstration
Blended Delivery
Demonstration
Reporting and Analytics
Demonstration
Integration
Q&A
Agenda
Slide 2
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Slide 3
To ask questions, use the “Q&A” feature
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
• Founded in 1988
• Offices in US, UK, Canada; Partners worldwide
• Assessment technologies and portal solutions that enable organizations to measure knowledge, skills, and attitudes securely for certification, regulatory compliance and improved learning outcomes.
Background
• Enterprise customers worldwide
• Corporate and Government (Learning/HR function); Higher Education; Awarding Bodies and Test Publishers
Customers
About Questionmark
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Assessment Management System
Collaborative, secure authoring of surveys, quizzes, tests, and examsReporting to analyze results and
inform stakeholders
Blended delivery to support a broad range of devices and security needs
Multilingual: 36 Languages
Global 24/7 Support
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
• Rapid deployment, turnkey-service
• Audited data centers in US and Europe
• Scalable for high-volume/peak delivery
Cloud/SaaS Installable Software
Deployment Options
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Measure and Enhance Learning
•Measure knowledge, skills and attitudes –before, during and after learning
•Correct misconceptions and increase knowledge retention
Qualification and Certification
•Deliver reliable, defensible tests to certify employees
•Ensure sound personnel decisions
Regulatory Compliance
•Ensure knowledge of essential laws, regulations and health/safety guidelines
•Demonstrate compliance and reduce risk
Sales/Technical Channel
•Competence/product expertise of sales and technical professionals
•Increase revenues, customer satisfaction
•Reduce costs
Credentialing
•Efficiently create and maintain valid, reliable tests
•Generate revenue from exams and practice tests
Higher Education
•Higher Education
•Increase student satisfaction
•Reduce cheating
•Improve learning
Why our customers use assessments
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
SAVE THE DATE!
March 21-24, 2016Inn and Spa at LorettoSanta Fe, New Mexico
Registration is open!
Questionmark Conference 2017Innovate. Inspire. Succeed.
Slide 9
"My team always walks away with new knowledge -and we have been using the product for over 12 years!"
www.questionmark.com/go/conference
March 21-24, 2017Inn and Spa at Loretto
Santa Fe, New Mexico
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Authoring and Item Banking
Slide 10
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Authoring module within Assessment Management System
Browser-based
Create items and assessments
Manage item bank
Import content
Publish assessments for delivery
Access via Questionmark website Ideal for SMEs / content harvesting
Browser based
Author items
Export as Qpack
Admin imports via Authoring Module
Authoring Options: Questionmark OnDemand
Slide 11
Authoring module Questionmark Live
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Windows-based application
Create items and assessments
Manage item bank
Access via Questionmark website Ideal for SMEs / content harvesting
Browser based
Author items
Export as Qpack
Admin then imports via Authoring Manager or Enterprise Manager
Authoring Options: Questionmark Perception and “Classic” OnDemand
Slide 12
Authoring Manager Questionmark Live
Not covered in today’s webinar
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Slide 13
Multiple choice Multiple Response
(Dichotomous) Multiple Response
(Score per choice) Matching Ranking Pull-down List Select-a-blank True/False Yes/No Text Match
Job Task Analysis Knowledge Matrix Survey Matrix Likert scale Drag-and-Drop Hotspot Numeric Essay Fill-in-the-blanks Matching File Upload Explanation screen
Many item types to choose from…
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Demonstration
Slide 14
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Provide users role-based access to content
Provide users role-based access to content
Roles & Permission Management
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Item version history and rollback
Slide 18
Revision History
Audit trail of edits and option to “rollback”
Easily “track changes” made in collaborative authoring or review
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Search description, stimulus, choices, status, meta tags,
dates and more
Item SearchSearch item bank to find items that contain certain words and/or tags
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Arabic ( (العربية
Bulgarian (Български) Catalan (Català) Chinese, Simplified (简体中文) Chinese, Traditional (繁體中文) Croatian (Hrvatski) Czech (Čeština) Danish (Dansk) Dutch (Nederlands) English Finnish (Suomi) French (Français) German (Deutsch) Greek (Ελληνικά) Hebrew ( (עברית
Hindi (हिन्दी)
Hungarian (Magyar) Indonesian (Bahasa Indonesia)
Italian (Italiano) Japanese (日本語) Korean (한국어) Norwegian Bokmål (Bokmål) Persian ( (فارسی
Polish (Polski) Portuguese, Portugal (Português) Romanian (Română) Russian (Русский) Serbian (Српски) Slovenian (Slovenščina) Spanish (Español) Swedish (Svenska) Thai (ภาษาไทย) Turkish (Türkçe) Ukrainian (Українська) Urdu ( (اردو
Vietnamese (Tiếng Việt)
Multilingual Interfaces
Slide 21
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Translation Management for Multilingual Assessments
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Blended, Secure Delivery
Slide 25
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Author Once… Deliver Anywhere
Slide 26
Single results set
Available Securely
Author once Deliver to any devicein context
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Responsive Design, Blended Delivery
One Assessment…
…many devicesand browsers
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Demonstration
Slide 29
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Print your own bubble sheets
Participants enter responses
Scan and upload for scoring
Analyze and report on results
Printing and Scanning:Deliver assessments online or on paper
Slide 31
Deliver assessments on paper or online Print your own question and bubble sheets Use your own scanner to read responses Upload scanned bubble sheets for scoring Scanned assessment results available for reporting
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Delivering high stakes assessments
Formative Quizzes;Course Evaluations
Diagnostic Tests
Low Stakes Tests
Medium Stakes Exams
High Stakes Exams
S e c u r i t y R e q u i r e m e n t s
Sta
ke
s o
f A
ss
es
sm
en
t
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
“Save as you go”
Results saved at the server “as you go” so IT failures don’t lose data
Hope for the best; prepare for the worst
Slide 33
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Secure browser for high-stakes assessments
Reduce content theft, cheating; improve exam integrity
Questionmark SecureOne Setting – Multiple Platforms
Slide 34
Windows Mac iPad
PrintingScreen captureTask switching
Control/system keysRight-click / context menu
optionsView source
Accidentally exiting exam
Prevents….
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Require Proctoring
Slide 35
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Convenience of online exam delivery at home or workplace under the watchful eye of a proctor
More info: Website
Your account manager
Online Proctoring
Slide 36
www.questionmark.com/proctoring
Powered by
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Accessibility
Also support for…- Non-mouse navigation; alternate pointing devices- Screen readers
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Launch and Track
• AICC
• SCORM
Single-Sign-On
• SAML
• LDAP
API / Data Integration
• QMWISe
• OData
• Results API
• Delivery API
Connectors for…
• SAP
• Blackboard
• LTI Connector
• Moodle / Angel / Canvas / D2L
• And more!
Embed in..
• Wikis
• Blogs
• Portals
• Websites
• SharePoint
Delivery via Integration
Slide 38
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Publish to LMS
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Reporting and Analytics
Slide 40
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Reporting and Analytics
Slide 41
What can we learn from assessments?
Did they like it?
Did they learn it?
Can they do it?
Can we prove it?
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
What you can learn from Questionmark reporting and analytics
How are our Items and Assessments doing? How are our Participants doing?
Did the learning investment impact
performance?Where are the skills and knowledge gaps in my
organization?
Are my questions and tests valid and reliable?
What did participants think of the learning
experience?
What did participants learn from the training
or course?
How do I coach participants to improve
performance?
How did one group of test takers compare to
another?
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Item and Test Analysis Item Analysis report Test Analysis report Score list report Test center analysis report Assessment completion time report Question frequency analysis report Demographic report Question type report Assessment Overview report Assessment results over time report
Participant-focused reports Score list report Coaching Report Transcript Report Grade Book Report
Survey Reports Survey Frequency Analysis Job-task-analysis (JTA)
JTA Summary JTA dimension by demographic
Course Evaluation Course Summary Report Instructor Summary Report Class Summary Report Class Detail Report
Types of reports available
Slide 43
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Score List Report
Slide 44
Demonstration
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
What it measures: % answered correctly
How it is expressed: Number from 0.0 to 1.0
How to interpret: Too high = too easy Too low could mean…
Too difficult
Confusing/ambiguous
Miscoded
What it measures: Correlation between
item score and test score
How it is expressed: Number between
-1.0 and +1.0.
How to interpret: Generally…
< 0.2 low
> 0.3 good
Item Analysis
Difficulty index (aka P “value”) Item correlation discrimination:
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Slide 46
Difficulty/P-Value
Correlation -Discrimination
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Slide 47
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.Slide 48
Coaching ReportCompare against
benchmarks or other participants
Participant
Topiclevel
Overall
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Job Task Analysis (JTA) Reports
Slide 49
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Course SummaryInstructor Summary
Class Summary Class Detail
Four Course Evaluation Reports:
Slide 50
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Built-in Reports Custom Reports and Dashboards
Microsoft ExcelPowerPivot
BI Tools
Big Data Applications: Results API
Slide 51
R e s u l t s W a r e h o u s e
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
•Planning
• Installation
• Integration
Deployment
•Assessments
• Importing items
•Templates / Look & feel
Authoring
• Item reports
•Other custom reports
Reporting
•Testing and Assessment Portals
•People Data syncing
•Results Data syncing
•Self Registration
•Auto Scheduling
•Assessment Clustering
•Certificate Generation
•Distributed Administration
•Assessment Catalogs
Solutions & Integration
Consulting / Solution Services
Slide 52
w w w. q u e s t i o n m a r k . c o m / c o n s u l t i n g
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Summary of key benefits
Collaborative Authoring
• Secure
• Browser-based
• Available globally
Blended Delivery
• Responsive Design
• Browser support
• Tablets/smartphones
• Accessibility
• Secure delivery for high stakes exams
Reporting and analytics
• How are my participants doing?
• How are my tests doing?
• Results API
Multilingual
• Translation Management
• Screens/Controls in 36 languages
• Right-to-left languages
Integration
• Supports industry standards
• APIs
• Connectors
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
What questions do you have?
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
How to Evaluate• Request a one-on-one demo
The Questionmark team will contact you to arrange a demonstration tailored to your needs and questionswww.questionmark.com/go/request-demo-enus
• Request a 30-day trial of Questionmark OnDemandComplimentary technical support during trialwww.questionmark.com/go/od30us
Slide 56
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark.All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
www.questionmark.com/go/whitepapers
Recommended White Papers
Slide 57
Copyright © 1995-2017 Questionmark Corporation and/or Questionmark Computing Limited, known collectively as Questionmark. All rights reserved. Questionmark is a registered trademark of Questionmark Computing Limited. All other trademarks are acknowledged.
Follow us…
Thank you for attending!
We hope to see you at a future webinarKeep up to date a blog.questionmark.com!