using open badges as a certification solution for evaluators

Using Open Badges as a Certification Solution for Evaluators

Randall S. DaviesDaniel L. RandallRichard E. West

BYUImage by photosteve101. Used under CC BY License.

The Issue of Evaluator Certification

In 1999 the topic of certification was fervently debated

Not everyone feels the AEA needs an evaluator certification

Others feel a certification process would be valuable, and that it may become required for practitioners working with government organizations (Smith et al., 2011)

AEA is no closer to having an evaluator certification than we were a decade ago. Many seem to have concluded that an evaluator certification system may be overly complicated and controversial (Morris, 2011)

AEA

Challenges to Certification

Credentials, Certifications, and Licenses

An Evaluator Certification would evaluate an individual’s Knowledge, Skills, and Experience, verifying that he or she has attained a certain level of expertise.

Any solution AEA might adopt must be– Transparent (Clear Expectations)– Rigorous BUT Flexible and Efficient– Sustainable

AEA

Working Examples of an Evaluator Credentialing Designation

Canadian Evaluation Society’s (CES)

Credentialed Evaluator (CE) designation

This is a designation is not a certification of evaluation proficiency or a license to practice evaluation.

It simply verifies “the holder has provided evidence of the education and experience required by the CES to be a competent evaluator” (http://www.evaluationcanada.ca, np).

AEA

The CE designation is voluntary.

Candidates must have at least two years of evaluation-related work experience within the last ten years, with letters of reference from clients.

They must pass a test requiring a series of narratives (each 150 words or less) adequately addressing at least 70% of the specified evaluator competencies in each of five domains.

AEA

CES Credentialed Evaluator Designation

CES Credentialed Evaluator Designation, Cont.

Members of a credentialing board consider the application and award the designation.

In addition, credentialed evaluators are required to complete ongoing training (40 hours every three years) in order to maintain their status.

This application and renewal process is implemented though an online system designed to manage applications and candidate accounts, with a fee to cover administration costs.

AEA

Issues & Concerns AEA Must Consider

Training Concerns:– Degrees vs training, Experience without a degree

(capacity building). What combination of training, Skill, and Experience is needed.

Assessment Concerns:– Certification Exams vs Credentials, Validity of

assessment.

Administration Issues:– Costs involved in establishing and maintaining a

systemAEA

Potential for an Open Badges Solution

Transparency:– Badges communicate specifics, Meta-data,

Flexibility:– Modification is relatively easy, Skill subsets can be defined, badges

can be added or revised individually.

Rigor:– Individuals Authorized to Award Badges, Allows for levels of

proficiency to be identified (basic – advanced)

Management Sustainability:– Top down design, Automation of much of the process possible,

Division of labor model

AEA

What are “Open Badges”?

Images from OpenBadges.org

Merit Badges and Digital Badges

Boy Scout Merit Badges

Digital Badge

- Acknowledge accomplishment- Display skills gained- Motivation

Same Benefits as physical badges -Typically not sharable -

Gamification -

Khan Academy Badges (not Open!)

Retrieved from https://www.khanacademy.org/badgeson 11/30/13.

Duolingo (not Open Badges)

Screenshot from the Duolingo app on an Android Tablet. 1/24/14.

Encourages daily practice

Completed badge

Progress indicators

Competition with others is optional

Previously earned badge not at full strength

Open Badges

Open Badges

Same Affordances as Digital Badges, Plus: - Uses Open Badge Infrastructure (OBI)- Display badges via web- Metadata (Criteria and Evidence links)

Open Badge Metadata

Mozilla Backpack

Backpack Collection

Multiple collections can be created.

Collections can remain private or can be made public and shared.

Mozilla’s Vision of Credentialing

Learn and gain skills in formal and informal settings

Collect and Display Badges

New Opportunities

Applications

Gamification: breaking education into achievable levels where progress is recognized and rewarded

Duolingo, Khan Academy, many others

Recognizing expertise within a community Coding communities, translation communities

Marrying informal and formal learningConcordia, WGU

Providing more authentic credentials, tying evidence to criteria to credential

IPT Ed Tec

Strong criteria• A badge’s value is based on the rigor of the criteria for

earning itEasy submission

• Make it simple for people who deserve a badge to get itWeb-based evidence

• Portfolios, links, Google documents, etc. Allows for evidence to be viewed with a badge

Skilled assessments• It is essential to trust the person making the assessment

Open and shareable badge collections• What’s the point if you can’t share your badges in multiple

ways?

Key Principles of Badging

iMovie PersonalTech

TechnologyIntegration

One Possibility for an AEA Badge SystemA E A

Mozilla’s Webmaker badges plan used the “constellation” concept in which smaller

badges are earned and added together to receive a higher level badge.

Badges image adapted from the Mozilla Open Badges Project. Retrieved from http://erinknight.com/post/29830945702/webmaker-badges on 11/9/12.

Early Badge System by Mozilla

Google Sites

PersonalTech

Choice1

Choice2

Choice3

IPT EdTec Badge System

Student selected Internet Communication Technology

Student selected Multimedia Technology

Student selectedPersonal Technologies

AdditionalConcepts

MobileLearn

Internet Safety

Copy right

iMovie

Lower level badges are not issued for these projects

Project level badge not issued for these

additional concepts

Educational Technology

Course Level Badge

Project Level Badge

Lower Level Badges

iMovie PersonalTech

First Iteration

Theory

Skill

Experience

TechnologyIntegration

Certified EvaluatorLevel Badge

2ndLevel Badge

3rdLevel Badges

Possible AEA Badge System

Anatomy of a CE Badge

Pips act as stars:1 = Basic 2 = Intermediate3 = Advanced

Outer ribbon signifies this badge

is the “Certified Evaluator” (CE)

badge

Each badge indicates what category

the badge is from (theory, experience, skill, or

green for the CE badge). Center of badge is reserved for

images to individualize

badges

Pips and the CE Badge

7 - 9 pips earned from 2nd level

Basic

Intermediate

Advanced

4 - 6 pips earned from 2nd level

At least 1 pip from each category

Randall, D. L., Harrison, J. B., West, R.E. (2013). Giving credit where credit is

due: Designing Open Badges for a technology integration course.

TechTrends, 57(6), 88-95. doi:10.1007/s11528-013-0706-5

View our presentation at the Open Education Conference by clicking here.

To Learn More…

Randall S. Davies – randy.davies@byu.edu

Daniel L. Randall – dan.randall26@gmail.com

Richard E. West – rickwest@byu.edu

Contact us with Questions

Thank You