how to effectively use esc works - edtechcdn.ericsilva.me · joshua essary & eric a. silva...

Joshua Essary & Eric A. Silva Spring 2018 | EDTECH 503: Instructional Design

How to Effectively Use ESC Works A Data and Event Entry System

HOW TO EFFECTIVELY USE ESC WORKS SPRING 2018

JOSHUA ESSARY & ERIC A. SILVA 1

Table of Contents Topic ................................................................................................................................................ 3

Learning Goal .............................................................................................................................. 3

Audience Description .................................................................................................................. 3

Analysis ........................................................................................................................................... 4

Needs Assessment Survey .......................................................................................................... 4

Needs Assessment Data .............................................................................................................. 5

Learner Needs and Characteristics ........................................................................................... 10

Learning Context ....................................................................................................................... 13

Physical Characteristics ......................................................................................................... 13

Personnel or Time Constraints.............................................................................................. 15

Compatibility or Learner Needs ............................................................................................ 15

Performance Context ................................................................................................................ 15

Content Analysis/Flow Diagram................................................................................................ 17

Planning......................................................................................................................................... 18

Rationale ................................................................................................................................... 18

Learning Objectives ................................................................................................................... 19

Objectives Matrix Table ............................................................................................................ 20

ARCS Table ................................................................................................................................ 21

Attention ............................................................................................................................... 21

Relevance .............................................................................................................................. 21

Confidence ............................................................................................................................ 22

Satisfaction ............................................................................................................................ 22

Instructor Guide ........................................................................................................................ 23

Introduction - (20 Minutes) .................................................................................................. 23

Body - (1 hour) ...................................................................................................................... 23

Conclusion - (10 Minutes) ..................................................................................................... 25

Learner Content ........................................................................................................................ 25



Learning Materials ................................................................................................................ 25

Assessment Materials ........................................................................................................... 26

Technologies Used .................................................................................................................... 26

Evaluation ..................................................................................................................................... 27

Evaluation Plan.......................................................................................................................... 27

Subject Matter Expert ............................................................................................................... 28

Evaluation Rubric ...................................................................................................................... 28

Analysis ................................................................................................................................. 28

Design .................................................................................................................................... 29

Development......................................................................................................................... 29

SME Report and Review ............................................................................................................ 30

Analysis ..................................................................................................................................... 30

Design ........................................................................................................................................ 31

Development............................................................................................................................. 31

Comments and Suggestions ...................................................................................................... 32

Appendices .................................................................................................................................... 34

Appendix A: Student Survey ..................................................................................................... 34

Appendix B: Handouts .............................................................................................................. 35

Appendix C: Performance Tasks Group Activity ....................................................................... 36

Part A: Creating Titles ........................................................................................................... 36

Part B: Creating Sessions....................................................................................................... 36

Appendix D: Performance Tasks Rubric .................................................................................... 37

Appendix E: Summative Assessment ........................................................................................ 39

Appendix F: Summative Assessment Rubric ............................................................................. 40



Topic

Learning Goal

The learners will create a deeper understanding of the proprietary data entry system,

ESC (Education Service Center) Works, along with best practices in order to create sessions

without error or hesitation.

Audience Description

There are 221 employees at the Education Service Center. However, the majority of

events that are entered and created in ESC works originates from one department, General

Education. There are 15 employees in this department. In regard to the highest degree held,

43% hold master’s degrees and 47% hold bachelor’s degrees. The ages from 27-58. 93% of the

surveyed audience is female. 88.3% identify as white, 10.1 identify as Hispanic, and 1% identify

as Hispanic/Indian American. Most employees recognize as being moderately comfortable with

technology systems and in their ability to utilize data entry systems. The longevity of employees

ranges from less than one year to over 16 years of experience.



Analysis

Needs Assessment Survey

Upon Joshua taking the position as an internal trainer for the Service Center, he was

informed of three immediate training concerns. One concern was the proper use and efficient

training of the data and event entry system, ESC Works. Administration conveyed frustrations

concerning the ambiguity of the system and the lack of mastery from employees. Resources

were being excessively used to train employees multiple times within the year. Even though the

system had not changed and neither had the information offered in the training sessions. The

proper use of ESC Works is essential to the Service Center. It provides data on all customer

interactions, events created, customers attended, targeted goals, and much more. According to

registration, there are weekly, and sometimes daily errors in the system. This requires service

center resources to fix errors, contact customers, issue refunds, and sometimes cancel revenue-

rich events. After verbally contacting known system strugglers, a survey was created to target

the most used areas of ESC Works, and then the most questioned fields and data points.

The Education Specialists from the General Education department were emailed the

electronic survey. 13 out of 15 completed the survey asynchronously on their own time. The

survey was structured to provide anonymity to gather better demographic information that

could better focus the instructional design. The questions and results can be accessed at

http://bit.ly/ESCWneeds.

http://bit.ly/ESCWneeds



Needs Assessment Data

The ESC Works data and event entry system is a content-rich system. There seems to be

many data entry points as well as specific rules for different types of information. We can infer,

and use data from Question 1.4, that the most challenging aspect of the system is the type of

information that needs to be entered.

Figure 1.4

The answers all appear different. However, nitpicking information, non-intuitive, the

number of entry boxes, remembering details, and best practices can all be attributed to the

clarity of data to be entered. When using other data, we can infer that knowing the correct

information to enter for various situations would increase the comfort level with the users.

The following data points serve to determine specific areas of weakness in learner

knowledge of the system. Participants were asked to rate their skills on a scale of knowing only

the fundaments to being an expert. It is evident there is a small group of learners that know the



system well and self-assess as experts. It is the drop in “expert” rating that aid in highlighting

areas in need of addressing.

Figure 1.1.5

Question 1.1.5 showed the most significant drop in “Expert” ratings. This brings up more

questions than answers. The “Keyword” entry section has the smallest entry area and is not

required for event creation. This could possibly be due to best practices. The learners

understand what can be entered into the keyword field, but they are unsure as to what types of

words or how many. The question was engineered to target best practice by asking about the

“use of keywords” instead of the act of entering.



Question 1.1.8 is designed to determine the comfort level of entering information

regarding T-TESS and T-PESS information. These refer to the Texas Education Agencies educator

and administrator evaluation system.

Figure 1.1.8

Not only do we see a drop in the level of experts, but we also see a rise in the

fundamental field. There are only four questions in which this occurred, so we believe it is

important to note. These fields are primarily left unused due to the lack of knowledge

concerning these data fields. These fields are used for targeting customers.



Question 1.2.14 also showed a spike in the intermediate and novice numbers due to a

drop in the expert categories.

Figure 1.2.14

This data is critical to note because it is one of the few fields that requires the user to

navigate to a separate sub-system of the program. The budget numbers also have to be

entered into the sub-system, but budgeting (Question 1.2.15) did not show any increase or

decrease in the average reporting numbers.



The learners were also given an opportunity to respond in short answer form in order to

cover areas that were missed. Question 1.3 allowed learners to communicate the aspects of the

system they felt were the most important.

Figure 1.3

“Keywords” and “Descriptions” show up two times. However, the information may not

be significant. The fifth entry down is interesting as it is a suggestion for an entirely new

workflow. This will be investigated as a possible solution for room scheduling issues as noted

from Question 1.2.14.



Learner Needs and Characteristics

The first eight questions of the survey served to determine basic demographics. This

data is helpful to infer and determine any trends. The most important trends seem to come

from work experience and system experience. See figure 2.1.

Figure 2.1

This information shows that 64% of surveyed employees have only had five years or less

work experience at the service center. This information is different from the system use. These

data points are different due to the fact that the ESC Works system is used at other service

centers. If employees worked at another service center and then transferred, he/she could be

familiar with the system even though they are a new employee. It is essential to design

instruction that allows employees to show their knowledge of the system beforehand since

35% of the employees have six or more years of experience.



Question 2.2 raised some interesting questions. One-third of the employees have been

working at the center for over six years. However, the ESC works data only showed a maximum

of 5 years of usage. Upon receiving the data, we were able to confirm the system has only been

in use for the last five years. This data was helpful in determining this fact since it was not

divulged initially.

Figure 2.2

Questions 2.3 and 2.4 were utilized to determine the self-assessment level of

technology comfort for the user. Before planning any instruction, it is necessary to determine

this information when considering virtual or blended instructions. This information is also

necessary for determining a starting point for instruction. If the learners all consider themselves

beginners, then we could determine that the problem could be more knowledge-based. We can

see from both results the learners are most comfortable using technology and creating sessions

within the system. We can infer from the data though that there is a gap of instructions in the

system creation considering 45% self-assessed an intermediate level.



Figure 2.3

Figure 2.4



Question 2.5 was designed to unmask any unsubstantiated comfort the user may have.

Even though the form is anonymous, some users still report rating themselves higher than

necessary. Question 2.5 asks how comfortable the user would be in training others on the

system. The data, however, did not skew as predicted. The same number of users that self-

assessed as an expert also stated they would be extremely comfortable instructing others in the

system. This data serves to reinforce that there is a significant number of users that are

confident with the system and its operation.

Figure 2.5

The complete raw data can be viewed online via Google Sheets.

Learning Context

Physical Characteristics

The majority of employees are located in the Service Center building. 25 employees

work off-site, but they are not required to use the training system. The building is large with

https://docs.google.com/spreadsheets/d/1ImSJuFvnDJA5qOQXCp1Q5honSMsHvhTeZTyL00eyk8I/edit?usp=sharing



many physical classrooms. The center’s purpose is for professional development, so the rooms

are equipped appropriately for information distribution. There are 21 rooms. All rooms can

hold a minimum of 49 people. Each room is equipped with high-speed wireless internet,

document cameras, surround audio, projector screen, desktop computer, presentation podium,

microphone, and multiple visual inputs for the projector.

The employees, when they are not delivering professional development, reside in the

employee work annex. This area mimics most office cubicle environments. There are six areas

which have been opened, fewer walls, to make a more collaborative environment. This area is

still an open-air workspace. The walls are 5 feet tall with 12-foot ceilings. This design still allows

for sound travel. It is easy to distract other workers when three or more people are working

together. Listening to audio through speakers is generally not acceptable as part of the

unwritten workplace code of ethics. This annex also contains three meeting rooms. One room is

equipped with seating for 8, a monitor, web conferencing equipment, and computer with

peripherals. The other two rooms within the annex contain seating for 20, but there is not any

integrated technology equipment. All rooms within the annex also have access to high speeds

wireless internet and physical, portable whiteboards.

The service center also has a cooperate with the local university at an innovation and

research facility. The service center uses three classrooms at this location. There is a monitor,

audio, flexible seating for up to 50 and high-speed internet.



Personnel or Time Constraints

Most of the employees are designated to work at the service center. However,

customer contracts require specialists to visit off-site to provide services. All specialists serve

multiple customers throughout the year. Most employees are rarely at the service center at the

same time. There are two designated Fridays per month which all specialists are required to

work at the service center. These days are typically used for company-wide meetings,

department meetings, planning, collaboration, and training. After speaking with the employees

and comparing calendars, there is rarely a day that three randomly chosen employees are at

the service center, on the same day, for the same amount of time. This would lend itself to

synchronous small group instruction paired with asynchronous digital instruction.

Compatibility or Learner Needs

All employees have access to a Dell laptop and an Acer Chromebook. These devices have

access to high Ethernet and wireless connections. The service center utilizes Google enterprise

suite and subscribes to web conferencing service that is accessible to all employees. The

physical space can be utilized in small groups for face-to-face delivery. Since the employees

have access to technology and internet, they will be able to access training materials in an

online learning environment. Some employees voiced concerns about training at the service

center because of other employees interrupting for unrelated issues. For this reason, it might

also be prudent to use the off-site location at the local university.

Performance Context



The learners, or specialists, will use the skills acquired in various settings and

environments. Since many of the specialists are fulfilling off-site contracts, they find themselves

fulfilling work requirements while away from work. This could be at another customer’s district,

or it could be at the employee’s home.

When the specialists use the system to enter events at the service center, the

performance context will be different. The employees will have access to a personal dedicated

workspace with access to dual monitors. However, when employees use the system off-site,

they will not be working in that physical environment. There are endless possibilities for a

personal workspace away from the service center. All employees regardless of location will

have access to a computer and an internet connection.

It is important to note, that all employees are very connected during work hours.

Learners have multiple methods of contact. Because of this, learners will have opportunities to

use colleagues as resources or assistance. It would be important to utilize this connection since

learners will be accustomed to connecting.



Content Analysis/Flow Diagram

Learning Goal: The learners will create a deeper understanding of the proprietary data entry system, ESC (Education Service Center) Works, along with best practices in order to create sessions without error or

hesitation.

Goal 1: Learners will understand how to navigate

the ESC Works user-interface.

1.1 Participants will learn the different pages, page types, buttons, terminology, and

commonly used areas.

Goal 2: Learners will recognize and utilize best practices in creating titles in ESC Works.

2.1 Participants will learn techniques to create effective session titles and

descriptions by having an instructor give examples of good and bad session titles.

2.2 Learners will learn the need for providing additional information such as keywords, subject, audience, and T-PESS fields by viewing a short presentation of

how these affect the customer experience.

2.3 Learners will be given the opportunity to practice entering data

into ESC Works using sample data which will be reviewed by the facilitator.

Goal 3: Learners will recognize and utilize best practices in

creating sessions in ESC Works.

3.1 Participants will learn the fundamentals of the "Create Session"

screen so they have an understanding of all the required information and why it is

necessary.

3.2 An instructor will give a short presentation that gives an in-depth look at a few specific fields that require more

detailed information.

3.3 Learners will be given a list of sample sessions that they will then enter into

ESC Works to test their knowledge and understanding of the software.



Planning

Rationale

All employees of the Education Service Center (ESC) Region 12 are required to use a

data entry system called ESC (Education Service Center) Works. The use of this system is

imperative to the day to day operations, near-term planning, and long-term scheduling. Using

interviews and objective data, the lack of mastery of the system is evident. One of the primary

functions ESC Works is to create professional development sessions. These sessions use

multiple data points, and when entered, is automatically generated to a catalog for marketing.

Inaccurate and missing information is detrimental to the customer’s understanding of the

products and offerings. This, in turn, wastes time, money, and productivity for the ESC Region

12.

Employees are currently learning the system in various, non-uniform, and unfocused

teaching methods. There are also essential tasks in the system that are learned by word of

mouth and trial-and-error. These methods are inconsistent and allow for multiple skills gaps.

These gaps slow down productivity and effect long-range planning for the Service Center. Using

data from surveys and interviews, one of the most significant holes is in the understanding of

the user-interface (UI). Acclimating to the UI should be the first step in mastering ESC Works.

Learning software is a strict and definite process. There can be multiple tasks in a

system. However, whatever the tasks, the systematic procedure of buttons to click remains the

same with few changes. This information necessitates a mostly behaviorist approach blended



with some constructivist points. Due to the variety of tasks, based on the employee’s job title,

the constructivist methods will allow these people to create a personal and relevant

understanding. This knowledge, since personally connected, will allow for a more extended

retainment of knowledge and therefore require less revisiting and reteaching.

This instructional design will require mostly supplantive scaffolds. However, the

following skills will need to use a blend of generative and modeling. The learners, usually taught

when new to the company, typically have low aptitude, are short on time, and have higher than

normal anxiety considering they are starting a new job. This combined with the complexity of

the skills lends itself to the use of modeling. The specific job tasks related to a title creation

requires previous knowledge and understanding of the UI. These tasks can therefore use

generative scaffolding.

Learning Objectives

1. Learners will understand how to navigate the ESC Works user-interface without the need of

the manual.

1.1. Participants will identify the different pages, page types, buttons, terminology, and

commonly used areas of ESC Works.

2. Learners will recognize and utilize best practices in creating titles in ESC Works.

2.1. Participants will recognize techniques to create effective session titles and descriptions

by having an instructor give examples of good and bad session titles.



2.2. Learners will discover the need for providing additional information such as keywords,

subject, audience, and T-PESS fields by viewing a short presentation of how these affect

the customer experience.

2.3. Learners will apply steps taught to create a sample session in ESC Works without error.

3. Learners will recognize and utilize best practices in creating sessions in ESC Works.

3.1. Participants will learn the fundamentals of the "Create Session" screen so they have an

understanding of all the required information and why it is necessary.

3.2. Learners will be given a list of sample sessions that they will then enter into ESC Works

to demonstrate their knowledge and understanding of the software.

Objectives Matrix Table

Objective

Number

Revised Bloom’s Taxonomy

Classification

Scaffolding

Strategy Type of Assessment

1. Understanding Supplantive Discussion

1.1. Applying Supplantive Performance Task

2. Applying Generative Performance Task

2.1. Understanding Supplantive Performance Task


2.3. Applying Generative Summative

Assessment

3. Applying Generative Performance Task


3.2 Applying Generative Summative

Assessment



ARCS Table

Attention

A1. Perceptual

Arousal

Capture Learners Interest by:

• Showing the number of incorrect entries made in a week,

month, and year.

• Explaining how cost is related to session creation.

• Show aspects of ESC Region 12 which relies on accurate data

entry in ESC Works.

• Explain how many departments use the system.

A2. Inquiry

Arousal

Stimulate an aptitude of inquiry by:

• Act as a customer and find specific events and sessions using a

specific criterion to understand the difficulty from the

customer’s point-of-view.

• Showing an example of blatant ambiguity from the system and

having learners discuss what their instinct would be without any

knowledge.

A3. Variability Maintain learner’s attention by:

• Using direct teaching methods.

• Engaging them in discussions.

• Delivering information through static and dynamic visuals.

Relevance

R1. Goal

Orientation

Identify and meet the learners’ needs by:

• Showing the learners’ statistics of gaps of knowledge of current

employees. Show that there’s a need.

• Explain system changes and how the objectives support future

learning.

• Explain the goals and objectives.

R2. Motive

Matching

Provide learners’ choices by:

• Explaining choices they will have in the instructional process.

• Show choices of path, pace, place, and time.

R3. Familiarity Tie instruction to the learners’ experiences by:

• Showing a familiar process from ESC works to known UI in social

media, multimedia, and productivity platforms.



• Discuss leaders from the service center endorsing and showing

mastery of ESC Works processes.

Confidence

C1. Learning

Requirements

Build a positive expectation of success by:

• Show goals as a collective aspiration for the entire company.

• Clearly communicate and reiterate goals and objectives.

• Reinforce communication for aide between co-workers and

instructor.

C2. Success

Opportunities

Enhance the learner's beliefs in their competence by:

• Issue badges for objectives or milestones of masters.

• Personalize learning experiences for learners with repeated

deficiencies.

C3. Personal

Control

Identify to learners that their success is based on their efforts by:

• Learners will be able to construct their own processes for

department specific functions within the system.

• Provide time and location for learners to suggest improvement

to the system, instruction, or process.

Satisfaction

S1. Intrinsic

Reinforcement

Provide an opportunity for learners to use their newly acquired skills

by:

• Creating practice titles.

• Creating near term sessions.

S2. Extrinsic

Reinforcement

Provide reinforcement to the learners’ successes by:

• Provide digital and physical badges for mastery of specific skills.

• Provide certificates for objectives completed.

• Provide random positive recognition of work and diligence.

S3. Equity Assist learners in anchoring a positive feeling about their

accomplishments by:

• Maintain consistent instructional standards.

• Maintain consistent rewards and consequences.

• Maintain a consistent rubric for assessments/evaluations.



Instructor Guide

Introduction - (20 Minutes)

1. Provide an overview of the training agenda.

1.1. The instructor script will outline the different objectives that will be covered during the

training. It will also describe the purpose and connection it has with their jobs.

2. Introduce the ESC Works Platform.

2.1. The instructor script will introduce the ESC Works Platform and answer frequently

asked questions such as “What is ESC Works?”, “How do I access it?”, and “Why do we

need to use it?”

3. Answer any questions participants may have before beginning the lesson.

Body - (1 hour)

1. Administer a short survey to assess participants prior knowledge.

1.1. Using a show of hands, utilize the survey located in Appendix A to gauge the skill level

of the participants.

1.1.1. If the survey results are positive, continue to the next step.

1.1.2. If the survey results are negative, review any areas of concern prior to

proceeding to the next step.

2. Introduce participants to Creating Titles in ESC Works.

2.1. Provide a copy of the “Creating Titles in ESC Works” booklet in Appendix B to all

learners.



2.2. Utilizing the handout and your computer, navigate to escworks.net and login with your

credentials.

2.3. Briefly go over the first page of the document so that employees are aware of the

resources and points of contacts available to them.

2.4. Follow the steps, and begin to demonstrate the steps. These steps will cover using

existing titles, creating new titles, as well as best practices.

2.5. Provide learners with a copy of “Performance Tasks” in Appendix C and the

Performance Tasks Rubric in Appendix D.

2.6. Allow participants a 15 minutes to complete activity.

2.7. Upon completion of the activity, have the learners work with the person next to them

so they can evaluate each other using the Peer-Assessment Rubric.

3. Introduce participants to Creating Sessions in ESC Works.

3.1. Provide a copy of the “Creating Sessions in ESC Works” booklet in Appendix B to all

learners.

3.2. Follow the steps, and begin to demonstrate the steps.

3.3. Provide learners with a copy of “Performance Tasks” in Appendix C and the

Performance Tasks Rubric in Appendix D.

3.4. Allow participants a 15 minutes to complete activity.

3.5. Upon completion of the activity, have the learners work with the person next to them

so they can evaluate each other using the Peer-Assessment Rubric.



4. Assess learners.

4.1. Provide learners with a copy of the “Summative Assessment” in Appendix E and the

Self-Assessment Rubric in Appendix F.

4.2. Allow learners 30 minutes to complete the assessment.

4.3. Have participants notify the facilitator so they can review their work using the rubric.

Conclusion - (10 Minutes)

1. Provide a Summary and Short Recap.

1.1. The instructor script will outline a short recap of what was covered, along with a

summary of where they can receive additional assistance.

2. Request Feedback.

2.1. Notify participants that they will be receiving an email that will contain a short survey

and their certificate of completion.

Learner Content

Learning Materials

Learning Materials Description

Creating Titles

By Joshua Essary

This booklet provides detailed instructions on how the participant will

create titles using ESC Works. This document contains annotated

screenshots of the process.

Creating Sessions

By Joshua Essary

This booklet provides detailed instructions on how the participant will

create sessions using ESC Works. This document contains annotated

screenshots of the process.

https://drive.google.com/open?id=1ff_TPjwVphGWNPZ7g1yeTp3I2tRuhYt-

https://drive.google.com/open?id=1_nxhR_3tcM3UQwrUbkDdQksA07u8Iek5



Assessment Materials

Assessment Materials Description

Performance Tasks – Appendix C

The Performance Tasks assessment allows the participants to put into practice the material that it being taught. It also allows for learners to ask questions should they have any.

Performance Tasks Rubric – Appendix D

The Peer-Assessment Rubric will allow classmates to evaluate their peers and provide feedback on areas of improvement.

Summative Assessment – Appendix E

The Summative Assessment requires the learner to complete the task of creating a session from start to finish.

Summative Assessment Rubric – Appendix F

The Self-Assessment Rubric will allow the learner to evaluate them self to ensure they’ve successfully mastered the material.

Technologies Used

Technology Tool Purpose

Laptop (Company Issued)

The laptop is necessary so that the participants are able to access ESC Works along with the training material.

Internet Access (Company Issued)

Since ESC Works is a web-based application, participants will need access to the internet to be able to access the portal.

ESC Works Account (Company Issued)

Account credentials will be provided to participants, so they can access ESC Works and various applications applicable to their role.



Evaluation

Evaluation Plan

To ensure a successful instructional design project was created, we utilized Kirkpatrick's

four-level evaluation model. The critical insight we wanted to receive was to know if the set

learning objectives meet the goals of the learner if the learning materials and activities are

relevant to the learner and if the project is designed in a way that’s easy for the learner to

understand. Through the use of the evaluation model and a Subject Matter Expert (SME), we

were able to obtain this valuable information. The four-level evaluation model identity’s the

learner’s reaction, learning, behavior, and results as it relates to the instruction. By using an

SME, we are able to receive constructive feedback on necessary changes to ensure the training

is efficient and easy to follow for the user.

Evaluation

Type Questions to Consider Evaluation Methods

Level 1

Reaction

• Did the learners like and enjoy the training?

• Did the learner consider the training relevant?

• Did the learner feel engaged in the material?

• Post workshop

survey

Level 2

Learning

• Did the learners master the set learning

objectives?

• Are the learners able to completed the tasks

without issue?

• Performance

Tasks

• Summative

Assessment

Level 3

Behavior

• Did the learner make the connection between

the presentation and their job?

• Did the learner apply knowledge from past

experiences?

• Summative

Assessment

Level 4

Results

• Has there been an increase in productivity

following the training?

• Post workshop

survey



Subject Matter Expert

Kristen Arterbury served as the consultant and SME, subject matter expert, for this

report. Mrs. Arterbury was employed at the service center when ESC Works was first deployed.

She is one of the very few people employed at the service center who has extensive experience

with the system. Kristen also serves as the Assistant Director of the General Education

department. This department uses ESC Works more than any other department. An essential

role of this position is to develop and coordinate workflows and processes for ESC Works. She

typically delivers new instructions or changes in procedures regarding data entry and reporting.

Her skill set and knowledge were invaluable during all phases of the of the report.

Evaluation Rubric

The following rubric was created to allow the SME to evaluate the process of the

instructional design. The rubric was adapted from K. Indermill, By Design from Figure 11.1 of

Larson and Lockee.

Analysis

Description 1 Unacceptable

2 Needs

Improvement

3 Acceptable

4 Exceptional

The learner profile is appropriate

and reflective of the specified

workers

X

The learner and performance

context is accurately assessed and

analyzed.

X




2 Needs

Improvement

3 Acceptable

4 Exceptional

The learner’s survey is complete

and covers all needed areas. X

The complete analysis

appropriately assesses all areas of

the learning/performance context

concerning ESC Works.

X

Design


2 Needs

Improvement

3 Acceptable

4 Exceptional

Desired knowledge skills and

attitudes are appropriately

identified.

X

Outcomes are identified and

sequenced appropriately

according to analysis.

X

Assessments are appropriate to

the learning outcomes and

support weak areas of ESC Works.

X

Content identified is appropriate

and up to date with the current

ESC Works interface.

X

Development


2 Needs

Improvement

3 Acceptable

4 Exceptional

Instructional materials are

appropriate and up to date.

X

Revisions after initial

development are appropriate. X




2 Needs

Improvement

3 Acceptable

4 Exceptional

The instructional guide is

appropriately paced and thought

out.

X

SME Report and Review

The following report was developed and written by the SME on 4/22/18.

Analysis

“The instructional designers completed a very thorough assessment of the learners. This

process appears very useful and helpful in extension instructions for other aspects of ESC

Works. It would be interesting to see how other how the learner profile would be different in

other departments such as Federal Programs and Accountability. In regard to the learning and

performance context, this section was very well thought out. Would a completely virtual

learning context be appropriate or not? I noticed in the analysis you neglected to take into

account the BRIC (Baylor Research and Innovation Collaborative) where there are four

additional learning spaces with flexible furniture and adaptive breakout spaces. The survey is

complete and returned some very useful information. Also, many of the survey results returned

ambiguous data which lead to more questions. Do you think it would be appropriate to send

another survey with clarifying questions? Overall, the entirety of the analysis is well thought out

and thorough.”



Design

“The identified knowledge and skills are very useful and well identified. I would suggest

spending some more time getting to know the beginning steps such as logging in, what all of

the buttons do, and how to get to different areas. The skills identified for creating titles and

sessions are very specific and do not leave any room for ambiguity. They are also clear enough

for the learner to follow in small chunks as not to get lost in the material. Allowing the learners

to self-assess and peer assess gives the ownership of the system and this is extremely

important for brand new employees who feel a disconnect for the first few days at the service

center. How do you plan on dispersing this information when the time comes? Is any of this

information transferable if the Service Center decides to purchase and implement a new data

and event creation system?”

Development

“The handouts with step by step instructions are extremely helpful, detailed, and

thorough. I do like the use of the technology tools mentioned. Do you think there are too many

or just enough? Which ones would be your focus if you could only choose 1 or 2?

Overall, the entire design was impressively thorough and detailed. This report was eye-

opening regarding the process and detail. I would suggest allowing for more time to grow

accustomed to the login process, send a second survey, and identify focal technology tools in

case the instructor discovers the current set to be overwhelming.”



Comments and Suggestions

After reading the written report and a face-to-face debrief, adjustments will be required

during further development of the plan. The following areas will be addressed:

1. Integrate objectives and goals for the login process and orientation with the system

layout.

2. Develop and deliver a follow up survey based on the results from the initial delivery.

3. Expand the survey targets to additional employees beyond the original General

Education department.

The first suggestion is an important consideration and would require another analysis of

the objectives. These skills are addressed in the objectives but possibly more depth would be

required in the instruction and delivery of material. If this consideration is given additional goals

and instructional time, it would be necessary to create additional learning material and

assessments when developing this plan again.

The second and third suggestions both target the learner survey. The first and easiest

adjustment would be to the audience of the initial delivery. It could be helpful to obtain

additional information from other departments when creating a learner analysis. Even though

the material is being delivered to one department, additional perspectives could prove to be

valuable. Also, the instructional design is intended to be deployed on all departments. This

would be a natural step in the progression of the plan.



Finally, a second survey could prove very helpful and determine completely different

directions with the goals and objectives. For example, when asking learners to rate their level of

competency on specific tasks, many of the results were inconclusive and did not reveal any

clear data. Some of the results were evenly distributed across four answer choices. An

additional survey would allow the designer to revisit specific questions with rewording or

targeting different information.



Appendices

Appendix A: Student Survey

To gauge the skill level of your participants, ask the following questions using a show of hands.

1. How long have you been using ESC Works?

a. Less than 1 year

b. 1 – 2 Years

c. 3 – 4 Years

d. 5+ Years

2. How familiar are you with creating titles?

a. Beginner

b. Intermediate

c. Advanced

3. How familiar are you with creating sessions?

a. Beginner

b. Intermediate

c. Advanced



Appendix B: Handouts

The following documents will be utilized during this workshop. Please reference the Instructor

Guide on when to use these resources.

1. Creating Titles in ESC Works By Joshua Essary

2. Creating Sessions in ESC Works By Joshua Essary

https://drive.google.com/open?id=1ff_TPjwVphGWNPZ7g1yeTp3I2tRuhYt-

https://drive.google.com/open?id=1_nxhR_3tcM3UQwrUbkDdQksA07u8Iek5



Appendix C: Performance Tasks Group Activity

To test your knowledge of the material just presented, please complete the following activities.

Part A: Creating Titles

1. Select a partner to work with.

2. Using one computer, navigate through the steps presented in “Creating Titles in ESC Works”

to create a sample training session.

3. Upon completion of your work, please utilize the Performance Tasks Rubric (Appendix D) to

review your ESC Works Title.

4. Reflect on task completion with partner using these questions:

4.1. What areas do you need to address for improvement?

4.2. What helped your learning?

4.3. How does what you learned connect with what you already know?

Part B: Creating Sessions

1. Working your partner from Part A, use one computer to navigate through the steps

presented in “Creating Sessions in ESC Works” to create a sample training session.

2. Upon completion of your work, please utilize the Performance Tasks Rubric (Appendix D) to

review your ESC Works Title.

3. Reflect on task completion with partner using these questions:

3.1. What areas do you need to address for improvement?

3.2. What helped your learning?

3.3. How does what you learned connect with what you already know?



Appendix D: Performance Tasks Rubric

Category 4 3 2 1

Accuracy of Data

All data was entered accurately with no editing required.

Most data was accurate the first time and little editing was required

About half of the data entered was accurate. Some data was unclear, and edits needed to me made.

Majority of data was incorrect and needed to be edited. Multiple fields had to be revisited.

Department Specific Data

Employee communicates effectively with a department to ensure accurate data entry. He/she uses department specific best practices.

Employee understands department specific needs and usually addresses those needs.

Employee sometimes neglects department specific needs and rarely addresses those needs.

Employee ignores or is not aware of department specific needs. These needs are never addressed.

Use of company best practices

Employee effectively utilizes all best practices for the company. User also seeks out improvement to refine best practices.

Employee effectively utilizes most of the company's best practices.

Employee rarely utilizes best practices for data entry.

Employee does not use best practices.

Problem-solving

Actively looks for and suggests solutions to problems.

Refines solutions suggested by others.

Does not suggest or refine solutions but is willing to try out solutions suggested by others.

Does not try to solve problems or help others solve problems. Let’s others do the work.

Focus on the task

Consistently stays focused on the task and what needs to be done. Very self-directed.

Focuses on the task and what needs to be done most of the time. Other team members can

Focuses on the task and what needs to be done some of the time. Other team members must

Rarely focuses on the task and what needs to be done. Let’s others do the work.



Category 4 3 2 1

count on this person.

sometimes remind to keep employee person on-task.

Time-management

Routinely uses time well throughout the project to ensure things get done on time. team does not have to adjust deadlines or work responsibilities because of employee's procrastination.

Usually uses time well throughout the project but may have procrastinated on one thing. team does not have to adjust deadlines or work responsibilities because of the employee's procrastination.

Tends to procrastinate, but always gets things done by the deadlines. team does not have to adjust deadlines or work responsibilities because of the employee’s procrastination.

Rarely completes tasks by the deadlines AND team has to adjust deadlines or work responsibilities because of the employee's inadequate time management.



Appendix E: Summative Assessment

To show mastery of this workshop, please follow the steps you’ve learned today to create the

following session in ESC Works.

Field Content

Title High School English 11 Curriculum Development

Description This session is for teachers participating in the High School English 11 Curriculum Development Program.

Event Type Meeting

Audience Teachers – External

Subject Curriculum Development

Audience Limit 15

Fee FREE

Credits Available CPE

Facilitator/Directory Select your name

Contact Person Select your name

Instructors Select your name

Session Approver Select your supervisor

Evaluations Instructor Evaluation

NOTE: Do Not check the option for “Ready for Approval” as this session will not be posted.

When finished, please review and score your work according to the Summative Assessment

Rubric (Appendix F).



Appendix F: Summative Assessment Rubric

CATEGORY 4 3 2 1 Accuracy of Data

All data was entered accurately with no editing required.

Most data was accurate the first time and little editing was required

About half of the data entered was accurate. Some data was unclear, and edits needed to me made.

Majority of data was incorrect and needed to be edited. Multiple fields had to be revisited.

Department Specific Data

Employee communicates effectively with a department to ensure accurate data entry. He/she uses department specific best practices.

Employee understands department specific needs and usually addresses those needs.

Employee sometimes neglects department specific needs and rarely addresses those needs.

Employee ignores or is not aware of department specific needs. These needs are never addressed.

Use of company best practices

Employee effectively utilizes all best practices for the company. User also seeks out improvement to refine best practices.

Employee effectively utilizes most of the company's best practices.

Employee rarely utilizes best practices for data entry.

Employee does not use best practices.

Problem-solving

Actively looks for and suggests solutions to problems.

Refines solutions suggested by others.

Does not suggest or refine solutions but is willing to try out solutions suggested by others.

Does not try to solve problems or help others solve problems. Let’s others do the work.

Focus on the task

Consistently stays focused on the task and what needs to be done. Very self-directed.

Focuses on the task and what needs to be done most of the time. Other team members can

Focuses on the task and what needs to be done some of the time. Other team members must

Rarely focuses on the task and what needs to be done. Let’s others do the work.



CATEGORY 4 3 2 1

count on this person.

sometimes remind to keep employee person on-task.

Time-management

Routinely uses time well throughout the project to ensure things get done on time. team does not have to adjust deadlines or work responsibilities because of employee's procrastination.

Usually uses time well throughout the project but may have procrastinated on one thing. team does not have to adjust deadlines or work responsibilities because of the employee's procrastination.

Tends to procrastinate, but always gets things done by the deadlines. team does not have to adjust deadlines or work responsibilities because of the employee’s procrastination.

Rarely completes tasks by the deadlines AND team has to adjust deadlines or work responsibilities because of the employee's inadequate time management.

Application of Skills Learned

Learner applies of learned skills appropriately

Learner applies most skills appropriately

Learner applies a few skills appropriately

Learner applies very little, if any, skills to the context.

how to effectively use esc works - edtechcdn.ericsilva.me · joshua essary & eric a. silva...

Documents