Final Presentation

Group 2

J. Bennett, K. Hooker,

J. Manor, L. Wright

EDP 617

Executive Summary

• Program Description

• Evaluation Data and Analysis

– Goal: Evaluate Utility, Clarity, and Implementation of the Online Math I Unit Plans

– Responsive Evaluation Framework

• Findings and Recommendations

1. Additional Teacher Resources

2. More Student Learning Activities

3. Increase Investment in Technology

4. Increase Awareness of Online Plans

Program Description

Goals and Objectives

• Academic Innovation Units

• Common Core Program Objectives

• Common Core Objectives for Students

• Common Core Objectives for Teachers

Academic Innovation Goals

Common Core Program

Link to Common Core Standards Common Core Math Standards

Common Core Objectives for Students

Common Core Objectives for Teachers

• No need to reinvent the wheel

• Complete units created around the Common Core

• Assessments and data available online

Who was the program designed to help?

The standards are:

• Research- and evidence-based

• Clear, understandable, and consistent

• Aligned with college and career expectations

• Based on rigorous content and application of

knowledge through higher-order thinking skills

• Built upon the strengths and lessons of current

state standards

• Informed by other top performing countries in

order to prepare all students for success in our

global economy and society

• High School

Math Teachers

• Math I

Students

Who was the program designed to help?

Math I Discussion Boards

Joint Venture

Math

I Unit

Plans

Math Online Unit Plans

http://k12.wvu.edu

Publicity

New Approach to Online Learning

Discussed at the Bridgeport Conference

Center

(February 19, 2014)

http://www.wbohttp://wboy.images.worldno

w.com/images/24767732_BG1.jpgy.com/sto

ry/24767732/new-approach-to-online-

learning-discusses-at-the-bridgeport-

conference-center

Program Activities

Students

• Six Units

• Multimedia delivery

• Interactive games,

simulations, practice

problems

• Online Assessments

Teachers

• Access to all student

resources and

assessment data

• Extra resources

• Teacher Community

Discussion Board

Measuring Student Success

• Goal achievement

through testing

• A unit must be

completed successfully

before moving on

• Pre and Post tests

Example Student Activity

Measuring Teacher Success

• Community

Discussion Board

• Electronic Surveys

• Paper Surveys

Evaluation FrameworkResponsive Evaluation

Responsive Evaluation Framework

• Advantages of Responsive Evaluation – Work closely with stakeholders

– Allow for evaluators to determine the merit and worth of the Math I Unit Plans

– Goal of the evaluation is to improve the program

– Provide constructive recommendations for improvement

– Utilize Quantitative and Qualitative Data to assess intended and unintended outcomes

– None definitive judgments, rather recommendations and accolades

Stakeholder Key Issues

1) To understand the utility of the Next

Generation Math 1 Unit Plans for

teachers

2) To assess the unit plan’s ease of use

3) Address any additional resources needed

for implementation.

Data Collection & Analysis

• Sources of Data

– Survey 1

– Survey 2

– Blue Ribbon Follow Up Meeting

– Online Community Discussion Boards

Survey 1

• Generated by Evaluation Team specifically for Next Generation Math I Unit Plans.

• Administered online through Survey Monkey

• Sent to nineteen pilot instructors

• Address teacher demographics of years of experience, preferred method of content delivery, number of unit plans utilized

• 5 Likert-Type scale items to address utility, ease of use, and resources

• 2 open ended questions to provide information concerning utilization and improvements

Survey 1

• Analysis

– Teacher Demographics-Frequency Tables and

percentages

– Likert-Type Questions-Frequency Tables,

Mean, and percentages

– Open ended Questions-examined for

common words, phrases, or topics.

Frequency tables will help organize the data.

Survey 2

• Created by Stakeholders to evaluate both the Next Generation and the Teach 21 websites

• Administered at the Blue Ribbon Math I Follow Up meeting in Morgantown, WV on March 29, 2014

• Evaluators only analyzed data from relevant questions

• Eleven Likert-Scale questions

• Five free response questions

Survey 2

• Analysis

– The Likert-Scale Items were analyzed using

frequency tables and percentages

– Free response questions were examined for

common words, phrases, or topics.

Frequency tables will help organize the data

Related Survey Questions to

Stakeholder Issues

16j) I will use The Next Generation Mathematics for WV Schools website assessment system to gather student data to inform my instruction.

19) How would you use The Next Generation Mathematics for WV Schools website to support your classroom? (Check all that apply)

Unit ease of use 4) I am comfortable with using a computer and online tools:

1b) How would you rate the user interactivity on the Teach 21 Website?

6) The unit plans for Math I was easy to follow?

3) Please list any specific concerns you have regarding the effectiveness of the content and

technology used on the current Teach 21 website.

4) What would you recommend to improve the

content and user interactivity of the Teach 21 website?

17) What features of The Next Generation Mathematics for WV Schools website do you like?

18) What features of The Next Generation Mathematics for WV Schools website do you like? What can be improved?

20) What would be the challenges with using The Next Generation Mathematics for WV Schools

website?

21) Please provide any specific feedback for The

Next Generation Mathematics for WV Schools courses.

22) Please rate the following items regarding The

Next Generation Mathematics for WV Schools website: Appearance, Usability, Structure, Content,

Instructional Resources, Teacher Support.

Address Additional Resources needed to

Implement Unit Plans

4) I am comfortable with using a computer and online tools?

5) What other classroom resources are you currently using to implement the integrated mathematics

courses?

5) If I have any questions or need

assistance with a Math I Unit Plan, it is easy for me to find help?

8) I am comfortable with using a computer and online

tools?

8) I support the Common Core Math I Implementation in West Virginia.

14) Please describe what other resources in content and technology for teachers, student and parents that the Next Generation Mathematics for WV Schools

Website can provide?

15) Does the Next Generation Mathematics for WV

Schools website provide adequate resources in content and technology for teachers, students and parents?

16d) I have the resources at my school to implement The Next Generation Mathematics for WV Schools

Mathematics courses.

Key Stakeholder

Needs Survey 1 Survey 2

Utility of Online Units 3) How many different units in Math I

have you used? 1a) How would you rate the current content on the

Teach 21 website?

7) I feel prepared to use the Math I

Unit Plans in my classroom? 1c) How would you rate the way that the Teach 21

website assists you in the meeting the Next Generation Content Standards and Objectives?

2) How does the current Teach 21 website content and technology assist you with meeting the goals of the Common Core State Standards? Please Explain

6) How do you currently use website content and technology in the classroom?

7)What type of content delivery do you prefer? Group Instruction Individual Pace Other:___________

16a) The Next Generation Mathematics for WV

Schools website will better assist me in meeting the Common Core State Standards in my classroom

16b) The Next Generation Mathematics for WV Schools website will engage my students in learning mathematics

16c) The Next Generation Mathematics for WV Schools website will enhance my classroom

instruction.

16e) The Next Generation Mathematics for WV Schools website can replace static textbooks.

16f) The Next Generation Mathematics for WV Schools website assessment systems is a powerful

instructional tool.

16g) The Next Generation Mathematics for WV

Schools website Teacher community is a powerful educational tool.

16h) I use The Next Generation Mathematics for WV Schools website to support my instruction.

16i) I will use The Next Generation Mathematics for WV Schools website teacher community to collaborate with other teachers.

Additional Data Sources

• Blue Ribbon Follow Up Meeting on March

29, 2014 in Morgantown, WV.

• Online Community Discussion Boards

Our Findings

Findings:

• Our data analysis was based on the 3 key

issues noted in our evaluation framework:

1. To understand the utility of the online units

2. Assess the units ease of use

3. Address any additional resources needed to

implement the online units.

Findings for Key Issues:#1 To understand the utility of the online units

Evaluator-created Survey:• Individuals surveyed had interacted with at least one unit in the Math

I Lesson section

• There were mixed results when asked if the unit lessons were easy

to follow

2

1

2 2

1 1

0

0.5

1

1.5

2

2.5

1 2 3 4 5 6

Responses

n=9

0

1

3

5

00

1

2

3

4

5

6

Strongly Disagree

Disagree Not Sure Agree Strongly Agree

Responses

n=9

Stakeholder-created Survey:• Individuals surveyed were not sure the website would properly

assist them with meeting the new standards with the Math I

Common Core.

• There were mixed results when asked if the website would engage

student learning or enhance classroom instruction.

n=6

N=6

0 0

3

2

1

00.5

11.5

22.5

33.5

Responses

0 0

3 3

001234

Strongly Disagree

DisagreeNot Sure Agree Strongly Agree

0 04 3

00

5

Strongly Disagree

Disagree Not Sure Agree Strongly Agree

Student Engagement Improve Classroom Instruction

Findings for Key Issues:#1 To understand the utility of the online units

Findings for Key Issues:#2: Assess the units ease of use

• Statement: I am comfortable with

computers and online tools.

0 0

1

2

3

0

0.5

1

1.5

2

2.5

3

3.5

Strongly Disagree

Disagree Not Sure Agree Strongly Agree

Responses

0 0

1

2

6

0

1

2

3

4

5

6

7

Strongly Disagree

Disagree Neutral Agree Strongly Agree

Responses

Stakeholder-created Survey Evaluator-created Survey

n=9n=6

• Rate the usability of the website

0 0

1

2

6

0

1

2

3

4

5

6

7

Strongly Disagree

Disagree Neutral Agree Strongly Agree

Re

sp

on

se

s

Stakeholder-created Survey Evaluator-created

Survey“The unit plans for Math I was easy to follow.”

1

2

1

2

0

0.5

1

1.5

2

2.5

Poor Fair N/A Good Excellent

Resp

on

es

n=9n=6

Findings for Key Issues:#2: Assess the units ease of use

Findings for Key Issues:#3: Address any additional resources

needed to implement the online units.

• High demand for additional resources.

“Most of my experience with the site has been flustering. Broken

links, activities I would not use, too much „fluff‟ and not enough „meat‟. “

“Too many sites and not enough time to explore/adequately plan”

Discussion Board

Stakeholder-Created Survey:

Evaluator-Created Survey:

“The content does not connect from one unit to another.”

“They can be long and the standards are sometimes not

taught when I feel they need to be.”

Findings for Key Issues:#3: Address any additional resources

needed to implement the online units.

“I would simply like a good sources for activities related to

each objective. “

Discussion Board

Stakeholder-Created Survey:

Evaluator-Created Survey:

“Even if you were to follow the

Math I unit plans precisely, I still

feel you would have to supplement

other resources to ensure student

mastery of the skills.”

• High demand for additional activities.

Recommendations

Recommendations:

• Recommendation #1: Additional Resources

Individuals voiced a strong desire to have a listing of additional resources for each lesson to serve as optional supplemental content. There were concerns over the current content contain a large amount of “filler”.

• Recommendation #2: Additional Activities for all Units

Individuals voiced a strong desire the have more activities included in each lesson to assist in student comprehension of the unit objectives.

Recommendations:

• #3: Investment in Technology:Math I is a learning system delivered online. With such a

technology-centric learning method, adequate computers and

internet connection must be in place in order for this program to

succeed.

I have the resources at my school to

implement The Next Generation

Mathematics for WV Schools mathematics

courses.

1 1

2

1 1

0

0.5

1

1.5

2

2.5

Responses

“A lot of browsers in the schools are very outdated. It‟s highly

possible that many of the school‟s computers will not be able

to access the applets, etc. because of the outdated applets.

To use this in the classroom would require 1 to 1 computing

which doesn‟t happen at most schools. Many labs/computer

are “tied up” at this time of the year for assessment purposes.

The website seems prone to freezing/buffering.”

“Some technology components cannot be used because a

newer version of a program must be installed. Some content

takes way too much time on simple concepts and other

concepts need deeper development.”

Stakeholder-Created Survey:

References

• ASCD. (2014). Association for Supervision and Curriculum Development. Retrieved from ASCD and Common Core State Standards: http://www.ascd.org/common-core-state-standards/common-core-state-standards-adoption-map.aspx

• Stake, R., (2004). Standards-Based & Responsive Evaluation. Thousand Oaks, CA: Sage Publications Inc.

• Stufflebeam, D., & Shinkfield A. (2007). Evaluation Theory, Models, & Applications. San Francisco, CA: Josey-Bass.

• West Virginia University. (2012). WVU Academic Innovation K12. Retrieved from Introduction to Common Core:http://k12.wvu.edu/Courses/introduction.php?c=1