preparing to succeed with the electronic ncss/ncate spa review process accreditation,...
TRANSCRIPT
Preparing to Succeed Preparing to Succeed with the Electronic with the Electronic NCSS/NCATE SPA NCSS/NCATE SPA Review ProcessReview Process
Accreditation, Accountability, and QualityAccreditation, Accountability, and Quality
An Institutional Orientation and Professional Development An Institutional Orientation and Professional Development ConferenceConference
September 17, 2006September 17, 2006
D. Mark MeyersD. Mark Meyers
Rowan UniversityRowan University
The Ten ThemesThe Ten Themes
Why themes?What difference does it
make for your institutions?Change in the Review
Process and its implication
Why themes?What difference does it
make for your institutions?Change in the Review
Process and its implication
New Program Review Process
The report Web-based form Limited pages by section Will be able to access drafts
Do we need to do more than one report? Common Assessments in Different Programs Baccalaureate, Master’s, Post-Baccalaureate, or Alternate Route? Program offered at Multiple Sites
Questions the Program Review Will Answer
Does the program have in place assessments that demonstrate mastery of the NCSS themes. Have candidates mastered the necessary knowledge for
the subjects they will teach or the jobs they will perform? Do candidates meet state licensure requirements? Do candidates understand teaching and learning and can
they plan their teaching? Can candidates apply their knowledge in classrooms and
schools? Are candidates effective in promoting student learning?
How much data do we need?
State licensure test data Must have 80% pass rate For 3 years with latest derived from most recent annual report, (or
own records to provide a more current set of data)
Assessment data For Fall 05 -- Spring 07
the program has at least 1 semester’s data on at least 5 assessments
After Spring 07 Three years of data on each set of assessments
Note: It is better to have limited data on good assessments than
tons of data on ineffective ones. However, there must be more than the ‘intention’ to assess
Sections of the Report
I. Contextual InformationII. Assessments and Related DataIII. Standards Assessment ChartIV. Evidence for Meeting StandardsV. Use of Assessment Results to Improve
Candidate and Program Performance
I. Contextual Information Describe any state or institutional policies that may
influence the application of NCSS standards Explanations and clarifications, not a page for excuses
Description of required field and clinical experiences Describe all experiences in program, duration, criteria for selection of
sites and teachers, and training of teachers and supervisors This is crucial for NCSS 3.2
NCSS expects supervisors to be SS specialists.
Description of criteria for admission, retention, and exit from the program. For the program, not the entire department, school, college unless
they are the same
Contextual Information Continued Description of the relationship of the program to the unit’s
conceptual framework A narrative describing how the CF is reflected in the program It should be clear that your program is a part of the unit, not a separate entity
Indication of unique program assessments and its relationship to the unit assessment system This should clarify how the key assessments used in the program are derived from the
unit assessment system
Attach the following A program of study that outlines required courses
Titles and brief catalog descriptions, be sure there is coverage of all 10 themes and applicable content standards
Remember to avoid the trap of ‘uncommon course titles” Chart with number of candidates and completers over past three years (Title II
Data) Chart on program faculty expertise and experience
Summary of Content Expectations of NCSS by Standard
1.1 Culture and Cultural Diversity. This standard expects candidates to understand the basic scholarly concepts of anthropology.
1.2 Time, Continuity, and Change. This standard expects candidates to be familiar with the history of the United States, western civilization, and non-western society.
1.3 People, Places, and Environments. This standard expects candidates to understand the basic scholarly concepts of geography.
1.4 Individual Development and Identity. This standard expects candidates to understand the basic scholarly concepts of psychology.
1.5 Individuals, Groups, and Institutions. This standard expects candidates to understand the basic scholarly concepts of sociology.
Standards Expectations Continued 1.6 Power, Authority, and Governance. This standard expects candidates to
understand the basic scholarly concepts of political science. 1.7 Production, Distribution, and Consumption. This standard expects candidates
to understand the basic concepts of micro- and macro-economics. 1.8 Science, Technology, and Society. This standard expects candidates to
understand the manner by which science and technology have enhanced or threatened the development of human society. In general, educational technology application courses alone do not meet the expectations of this standard.
1.9 Global Connections. This standard expects candidates to understand that our planet has been exposed to an ever-increasing human interdependence in a world made smaller by improvements in communication, transportation, and trade.
1.10 Civic Ideals and Practices. This standard expects candidates to understand the basic concepts of citizenship in a democratic society.
Specialized Standards
2.1 through 2.5. These standards expect the candidates receiving licensure in a specific social science discipline to understand the basic scholarly concepts of that discipline.
3.1 Course(s) on Teaching Social Studies. This standard expects that the social studies methods courses(s) will deal specifically with the nature of the social studies and with ideas, strategies, and techniques for teaching social studies at the appropriate licensure level. The NCSS frowns upon a generic methods course where social studies instruction is only a minor aspect of the emphases.
3.2 Qualified Social Studies Faculty. This standard expects that faculty in the social studies education components of the program are recognized as exemplary teachers and as scholars in history or the social sciences and/or social studies education. The credentials of faculty members are to be described in Attachment B.
II. Assessments and Related Data
In Chart form For each assessment, indicate name, type,
and administration point on chart
The Required Assessments1. State or Professional Exam Data
• 80% must pass as a prerequisite for program recognition. Subscores as well as totals
2. Assessment of Content Knowledge in the Social Studies• Comp exams, research reports, portfolios, course grades, test grades/unit assessments
(Attachment) Be careful about course grades!
3. Assessment that demonstrates candidates can effectively plan classroom-based instruction
• Practicum evaluations, lesson/unit planning
4. Assessment that demonstrates candidates knowledge, skills, and dispositions are applied effectively in clinical practice
• Must be connected to the NCSS standards, not merely ‘generic’
5. Assessment that demonstrates candidate effects on P-12 student learning
• Teacher work samples, portfolio tasks, case studies, Poster Sessions , etc
6. Assessment that demonstrates candidates are proficient in content knowledge, professional and pedagogical knowledge, skills, and dispositions and/or student learning
• Fill in any standards that might not have been covered with 1 - 5.
Be careful of Labels
On my campus, there have been some confusion with everyone using the term “Assessments”
What we did, was to move towards the use of agreed upon terms.
Assessments -- 6-8 SPA Assessment for Program Reviews Benchmarks -- Items at Transition Points in the Unit
Assessment System Signature Assignments -- Items inside of courses that are used
no matter who teaches the course These are given only as an example not to imply this as the only means of
approaching these tasks.
III. Standards Assessment Chart
For each NCSS Standard, identify the assessments that address each standard One assessment may apply to multiple standards and some standards will be addressed by
more than one assessment All programs must do 1.1 -- 1.10 and 3.1 2.1 -- 2.5 are only done by applicable major field programs In the next section you will describe the assessments in greater detail
IV. Evidence for Meeting Standards
For each of the 6-8 assessments, provide: 1. 2-page narrative including:
a) A brief description of the assessment and its use in the program b) The alignment with NCSS standards it addresses;
• Why does this assessment fulfill the theme(s) identifiedc) A brief summary of the data findings d) An interpretation of how findings provide evidence of meeting standards.
2. Make the information easy to follow and user-friendly3. Assessment Instrument, Scoring Guide, and Data Tables
IV continued
For state licensing tests or professional exams submit the following
The names of all licensure tests or professional examinations required by the state for content and pedagogical or professional knowledge.
For example, Praxis II Social Studies: Content Knowledge. Description of the correlation between licensure test data and
applicable NCSS standards. Praxis subscores are provided for History (1.2), Geography
(1.3), Civics/Gov’t (1.6 & 1.10), Econ. (1.7) & Behavioral Sciences (1.1,1.4, & 1.5). Please note: quartile scores are not appropriate evidence
Aggregated pass rates for each year over the past 3 years, including the most recent academic year. Data must be presented on all candidates
V. Use of Assessment Results
Evidence must be presented in this section that assessment results have been analyzed and have been or will be used to improve candidate performance and strengthen the program
Describe findings based on aggregated data, and discuss how findings have been or will be applied to program improvement.
Organize discussion around Content knowledge; Professional knowledge, skills, and dispositions,
and Impact on student learning.
Review calendar
Fall cycle – program reports due September 15 Spring cycle – program reports due February 1 It is recommended you submit 1 year in advance.
Example: XYZ College plans to submit program reports
one semester before its Spring 2007 visit; thus, reports are due by 2/1/06.
Program review decisions
Reviewers will make one of the following decisions based on your program report:
The program is nationally recognized.
The program is nationally recognized with conditions. (18 months)
The program is nationally recognized with areas in need of improvement
The program is not nationally recognized; a new program report may be submitted.
How will my program be reviewed?
Each program report will be reviewed by a 3-person team including a lead reviewer
The lead reviewer will compile the final team report and send to NCATE
NCSS Audit Committee will review reports for consistency and make final decision if review team was undecided
Guidelines for Program reviewers are available for you to review.
Some Questions
What if we do not have data for all submitted assessments?
What if our current program assessments are not a close match to the 6-8 SPA assessments?
Remember, for each assessment present title of assessment, group assessed, and number in group. Specify the cohort Provide data on each scored category Provide legends for abbreviations
NCATE Paper on Impact on Student Learning
Available on the website Suggests the candidate:
Undertakes a diagnosis (a pre-test) of P-12 student learning in some area he or she will teach;
Plans an appropriate sequence of instruction to advance P-12 student learning, and teaches in ways that engage P-12 students who bring differing background knowledge and learning needs;
Conducts some concluding assessment (or post-test); documents the student learning that occurred, or did not;
Reflects on changes in teaching that might have improved the results.
Sample Chart Courses/SegmentsApplicableStandard
Course,SegmentABCDF
NTotMnMdMoR
AcceptableOther
Sample Chart for Clinical practice
1.1
1.2
1.3
1.4
1.5
1.6
Fall 2005 Mean (n=10)
2 5 4 0
Spring 2006 Mean (n=12)
1 4 3 0
4,22
4.32
Behavioral Sciences -- 1.1, 1.4, 1.5 US History -- 1.2 World History -- 1.2 Geography -- 1.3 Civics/Government -- 1.6 , 1.10 Economics -- 1.7 NCSS considers these sub-tests to be aligned with the
enumerated standards.
Praxis Subscores for NCSS Themes