copyright © 2008 by abet, inc. 1 conducting a visit using the new criteria conducting a visit using...
TRANSCRIPT
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
1
Conducting a Visit Using Conducting a Visit Using the New Criteriathe New Criteria
1
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
2
New Criteria Visits Outline
1. Introduction2. Criteria3. Self Study4. Visit Preparation and Conduct5. Exit Meeting6. Visit Forms7. Draft Statement8. Final Statement9. Wrap Up
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
3
General Criteria
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
4
ABET Definitions (1)
• Program Educational Objectives – Program educational objectives are broad statements that describe the career and professional accomplishments that the program is preparing graduates to achieve.
• Program Outcomes – Program outcomes are narrower statements that describe what students are expected to know and be able to do by the time of graduation. These relate to the skills, knowledge, and behaviors that students acquire in their matriculation through the program.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
5
ABET Definitions (2)
• Assessment – Assessment is one or more processes that identify, collect, and prepare data to evaluate the achievement of program outcomes and program educational objectives.
• Evaluation – Evaluation is one or more processes for interpreting the data and evidence accumulated through assessment practices. Evaluation determines the extent to which program outcomes or program educational objectives are being achieved, and results in decisions and actions to improve the program.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
6
Notes on definitions
• These are very important: you (and your PEVs) should know them well.
• There is no requirement that the institution use these terms as defined by ABET in the self study. It is important to determine what terms the institution uses and the correspondence with the ABET definitions.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
7
Criterion 1 – Students
Students can complete the program in a reasonable amount of time. They have ample opportunity to interact with their instructors. Students are offered timely advising, by qualified individuals, about the program’s requirements and their career alternatives. Students who graduate from the program meet all program requirements.
Comment: Similar to previous criteria.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
8
Criterion 2 – Program Educational Objectives
The program has documented, measurable educational objectives, based on the needs of the program’s constituencies.
Comment: Appropriate constituencies (e.g., faculty, students, alumni, employers) should be used in determining the constituency needs.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
9
Criterion 3 – Program Outcomes
The program has documented, measurable outcomes, based on the needs of the program’s constituencies.
Comment: If there is broad constituency input to objectives and outcomes are derived from objectives, then broad constituency input to outcomes development may not be needed. (The faculty is the primary constituency involved in development of outcomes.)
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
10
Criterion 3 – (continued) The program enables students to achieve, by the
time of graduation:
a) An ability to apply knowledge of computing and mathematics appropriate to the discipline
b) An ability to analyze a problem, and identify and define the computing requirements appropriate to its solution
c) An ability to design, implement and evaluate a computer-based system, process, component, or program to meet desired needs
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
11
Criterion 3 – (cont.)d) An ability to function effectively on teams to accomplish
a common goale) An understanding of professional, ethical, legal,
security and social issues and responsibilitiesf) An ability to communicate effectively with a range of
audiencesg) An ability to analyze the local and global impact of
computing on individuals, organizations and societyh) Recognition of the need for, and an ability to engage in,
continuing professional developmenti) An ability to use current techniques, skills, and tools
necessary for computing practice.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
12
Notes on Criterion 3 – (1)
• Institution is expected to define its own outcomes.
• Additional elements will appear in program criteria.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
13
Notes on Criterion 3 – (2)
• Institution will need to show how a-i are enabled.– How can this be done?– What about partial compliance?– Is it sufficient for the institution to have an effective
assessment process that covers a-i?
• Notice that it says enables students to achieve; doesn’t demand that all students have achieved them.– How can this be evaluated?
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
14
Notes on Criterion 3 – (3)
• Criterion 3 (c) should be interpreted as appropriate for the nature and objectives of the program.
– 3(c): An ability to design, implement and evaluate a computer-based system, process, component, or program to meet desired needs
– Normally not all of “system, process, component, or program” are required.
• I.e., the “or” is not inclusive• One could be enough
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
15
Criterion 4 – Continuous Improvement
The program uses a documented process incorporating relevant data to regularly assess its educational objectives and outcomes, and to evaluate the extent to which they are being met. The results of the evaluation are documented and used to effect continuous improvement of the program through a documented plan.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
16
Notes on Criterion 4 (1)
• Must assess the extent to which outcomes are being achieved by students.
• Effective assessment usually implies:– Measures based on actual student performance for
outcomes– A priori establishment of performance goals
• No requirement that program improvements must have been made.– What is satisfactory evidence?
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
17
Notes on Criterion 4 (2)
• Must assess the extent to which objectives are being achieved by graduates.
• Effective assessment for objectives usually implies:– Collection of data relative to the achievement of each objective
• Surveys are normally used.
– A priori establishment of performance goals
• No requirement that program improvements must have been made.– But assessment and evaluation results must be documented,
including decisions as to whether improvement might be needed.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
18
Notes on Criterion 4 (3)
• Note that the use of data/ideas from sources other than the assessment of objectives and outcomes achievement can be used to improve the program.– This should not be discouraged, but it is not by itself
evidence of program improvement referred to in the 2008-09 CAC criteria.
• Proposed harmonized criteria recognize additional sources to suggest improvements.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
19
Criterion 5 - Curriculum
The program’s requirements are consistent with its educational objectives and are designed in such a way that each of the program outcomes can be achieved. The curriculum combines technical and professional requirements with general education requirements and electives to prepare students for a professional career and further study in the computing discipline associated with the program, and for functioning in modern society. The technical and professional requirements include at least one year of up-to-date coverage of fundamental and advanced topics in the computing discipline associated with the program. In addition, the program includes mathematics appropriate to the discipline beyond the pre-calculus level. For each course in the major required of all students, its content, expected performance criteria, and place in the overall program of study are published.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
20
Notes on Criterion 5• Program criteria provide further specificity.
• Note that the curriculum must be designed so that its outcomes can be achieved.– No mention of enabling a-i.– When might a-i enter into the picture relative to
the program’s curriculum?
• What does “… performance criteria … must be published” mean?
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
21
Criterion 6 – Faculty
A. Faculty QualificationsFaculty members teaching in the program are current and active in the associated computing discipline. They each have the educational backgrounds or expertise consistent with their expected contributions to the program. Each has a level of competence that normally would be obtained through graduate work in the discipline, relevant experience, or relevant scholarship. Collectively, they have the technical breadth and depth necessary to support the program.
Comment: Similar to present criteria.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
22
Criterion 6 – (cont.)
B. Faculty Size and WorkloadThere are enough full time faculty members to provide continuity, oversight and stability, to cover the curriculum reasonably, and to allow an appropriate mix of teaching, professional development, scholarly activities, and service for each faculty member. The faculty assigned to the program has appropriate authority for the creation, delivery, evaluation and modification of the program, and responsibility for the consistency and quality of its courses.
Comment: Similar to present criteria.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
23
Criterion 7 – Facilities Institutional facilities including the library, other electronic information retrieval systems, computer networks, classes, and offices are adequate to support the educational objectives and outcomes of the program.
Computing resources are available, accessible, systematically maintained and upgraded, and otherwise adequately supported to enable students to achieve the program’s outcomes and to support faculty teaching needs and scholarly activities. Students and faculty members receive appropriate guidance regarding the computing resources and laboratories available to the program.
Comment: Similar to previous criteria.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
24
Criterion 8 –Support
The institution’s support for the program and the financial resources available to the program are sufficient to attract and retain qualified faculty members, administer the program effectively, acquire and maintain computing resources and laboratories, and otherwise provide an environment in which the program can achieve its educational objectives and outcomes. Support and resources are sufficient to provide assurance that the program will retain its strength throughout the period of accreditation.
Comment: Similar to existing criteria.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
25
Criterion 9 – Program Criteria
Each program must satisfy applicable Program Criteria (if any). Program Criteria provide the specificity needed for interpretation of the General Criteria as applicable to a given discipline. If a program, by virtue of its title, becomes subject to two or more sets of Program Criteria, then that program must satisfy each set of Program Criteria; however, overlapping requirements need to be satisfied only once.
Note that this criterion will disappear (as a criterion: its requirements will remain) if the proposed harmonized criteria are adopted.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
26
(End of General Criteria)
• Questions/comments?
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
27
Existing CAC Program Criteria
• Computer Science
• Information Systems
• Information Technology
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
28
Program Criteria Notes (1)• Applicable program criteria are determined
by the title (name) of the program being evaluated.– Determined by HQ based on the program title
listed on the RFE.– Be alert to a possible mismatch between
expectations of program and HQ program criteria assignment.
• Check program requirements for consistency with assigned criteria.
• Do this well BEFORE the visit.– Graceful recovery from a problem is almost impossible
after the visit.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
29
Program Criteria Notes (2)
• If a program title does not imply one or more existing sets of CAC program criteria, it is evaluated under the General Criteria only.– Examples: Information Science, Digital
Forensics, Computational Science
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
30Computer Science Program Criteria Additions
Additional criteria added to General Criteria:
• Capabilities that must be enabled.
• Curriculum requirements.
• Faculty qualification requirement.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
31
CS Program Criteria (1)
3. Program OutcomesThe program enables students to achieve, by the time of graduation:
j) an ability to apply mathematical foundations, algorithmic principles, and computer science theory in the modeling and design of computer-based systems in a way that demonstrates comprehension of the tradeoffs involved in design choices;
k) an ability to apply design and development principles in the construction of software systems of varying complexity.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
32
CS Program Criteria (2)
5. Curriculum Students have the following amounts of course work or equivalent educational experience:
a. Computer science: One and one-third years that includes:1. coverage of the fundamentals of algorithms, data structures,
software design, concepts of programming languages, and computer organization and architecture.
2. An exposure to a variety of programming languages and systems.3. proficiency in at least one higher-level language.4. advanced course work that builds on the fundamental course work
to provide depth.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
33
CS Program Criteria (3)
5. Curriculum (cont’d)
b. One year of science and mathematics:1. Mathematics: At least one-half year that must include discrete
mathematics. The additional mathematics might consist of courses in areas such as calculus, linear algebra, numerical methods, probability, statistics, number theory, geometry or symbolic logic.
2. Science: A science component that develops an understanding of the scientific method and provides students with an opportunity to experience this mode of inquiry in courses for science or engineering majors that provide some exposure to laboratory work.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
34
Notes on CS Curriculum
• No probability and statistics requirement.
• No specific number of credits of science.
• No comments about communication skills, ethics, design, etc. since these are subsumed by the abilities in the program outcomes criterion.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
35
CS Program Criteria (4)
6. Faculty Qualifications
Some full time faculty members have a Ph.D. in computer science.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
36
Visit Forms
36
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
37
Visit Forms to be Discussed
• Program Evaluation Worksheet (PEW)
• PEV Visit Report (PER)
• Program Audit Form (PAF)
• Program Evaluation Tracking Form (PET)
• Short Form (SF)
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
38
Program Evaluation Worksheet (PEW)
• Completed by each PEV.• Columns for before visit, after day 0 and day 1,
and at the exit meeting.– Updated by PEV as the visit progresses.
• Used by the TC to track the evaluation status and as consistency check for the Draft Statement.– Collect a copy before the visit and at the end of the
visit.
• (View form.)
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
39
PEV Visit Report (PER)
• Replaces the previous VRF.• Much shorter than before.• Includes curriculum and transcript evaluation
that is completed before the visit.• Includes specific information about OO&A.• Principles for completion are the same as
before.• Includes the draft of the PEV’s section(s) of the
Draft Statement (exit meeting statement).• TC combines all PERs into a single file for
submission to HQ.• (View form.)
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
40
Program Audit Form (PAF)
• Similar to previous form.
• TC works with PEVs to produce one form for each program.– Consistency with exit meeting statement/draft
statement is essential.
• TC leaves forms with dean after exit meeting.
• TC combines forms for all programs into a single file for submission to HQ.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
41
Program Evaluation Tracking (PET) Form
• Similar to the previous PEA.• Created by TC for each program from the
PEWs.• Use the template.
– Eliminate Program Criteria sections not used.– Combine all PETs into a single file, in alphabetical
order of program name.• Used to track the evaluation status from the Exit
Meeting to the Final Statement, including editing.• (View form.)
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
42
Short Form (SF)
• One form for all programs at an institution.– Essentially the same as what is used by other
commissions.
• TC prepares SF after the visit.
• (View form.)
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
43
Deliverables
• From PEVs to TC (at end of visit)– Program Evaluator Worksheet (PEW)– Program Evaluator Report (PER)– Program Audit Form (PAF)
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
44
Deliverables (cont.)
• From TC to ABET HQ ([email protected])– Within 2 days after visit: Initial SF– Within 30 days after visit (5 files):
• Short Form (SF)• Draft Statement (DS)• PET for each program (one file)• PAF for each program (one file)• PER from each PEV (one file)
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
45
Draft Statement
45
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
46
Draft Statement (2)
• There can be at most one each of deficiency, weakness, and concern per criterion.– A shortcoming may have multiple factors that
contribute to the shortcoming.– See example statements.
• Note that shortcomings in program criteria are listed under Criterion 9 (Program Criteria), not merged into other criteria.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
47
Draft Statement (3)
• Do not use “category”.
• Use “criterion” instead.
• Applies to old (as well as new) criteria.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
48
Draft Statement (5)
Section for each program:
1. General description (faculty, students, etc.)
2. Strengths
3. Shortcomings
4. Observations
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
49
Draft Statement (6)
Strength statements:
• What was observed.
• What makes it stand out above the norm.
• What positive impact it has on the program.
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
50
Draft Statements (7)
Shortcomings
• Criterion citation
• What was observed that causes a shortcoming
• Negative impact as a result of the observed characteristics
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
51
Statement and Forms Processing Summary
51
Cop
yrig
ht ©
200
8 by
AB
ET
, In
c.
52
Statement and Forms Processing Summary (1)
1. TC collects PEWs, PERs, and PAFs from PEVs.
2. TC creates SF and PETs.• Submits SF to HQ.
3. TC creates draft DS.
4. TC submits files to HQ.• DS, SF, PERs, PETs, PAFs