graduate attributes - mcgill university graduate attributes 3 graduate attributes are the qualities,

Download Graduate Attributes - McGill University Graduate Attributes 3 Graduate attributes are the qualities,

If you can't read please download the document

Post on 20-May-2020

0 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • Graduate Attributes Assessment and Reporting

    For 2016 Accreditation

    October 2015

  • Accreditation 2016

    • 10 of 11 Engineering programs up for accreditation in 2016

    • We “must demonstrate that the graduates of a program possess

    the [12] attributes” and “there must be processes in place that

    demonstrate that program outcomes are being assessed in the

    context of graduate attributes, and that the results are applied to

    the further development of the program.”

    2

  • Graduate Attributes

    3

    Graduate attributes are the qualities, skills and understandings a

    university community agrees its students should develop during

    their time with the institution and consequently shape the

    contribution they are able to make to their profession and

    society. They are qualities that also prepare graduates as agents

    of social good in an unknown future. (Bowden et al., “Generic Capabilities of Australian Technology Network University Graduates”, http:/www.clt.uts.edu.au/ATN.grad.cap.project.index.html, 2000)

  • Graduate Attributes - CEAB “Generic characteristics, specified by the Accreditation Board, expected to be exhibited by graduates of accredited Canadian engineering programs at the time of graduation.”

    The graduate attributes broadly define the required outcomes, but they are difficult to measure directly. 4

    1. KB: Knowledge base for engineering 2. PA: Problem analysis 3. IN: Investigation 4. DE: Design 5. ET: Use of Engineering tools 6. IT: Individual and teamwork 7. CS: Communication skills 8. PR: Professionalism 9. IE: Impact of engineering on society and the environment 10. EE: Ethics and equity 11. EP: Economics and project management 12. LL: Life-long learning

  • Indicators

    • To measure graduate attributes, all universities must define their own measurable descriptors called “indicators”:

    o “Descriptors of what students must do to be considered competent in

    the attribute; the measurable and pre-determined standards used to

    evaluate learning (i.e. measureable characteristics of attributes or

    components of attributes).”

    Note: We are required to “measure” indicators, not GAs

    5

  • Complete Process

    Defining Graduate Attribute

    Indicators

    Curriculum Mapping

    Collecting Data of Student Learning

    Analysis and Interpretation

    Data- informed

    Curriculum Improvement

    6

    • Core Graduate Attribute indicators are defined o Some graduate attribute indicators will ultimately be customized for each department

    • Pilot projects have been undertaken to evaluate how best to report

    results of Graduate Attribute assessment

    • Graduate Attribute curriculum maps have been prepared for each

    program

    • Graduate Attribute indicators must be assessed and reported for

    2015-2016 academic year (as a minimum)

  • Objectives

    • Two principal objectives in developing accreditation

    procedures for the Engineering programs:

    1) Maintain the accreditation of our programs in the new CEAB

    accreditation context

    2) Do so with as little additional work imposed on instructors

    7

  • Procedure • Quantitatively assess all GAs at the level of “D” (Developed) or “A”

    (Applied) in a courses, in all years, for all students, by correlating

    performance in “assessment tools” with that in GAs / indicators

    • However, to get satisfactory data for all GAs in a given program,

    departments must also o use other approaches to evaluate “soft skills” GAs (e.g., IT, CS, PR, IE, EE, EP,

    LL) o employ other approaches to evaluate GAs other than just correlating GAs with

    graded work

    • A special emphasis on obtaining data for the “soft skills” GAs will be

    undertaken in FACC 100 and FACC 400 (which are common to all

    programs).

    8

  • Procedure (cont.) • The faculty is providing general support to assist departments in

    meeting the necessary requirements to obtain, analyze and

    present their GA data.

    • However, every course, program and department is different, so

    the process cannot satisfy all situations

    o Adaptation and customization will be required by each department

    o CEAB guidelines and expectations are very vague

     Although the faculty can provide guidance, professors and departments will sometimes need to use their judgment, as many questions do not have clear answers!

    9

  • Overview Department

    Program

    General approach - infer GAs from assessment tools in courses (i.e. correlating grades with individual GAs)

    Alternate approach - ensure good coverage of GAs is obtained - used to evaluate GAs by other means

    Excel (analysis is the responsibility of the prof. / dept.)

    myCourses (most courses, Faculty does the analysis)

    10

  • Reporting Assessments

    myCourses Gradebook Setup

    Excel Format

  • General Approach Use myCourses for the majority of courses to correlate individual student performance in certain GAs with the various assessment tools used by the instructor

    • The gradebook setup can be done centrally, if requested by the instructor

    • Professors (only) to associate graduate attribute indicators with each graded item

    • Professors (or TAs) are responsible for entering data (grades) in myCourses at a level that is sufficiently refined to distinguish between different graduate attribute indicators

    • Analysis of the data will be done centrally

    12

  • myCourses Setup Options Two options to setup myCourses gradebook

    1) The gradebook setup can be done centrally, if requested by the instructor

    o Information is submitted through a general questionnaire

    2) The gradebook setup can be done by the instructor or TAs

    o One-on-one consultation with professors will be available

    o Hands-on TA training will be offered in mid-October

    13

  • 14

  • Pre-Setup Tasks • Define the role of your TA:

    o http://kb.mcgill.ca/#tab:homeTab:crumb:8:artId:4131:src:article

    15

  • Pre-Setup Tasks (cont.) • Define the gradebook rules:

    Student View Display Options How to treat ungraded items

    16

  • Information for General Setup • The assessment tools for your course • The grade distribution

    17

  • Information for General Setup (cont.) • The grade distribution within each category:

    Example:

    18

    Question#1 Graded out of 20 10% 4.50% Question#2 Graded out of 20 10% 4.50% Question#3 Graded out of 20 10% 4.50% Question#4 Graded out of 60 30% 13.50% Question#5 Graded out of 80 40% 18%

    100% 45%

    Final exam 45%

    Distribution based on grade points

    • Even distribution

    • Distribution by grade point

    Assignment#1 Graded out of 20 20% 1.80% Assignment#2 Graded out of 80 20% 1.80% Assignment#3 Graded out of 80 20% 1.80% Assignment#4 Graded out of 50 20% 1.80% Assignment#5 Graded out of 100 20% 1.80%

    100% 9%

    Even distribution of grades Assignments

    9%

  • Information for General Setup (cont.)

    19

    Determine the Graduate Attribute indicators for each component within a graded assessment tool

  • General Setup

    20

    Step 1 – Create the assessment tool categories

    Step 2 – Create the sub-categories (items) & linking the Graduate Attribute indicators

    Graduate attribute indicators

  • Step 1 – Create Assessment Tools Categories

    21

  • Example

    22

  • Step 2 – Create the Sub-categories (items) & Linking the Graduate Attribute Indicators

    23

  • Example Case1: The grade distribution for the Final Exam is based on the grade points

    Case2: The grade for the Assignments category is evenly distributed between the items

    24

    Final Exam Question 1: • Graded out of 50 • Graduate attribute indicator: KB.7 • KB.7 - Recalls and defines information, first principles

    and concepts in specialized engineering science

    Question 2: • Graded out of 100 • Graduate attribute indicator: PA.2 • PA.2 - Develops models from first principles to analyze

    complex engineering problems

    Question 3: • Graded out of 80 • Graduate attribute indicator: PA.4 • PA.4 - Critically evaluates the validity and accuracy of

    existing solution methods

    Assignments Assignment 1: • Graded out of 10 • Graduate attribute indicator: KB.3 • KB.3 - Recalls and defines information, first principles

    and concepts in the natural sciences

    Assignment 2: • Graded out of 100 • Graduate attribute indicator: PA.1 • PA.1 - Identifies and formulates complex engineering

    problems

  • 25

Recommended

View more >