applying and assessing a hierarchical model of mental growth to educate undergraduate engineering...

Proceedings of the 2002 ASEE/SEFI/TUB ColloquiumCopyright © 2002, American Society for Engineering Education

Applying and Assessing a Hierarchical Model of Mental Growthto Educate Undergraduate Engineering Students

S.M. Husson†, D.A. Bruce†, C.H. Gooding†, G.M. Harrison†,D.E Hirt†, S.M. Kilbey II†, R.W. Rice†, D.M. Switzer‡

†Dept. of Chemical Engineering, Clemson University, Clemson, SC 29634 /‡School of Education, Clemson University, Clemson, SC 29634

This article is the second in a series of ASEE proceedings that describes the implementation andassessment of using a hierarchical model of mental growth as the basis for developing criticalthinking skills and engineering judgment in engineering undergraduates. Briefly, the hierarchicalmental growth model of Egan provides our roadmap for developing effective teaching andlearning strategies that are applied to core engineering courses taught in the sophomore andjunior years. These strategies strengthen low-level cognitive skills in sophomores and juniorsthat provide a proper foundation on which high-level cognitive skills can be developed. Ourassessment instruments track individual students, allowing us to monitor student growth andevaluate the effectiveness of these teaching and learning devices for populations with differentexposures to experimental treatments. Results to date indicate that a significant difference exists(p < 0.06) in metacognitive self-regulation between students who have one experimentaltreatment and those with two or more. The latter students are more effective at planning,monitoring, and regulating their cognitive activities than the former. They tend also to participatein a task more often for reasons such as challenge, curiosity, or mastery; and, they express morepositive attitudes towards professional development.

Introduction

Previously1, we introduced an approach to integrate a hierarchical mental growth model intoan undergraduate engineering curriculum, described teaching and learning strategies to supportthat model, and presented preliminary results for the assessment of implementation of thosestrategies on student development. Briefly, the hypothesis that drives this work is this: Mentalgrowth constitutes a progression through a hierarchy of cognition; the critical thinking andjudgment required of engineers lies at an upper level in the hierarchy, and, to reach high levels,an individual must master cognitive skills and reorganize knowledge gained at lower levels.

Our overarching goal is to develop higher-level thinking skills in chemical engineeringstudents before they reach their senior years. To reach that goal, we are applying the hierarchicalmental growth model of Egan2 as the basis for developing teaching and learning devices that areused in core sophomore- and junior-level chemical engineering courses. Figure 1 describes thecognitive levels of the model and summarizes other key aspects of the program.


The Hierarchical Model of Mental Growth

In this model, Ironic → exceptions, learning by modelingdevelopment of Philosophic → logic, learning by formal reasoninghigh-level skills Romantic → graphics, written learningrequires mastery Mythic → language, oral learningof skills at lower Somatic → body, tactile learninglevels.

Implementation of ActivitiesImplementation follows the hierarchy; font size implies emphasis of level.

Courses Involved in this Study

ChE 211 – Material and Energy BalancesChE 220 – Thermodynamics IChE 311 – Fluid FlowChE 312 – Heat and Mass TransferChE 321 – Thermodynamics II

SOPHOMORES

JUNIORS

*Senior courses are not formally a part of this study. What are shown are the typical emphasesfor a traditional senior-level course.

Main Objectives of this Research

• Devise teaching and learning devices that provide a foundation of low-level cognitive skills that support rapid and effective development of critical thinking,

• Devise assessment instruments for monitoring the development of that foundation in individual students, and

• Devise teaching and learning devices that build on the foundation to exercise high-level cognition required for critical thinking and engineering judgment.

*

Figure 1. Summary of the hierarchical cognitive model and key aspects of this research.


In our research to-date, we have designed and used activities in our sophomore and juniorcourses to involve students in the lower levels. Table 1 lists some of these activities, categorizedaccording to the cognitive level in the hierarchy that they exercise. Some activities, such as theself-reflections, provide opportunities for the students to evaluate their metacognitivedevelopment, that is, their evaluation of the process(es) by which they learn material mosteffectively.

TABLE 1Short list of activities used to exercise skills within levels of the hierarchy.

These activities were used in experimental sections.

• Somatic skills– In-class demonstrations & hands-on activities with materials – 3-D PvT clay models– In-class design experiments (also Romantic and Philosophic)

• Mythic skills– Glossary of technical terms– Photographs & stories about important scientists– Self-reflections on progress towards reaching stated personal goals– Group project (also exercised Romantic, Philosophic, and Ironic skills)

• Romantic skills– PvTgraphical projections– Learning exercise on how to read technical material– Developing graph/equation and graph/process equivalents– Equation interrogation

• Philosophic skills– Identifying patterns (e.g., forms of the fundamental equation)– Uniform problem solving strategy

• Ironic skills– Selection of property models

The specially designed activities related to the hierarchical model are implemented in“experimental” versions of the courses, while “control” groups are taught in the traditionalmanner. As shown in Table 2, the experimental (designated ‘e’) and control (designated ‘c’)courses are staggered over a 3-year period. The first four courses in the sequence are taughttwice per year to accommodate the large number of co-op students in the department.


TABLE 2Courses involved in the study. Experimental sections are labeled as ‘e’;

control sections are labeled as ‘c’.

Year Spring Fall

2000 --- 211c,e

2001 211c 211e220e 220c

311e

321e

2002 211c 211e220e 220c311c 311e312e321c 321e

2003 ---220e311c312c,e321c

Validity

The ideal plan to assess the effectiveness of the experimental treatments* would involve theseattributes:

• Parallel sections would be formed for each course, with some of the students enrolled in thecontrol sections and the remaining students enrolled in the experimental sections.

• The same professor would teach both sections of a given course.• The cohorts of students would be similar in terms of defined criteria (e.g., GPA, fraction of

commuting students, fraction of co-op students), and the students would not cross over fromthe control to the experimental sections and vice versa.

Given the constraints of our program (multiple co-op tracks, personnel limitations), we decidedto proceed without forming special cohorts and to evaluate the program using several assessmentinstruments. These instruments allow us to track individual students; data from individualstudents can then be used to identify populations with similar exposure to experimentaltreatments (e.g., population A had five ‘e’ courses, population B had four…).

* The term treatment refers to the technique or activities being assessed in the study.


One potential drawback to this approach is that few students may progress through theprogram with no experimental treatments. Such students, however, represent an importantcontrol; human beings change over time in the normal course of development and acquisition ofexperience.3 With too few students who progress with no experimental treatments, it will bedifficult to distinguish between normal changes over time and those caused by the experimentaltreatments.

To evaluate the effectiveness of the cognitive-model approach, several instruments were adoptedor developed; see Ref. 1 for more information about these instruments. The SBI and PDS can beviewed at www.ces.clemson.edu/chemeng/cog-mod/.

• Survey of Basic Information (SBI) — Demographic information• Professional Development Survey (PDS) — Personality traits, learning v. performance goals• Motivated Strategies for Learning Questionnaire (MSLQ)4 — Metacognitive strategies• Chemical Engineering Achievement Test — Technical knowledge• Student Portfolios — Documentation of student growth and self reflection

Results and Discussion

Table 3 lists the courses involved in the study to date. Details concerning demographics,teaching and learning devices, grading scales, and results are presented in that table and in thissection. Note that the grading scale percentages for each major category were kept nearly thesame, although there were slight variations in requirements in some of the categories, asindicated by the asterisks.

ChE 211e,c: This course is a 4-credit course comprising three 50-minute “lectures” and one 75-minute recitation per week. Table 3 outlines grading scale differences between “e” and “c”sections for this and the other courses. Other important differences occur in the recitation andmetacogitive activities. The recitation in the control section is a typical session devoted toworking example problems and reviewing homework problems. There is no explicit discussionon the process of learning (metacognition); the professor professes and the students “learn”. Bycontrast, the recitation activities in 211e are designed to involve the somatic, mythic, andromantic levels in the hierarchy, with emphases on calculations and relating plots and data. Thestudents work in groups of two (with a different partner each week) answering 5-10 questionsfrom a worksheet on a particular topic (e.g., measurements, manometers, energy balances). Thestudents collaborate within groups, across groups, and with the professor. The groups turn intheir results, which are graded and counted toward the course grade. The somatic level on thehierarchy is covered by the hands-on activity. Group discussions hint at the mythic level, whilethe calculations hint at the romantic and philosophic levels.

ChE 220e,c: This course is a 3-credit course comprising three 50-minute “lectures”. There is norecitation associated with this course, but the lower levels on the hierarchy are emphasized, aswell as the metacognitive activities of problem-solving strategies and reflective exercises.Students in experimental classes engage in in-class activities (e.g., windmill testing, clay modelmaking of PvT surfaces) to strengthen somatic skills. The instructor for “e” sections uses asingle, uniform methodology to solve all problems. To develop metacognitive skills, students in


“e” sections are assigned a series of instructor-driven reflective exercises. Figure 2 provides anexample reflective exercise, along with several student responses (in italics).

ChE 220: Chemical Engineering Thermodynamics ISpring Semester 2002

Reflective Exercise 2

Name: _____________________ SSN: _____________________ Date: ____________

Now that you have had the first 15-minute quiz, you should think about how you are progressingand how you plan to improve. Please respond to the following questions on a separate page.Please type your response and submit it with this page attached. This reflection should be placedin your portfolio after it is returned to you.

1. Look at the graded quiz and list important facts or concepts that you have learned from thecourse but that you did not know before starting the course.

I learned how to accurately draw PvT diagrams and label all the points on them.I am now able to distinguish among fluids, gases, liquids, solids, and vapors using data.I know what isothermal compressibility and volume expansivity are, and I can write their definitions.

2. Again, look at the quiz and list topics that you did not understand to your own satisfaction.

I did not know how to represent on the graph that water expands on freezing.The definition of an isotherm is clear, but is difficult to visualize where one falls on a Pv diagram.Where does the triple point on a Pv diagram go?

3. Your sources of information for the course have been (a) lectures, (b) textbook, (c)homework, and (d) instructional objectives. Of these, which one was the primary source foreach of the items listed in Part 2? Does this fact suggest a source to which you should bepaying more attention?

I need to attempt to understand the mistakes that I make on homework problems.The lectures and the textbook would be the primary sources for topics I failed to comprehend.I should look at pictures in the book and understand what they are trying to teach me.

4. How do you plan to improve before the first midterm and for subsequent quizzes/midterms?

Keep up with the material daily, instead of cramming.I need to follow the instructional objectives because they clearly outline what is expected of me.I am going to meet with classmates to do some group studying.

Figure 2. Example Reflective Exercise used in sophomore-level thermodynamics (ChE 220).

ChE 311e,c; ChE 312e,c; ChE 321e,c: All junior-level courses are 3-credit courses comprisingthree 50-minute “lectures” or two 75-minute “lectures”. There are no recitations associated withthese courses; romantic and philosophic levels on the hierarchy are emphasized, as well as themetacognitive activities of problem-solving strategies and reflective exercises. Some in-classdemonstrations and hands-on activities are used, but to a lesser extent than in 200-level courses.

TABLE 3Course information and grading scales. Experimental sections are labeled as ‘e’; control sections are labeled as ‘c’.

An asterisk identifies an item that was included in the homework grade.

Course 211e 211c 211c 220e 211e 220c 311e 321e 211c 220e 311c 312e 321cSemester Fall 2000 Fall 2000 Sp 2001 Sp 2001 Fall 2001 Fall 2001 Fall 2001 Fall 2001 Sp 2002 Sp 2002 Sp 2002 Sp 2002 Sp 2002Instructor Haile Haile Kilbey Husson Gooding Barron Harrison Husson Gooding Husson Hirt Bruce Barron

Lecture schedule (50 min for 8:00 MWF 9:05 MWF 9:05 MWF 10:10 MWF 8:00 MWF 10:10 MWF 9:30 TTh 9:05 MWF 9:05 MWF 10:10 MWF 11:00 TTh 9:05 MWF 10:10 MWFMWF; 75 min for TTh)

Recitation schedule (75 min) 2:00 T 3:15 T 2:00 T N/A 2:00 T N/A N/A N/A 2:00 T N/A N/A N/A N/AStudents enrolled 30 24 18 40 48 17 31 39 13 21 18 44 18

Homework 16 16 12.5 15 12.5 15 20 12 ext.cred. 15 10 14 10

Problems * * * * * * * * * * * *Surveys * * * * * * * * * * * *Refl. Exer. * * * * * *Porfolios * * * checked *

Recitation Exercises 16 16 12.5 N/A N/A N/A N/A N/A N/A N/AProject N/A N/A N/A 10 12.5 N/A N/A 10 10 13 N/A 5 10Class Participation 10 5Tests 44 44 50 36 50 45 50 39 60 36 60 36 45Quizzes 14 12 14 15Refresher Quiz N/A N/A N/A N/A N/A N/A N/A 5 N/A N/A N/A N/A N/A

Final Exam 24 24 25 25 25 30 30 22 30 22 30 30 30

Course GradesA 3 5 3 2 8 3 4 1 2 3 2 6 4B 6 3 1 11 14 8 8 13 2 6 4 10 4C 7 7 7 19 9 2 12 19 4 9 9 21 6D 5 3 1 5 1 2 3 6 1 1 2 6 3F 3 4 4 2 6 1 3 0 1 2 1 1 1

Withdrawals 6 2 2 1 10 1 1 0 3 0 0 0

GPR 2.04 2.09 1.88 2.15 2.45 2.47 2.23 2.23 2.30 2.33 2.22 2.32 2.39


Assessment

Profess ional Development Survey. As described in a previous ASEE proceedings1, we createdand tested a Professional Development Survey (PDS) taken by students at the start and end ofeach semester. Table 4 shows the PDS domains and the sources of the assessment items for eachdomain, and gives the most recent reliability indices for each domain. Table 4 also providesexamples of measured traits and test items within each assessment domain.

TABLE 4Creation and validation data for the Professional Development Survey (PDS)

AssessmentDomain

Source of DomainItems

ReliabilityIndex

Measured TraitExamples

Test ItemExample

ConscientiousnessInternational

Personality Item Pool5 0.91Preparedness,

Attention to detailsI do things according

to a plan.

IntellectInternational

Personality Item Pool5 0.83Ability to“catch on”

I carry conversationsto a higher level.

Learning GoalsRoedel, Schraw

& Plake6 0.87Internal need

to knowI prefer challenging tasks,even if I do not do well.

PerformanceGoals

Roedel, Schraw& Plake6 0.74

Desire forthe grade

I want others to think thatI know a lot.

Subject Matter Research team 0.65Attitude toward

curriculum contentThe coursework in Chemical

Engineering is easy.

ProfessionalDevelopment

ABET criteria 0.87Attitude toward

Prof. developmentI can identify, formulate, andsolve engineering problems.

ChemicalEngineering

Research team 0.76Attitude toward

field of Chem. Eng.Chemical engineers are

respected by society.

Reliability refers here to the internal consistency of the test scores. The reliability index isCronbach’s alpha; it measures how well the test items measure a single variable (e.g., intellect).Each test item is designed to measure some aspect of that variable; thus, they shouldintercorrelate. A reliability index of 0.91 means that 91% of observed score variance would becaused by true score differences among people taking the test; 9% of variance would be causedby errors. Such errors take many forms and include items such as familiarity with the surveyformat; motivation, health, fatigue of the individual taking the survey; physical conditions of theroom in which the survey is taken; among others.3

MSLQ Survey. In Spring 2002, 34 students “graduated” from the study by completing the finalcourse in the series. Of these students, 10 had been in one “e” course, 8 had been in two, 12 hadbeen in three, and 4 had been in four. Using ANOVA methods applied to the PDS and MSLQsurvey data, one significant result was found for those with only one “e” course, compared to theother groups: Metacognitive Self Regulation was lower (p < 0.06) for these students. [A p-valueof 0.06 means that there is a 6% chance that differences among these groups occurred bychance.] This result says that students with more exposure to experimental treatments tend toplan, monitor, and regulate their cognitive activities better than those students with less exposure.Students with more “e” courses tend to engage more often in activities such as goal setting, self-


testing and questioning, and fine-tuning their cognitive activities by checking and correctingtheir behavior as they proceed on a task.4

ChE Achievement Test. We have done an item analysis of the Chemical EngineeringAchievement test that, so far, has been given to 34 students. This test contains 17 items and istaken by students following the final junior-level course. To date, the mean total score is 54 outof a possible 85, with a standard deviation of 10.5. Based on the item analysis, the test reliabilitycoefficient was calculated to be 0.70. This value is quite good, and means that studentperformance on each item of the test correlates well to performance on the test as a whole.

Worth Noting. We recently conducted an analysis of those students who completed twosophomore-level courses (ChE 211 and ChE 220). There were 16 students who had been in one(1) experimental class, and 17 who had been in two (2). Using ANOVA methods applied to thePDS and MSLQ survey data, three differences were identified between the two groups: Therewere slight differences on Intrinsic Goal Orientation (p < 0.09) and Metacognitive Self-Regulation (p < 0.09), with students who had two experimental classes showing higher scores.Metacognitive Self-Regulation was described above; Intrinsic Goal Orientation relates to thedegree to which a student participates in a task for reasons such as challenge, curiosity, ormastery, i.e., participating as an end all to itself, rather than as a means to an end.4

There was a highly significant difference (p < 0.0056) on ABET scores. This domain measuresattitude toward professional development based on ABET criteria. Students who had twoexperimental classes had the more positive attitudes towards professional development.

Summary

Over the course of four semesters we have begun to test the effectiveness of a hierarchicalcognitive model on learning processes of chemical engineering undergraduates. Thus far, twosophomore-level courses (Material & Energy Balances and Thermodynamics I) and three junior-level courses (Fluid Flow, Thermodynamics II, and Heat & Mass Transfer) have been taught in amodified fashion by incorporating activities that specifically reinforce the lower levels in thehierarchical model. The goal is to build on the lower-level skills and progress to effective use ofhigher-level thinking skills to solve more complex problems. Results to-date indicate thatstudents who are exposed often to these experimental activities tend to plan, monitor, andregulate their cognitive activities more effectively than students with limited exposure. Therealso appear to be borderline significant benefits to students who have more experimental courses,including higher Intrinsic Goal Orientation and more positive attitude toward professionaldevelopment. Anecdotally, instructors in experimental courses have reported a perceptibledifference in students compared to students in control sections – students in experimental coursesexhibited more confidence with material, higher frustration threshold, greater willingness towork, better perception of goals, and lower resistance to professor-student interaction. We willcontinue to emphasize activities related to the hierarchy in sophomore- and junior-level coursesand quantitatively and qualitatively assess the students’ technical and metacognitive growth.

Acknowledgements

This research was supported by the National Science Foundation (Award DUE-9952348).


References

1. Husson, S.M.; Hirt, D.E.; Bruce, D.A.; Gooding, C.H.; Haile, J.M.; Harrison, G.M.; Kilbey, S.M., II; Rice,R.W.; Switzer, D.M. Applying a Hierarchical Model of Mental Growth to Educate Undergraduate EngineeringStudents: Preliminary Assessment. Proceedings of the 2002 American Society for Engineering EducationAnnual Conference & Exhibition, ASEE, Montreal, Quebec, 2002.

2. Egan, K. The Educated Mind; University of Chicago Press: Chicago, 1997.3. Tuckman, B.W. Conducting Educational Research, 3rd ed.; Harcourt Brace Jovanovich: San Diego, 1988.4. P. R. Pintrich, D. A. F. Smith, T. Garcia, and W. J. McKeachie. A Manual for the Use of the Motivated

Strategies for Learning Questionnaire (MSLQ), University of Michigan.5. Goldberg, L.R. “International Personality Item Pool” http://ipip.ori.org/ipip/ (last accessed Sept. 13, 2001).6. Roedel; Schraw; Plake. Validation of a Measure of Learning and Performance Goal Orientation. Educational

and Psychological Measurement 1994, 54, 1013-1021.

Biographies

SCOTT M. HUSSONDr. Husson is an Assistant Professor of Chemical Engineering at Clemson University. He received his bachelor'sdegree in chemical engineering from Penn State University in 1993 and his doctoral degree in chemical engineeringfrom UC Berkeley in 1998. Dr. Husson is the recipient of the 2000 NSF Presidential Early Career Award forScientists and Engineers. He has been a member of ASEE since 1998.

DOUGLAS E. HIRTDr. Hirt is an Associate Professor of Chemical Engineering at Clemson University. He received his bachelor's andmaster's degrees in chemical engineering from Virginia Tech in 1982 and 1984 and his doctoral degree in chemicalengineering from Princeton in 1989. Dr. Hirt received the Raymond W. Fahien Award of ASEE in 1998. He hasbeen an active member of ASEE since 1992, and has served as Chair of the ChE Division of ASEE.

DAVID A. BRUCEDr. Bruce is an Associate Professor of Chemical Engineering at Clemson University. He received bachelor's degreesin chemistry and chemical engineering from Georgia Tech in 1991 and 1992 and his doctoral degree in chemistryfrom Georgia Tech in 1994. Dr. Bruce is the recipient of the 2000 NSF Career Award. He has been an activemember of ASEE since 1998.

CHARLES H. GOODINGDr. Gooding is a Professor of Chemical Engineering at Clemson University. He received his bachelor's and master'sdegrees in chemical engineering from Clemson University in 1970 and 1972 and his doctoral degree in chemicalengineering from North Carolina State University in 1979. He has been an active member of ASEE since 1993.

GRAHAM M. HARRISONDr. Harrison is an Assistant Professor of Chemical Engineering at Clemson University. He received his bachelor'sdegree in chemical engineering from Stanford University in 1991 and his doctoral degree in chemical engineeringfrom UC Santa Barbara in 1997. Dr. Harrison has been an active member of ASEE since 1999.

S. MICHAEL KILBEY IIDr. Kilbey is an Associate Professor of Chemical Engineering at Clemson University. He received his bachelor'sdegree in chemical engineering from the University of Wisconsin in 1990 and his doctoral degree in chemicalengineering from Minnesota in 1996. Dr. Kilbey is the recipient the 2000 Dow Outstanding Teacher Award and the1999 Clemson University Alumni Master Teacher Award. He has been an active member of ASEE since 1998.


RICHARD W. RICEDr. Rice is an Associate Professor of Chemical Engineering at Clemson University. He received his bachelor'sdegree in chemical engineering from Clemson University in 1968 and his doctoral degree in chemical engineeringfrom Yale in 1972.

DEBORAH M. SWITZERDr. Switzer is an Associate Professor of Education at Clemson University. She received her bachelor's degree inmathematics from the University of Texas at Austin in 1976 and her master's and doctoral degrees in educationalpsychology from the University of Illinois at Urbana-Champaign in 1987 and 1993.

applying and assessing a hierarchical model of mental growth to educate undergraduate engineering...

Documents