Efficacy of Creative Problem Solving: Oh the Places

Deliberate Creativity has Gone

Parnes TributeMay 9, 2009

Gerard J. Puccio, Ph.D.

International Center for Studies in Creativity

Historical Research on CPS: Pre-CRS Project & CRS Project

CPS Research

• 1949-1956 Pilot Experimentation and Course Development

• 1957-1967 Systematic Research of the Impact of CPS Methods and Programs

• 1969-1972 Creative Studies Project, a Experimental Investigation of the Impact of a Sequence of Creativity Courses on College Students

1957-1967 Specific Research Issues

• Effects of a Semester long Course on the Development of Creativity

• Relative Effects of a Creativity Course used Alone or Taken with an Instructor

• Effects of Extended Effort in CPS

• Effectiveness of the Principle “Defer Judgment”

1957-1967: Effects of a Creativity Course

• Significant increase for two measures of quantity of idea production

• Significant increase for three of five measures of quality of ideas

• Long-term results seemed to be evident up to four years after the course in creativity

1957-1967: Alone or Taken with an Instructor

• On almost all measures the instructor-taught groups outperformed control groups

• Students who took program alone were superior to control groups

• Instructor-taught groups out performed students who took course alone

1957-1967: Extended Effort

• Extended effort resulted in a significantly greater proportion of good ideas among the later ideas generated

1957-1967: Defer Judgment Principle

• Significantly more good ideas generated by individuals working under the ‘defer judgment’ principle than those instructed to use concurrent judgment

• Groups trained in CPS generated significantly more good ideas than those using discussion

Creative Studies Project: An Overview

• Pilot study began in 1969

• Courses based on CPS, Synectics, and Creative Analysis

• Students randomly selected (30%) from the incoming freshman class

• Students randomly assigned to groups

• Experimental subjects took a sequence of 4 creativity courses

• National advisory committee

Some Members of the Advisory Committee

• John Curtis Gowan, Professor of Education, San Fernando Valley State College

• J. P. Guilford, Professor of Psychology, University of Southern California

• Donald MacKinnon, Professor of Psychology, University of California at Berkeley

• Calvin Taylor, Professor of Psychology, University of Utah

• Donald Treffinger, Chairman of Educational Psychology, University of Kansas

Creative Studies Project: Overview of

Findings

Creative Studies Project: Overview of

Findings• Semantic Tests: Experimental subjects significantly better on 16 of

27.

• Figural Tests: No significant differences on 8.

• Symbolic Tests: One significant differences in favor of the E’s for 1 of 8 .

• Cognition Tests: 7 of 10 significant differences.

• Divergent Production Tests: 9 of 14 significantly different.

• Convergent Production Tests: 4 of 8 significant.

• Evaluation & Memory Tests: No significant differences.

Parnes, 1987

Creative Studies Project: Additional

Findings

Creative Studies Project: Additional

Findings• E’s significantly out performed C’s on 2 of

5 creativity-related English tests.

• Significantly higher scores for E’s on the production of original verbal images.

• Numerous anecdotal evidence from BSC instructors on the impact of the creativity program.

Recent Research on CPS

Ideation and Problem Finding and Solving in an Industrial Research Organization

Ideation and Problem Finding and Solving in an Industrial Research Organization

• Focus: To study the transfer of training in a “complete process of creative problem solving”.

• Participants: 220 engineers, engineering managers, and technicians in a consumer goods industrial company.

• Training Program: Experimental group received two-days of CPS training; control group received placebo treatment (i.e., Koestler film & discussion), an additional group received no training.

• Outcomes: Two weeks after training assessed transfer of the CPS course to workplace (e.g., problem finding and problem solving abilities, co-workers observations, supervisor ratings, preferences for ideation, etc).

Basadur, Graen, & Green (1982).

Ideation and Problem Finding and Solving in an Industrial Research Organization

Some Results

Ideation and Problem Finding and Solving in an Industrial Research Organization

Some Results• Preference for ideation in problem solving increased significantly for

CPS trained group.• Placebo and non-training group expressed significantly more negative

judgments and spent significantly more time on negative judgments while engaged in a problem-finding task (i.e., explore wishes for new products of the future).

• Quality of the product wishes of the CPS trained group was significantly better than the placebo and untrained groups.

• Two-weeks after training others rated (blind assessment) the experimental group as significantly more open-minded, less likely to jump to conclusions, and able to take unusual approaches to problems.

• See Basadur et al. for further results

Basadur, Graen, & Green (1982).

Reducing Recidivism Among Native Canadians

Reducing Recidivism Among Native Canadians

• Focus: To help at-risk populations make better choices and life decisions.

• Participants: 31 inmates with matched control group.

• Training Program: Pre-release program focused on life-skills training (designed around CPS, M-F, 11 weeks) and job experience.

• Outcomes: Assessed recidivism rates one year after completing job experience.

Place, McCluskey, McCluskey, &Treffigner, 2000.

Reducing Recidivism Among Native Canadians

• More than 98% of control group had subsequent contact with justice system (blue column).

• Only 39% of experimental group re-offended (blue column).

Gameplay Decisions Among Badminton Players

Gameplay Decisions Among Badminton Players

• Focus: To determine the effect of CPS training on game-play decisions.

• Participants: 24 physical education students, participants matched on ability in experimental and control groups.

• Training Program: Experimental group received three 10-minute mini-sessions and a 1.5 hour session on CPS.

• Outcomes: Game-play video coded by ‘blind’ observers for tactical decisions.

Everhart, Kernodle, Turner, Harshaw, & Arnold, 1999.

Gameplay Decisions Among Badminton Players - ResultsGameplay Decisions Among Badminton Players - Results

• Statistical analysis of overall data for experimental and control showed a significant difference in frequency of decisions (tactics used).

• Analysis also revealed that the experimental group made more quality decisions.

• Experimental group participants engaged in more complex strategies.

• The most dramatic shift in tactics occurred after the third 10-minute session and the fourth 1.5 hour CPS session.

Everhart, Kernodle, Turner, Harshaw, & Arnold, 1999.

CPS & Communication Behaviors:Quality of Ideas Generated

UntrainedUntrained

2812815005003523522532532929

TrainedTrained

61861813421342917917648648140140

Firestien UnpublishedFirestien Unpublished

Quality RatingQuality Rating

5 (best)5 (best)443322

1 (worst)1 (worst)

CPS & Communication Behaviors

Total ResponsesTotal Responses

Verbal CriticismVerbal CriticismVerbal SupportVerbal SupportLaughterLaughterSmilesSmiles

Ideas GeneratedIdeas Generated

UntrainedUntrained

2626

2.22.21.41.42.12.12.62.6

1313

TrainedTrained

3939

0.090.093.73.7

6.06.06.76.7

2727

Firestien & McCowan, 1988Firestien & McCowan, 1988

Stimulants to Creative Behavior Among Engineers

Stimulants to Creative Behavior Among Engineers

• Focus: To determine which management philosophies and practices stimulate or inhibit creative behavior among engineers.

• Participants: 242 engineers. • Outcomes: Engineers asked to evaluate nine different

management practices for relevance to their work and the degree to which they increases the possibility of being more creative at work.

• Results: Project groups and Creative Problem Solving methods had the most positive response (88% and 85%, respectively).

Evkall, 2000.

Effects of CPS Training on R&D Performance

Effects of CPS Training on R&D Performance

• Focus: To assess the extent to which a complete program in CPS would enhance performance at work.

• Participants: 106 R&D workers in experimental group, 35 R&D workers in control group.

• Training Program: CPS training delivered over a three-month period (18 hours total).

• Outcomes: Pre-post test measures administered for creative-thinking ability and work performance (both measured 6 to 11 months after training).

Wang & Horng, 2002.

Effects of CPS Training on R&D Performance - ResultsEffects of CPS Training on R&D Performance - Results

• Significant gains in creative-thinking abilities (e.g., fluency & flexibility).

• Only one significant difference found for a performance indicator. Experimental group participants were involved in a greater number of ‘service’ projects (i..e, research projects based on customer complaints or technical problems).

• Authors conclude that there was insufficient time to see results on other performance measures (i.e., publications and reports).

Wang & Horng, 2002.

International Center for Studies in Creativity Virtual Space

World Creativity and Innovation Week Venue

International Center for Studies in Creativity Virtual Space

International Center for Studies in Creativity Virtual Space

International Center for Studies in Creativity Virtual SpaceExcerpted from Uribe & Cabra (in press, Creativity & Innovation Management)

International Center for Studies in Creativity Virtual SpaceExcerpted from Uribe & Cabra (in press, Creativity & Innovation Management)

International Center for Studies in Creativity Virtual SpaceExcerpted from Uribe & Cabra (in press, Creativity & Innovation Management)

International Center for Studies in Creativity Virtual SpaceExcerpted from Uribe & Cabra (in press, Creativity & Innovation Management)

International Center for Studies in Creativity Virtual SpaceExcerpted from Uribe & Cabra (in press, Creativity & Innovation Management)

ReferencesBasadur, M., Graen, g. B., & Green, S. G. (1982). Training in creative problem solving: Effects on

ideation and problem finding and solving in an industrial research organization. Organizational Behavior and Human Performance, 30, 41-70

Ekvall, G. (2000). Management and organizational philosophies and practices as stimulants or blocks to creative behavior: A study of engineers. Creativity and Innovation Management, 9, 94-99.

Everhart, B., Kernodle, M., Turner, E., Harshaw, Cl, & Arnold, D. (1999). Gameplay decisions of univesity badminton students . The Journal of Creative Behavior, 33, 138-149.

Firestien, R. L., McCowan, R. J. (1988). Creative problem solving and communication behavior in small groups. Creativity Research Journal, 1, 106-114.

Parnes, S. J. (1987). The creative studies project. In S. G. Isaksen (ed.), Frontiers of Creativity Research: Beyond the Basics (156-188). Buffalo, NY: Bearly Limited.

Parnes, S. J., & Noller, R. B. (1972). Applied creativity: The creative studies project: Part II – Results of the two-year-program. The Journal of Creative Behavior, 6, 164-186.

Place, D. J., McCluskey, A. L. A., McCluskey, K. W., & Treffinger, D. J. (2000). The second chance project: Creative approaches to developing the talents of at-risk native inmates. The Journal of Creative Behavior, 34, 164-186.

Wang, C. W., & Horng, R. Y. (2002). The effects of creative problem solving on creativity, cognitive type and R&D performance. R&D Management, 32, 35-45.

Does Creativity Training Work?

A Meta-Analytic Study

(a must read for all creativity educators & trainers)

Scott, Leritz & Mumford:Purpose of Study

1. To provide reasonably compelling evidence for the effectiveness of creativity training

2. To identify the key characteristics of training that led to the success of the training efforts

(Scott, Leritz & Mumford, Creativity Research Journal, 2004, Vol 16, pp 361-388)

Criteria for Inclusion of Study in the Meta-Analysis (n=70)

1. Expressed focus on creativity

2. Clear description of procedures used in training, the population involved and the strategies involved in training delivery.

3. Study provided an exact description of measures used to assess creative performance.

4. Study provided statistics needed to assess effect size.

5. If several studies used the same data, only one publication was used.

Design of Studies

• Pre-test or posttest control group design

or• Pre-test or posttest no control group design

(pre-tests used as baseline for comparison)

Dependent Variables

• Divergent thinking (e.g., fluency, flexibility, originality, elaboration)

• Problem solving (e.g., production of original solutions to novel problems)

• Performance (e.g., behavior)

• Attitudes and behavior (e.g., reaction to creative ideas, efforts initiated)

External Validity Checks:To What Extent do Training Effects Generalize Across People & Settings

• Age (above and below 14 years of age)

• Setting (educational vs. organizational)

• Academic Achievement of participants

• Use of gifted sample

• Gender

• Year study was conduced (pre/post 1980)

Assessment of Course Content:Foundational Models

• Cognitive models

• Social models

• Personality models

• Motivational models

• Confluence models (i.e., supplemented cognitive models)

• Others (e.g., attitudes, blocks to creativity)

Assessment of Course Content:Focus of Instructional Material

& Exercises • Problem finding• Information gathering• Information organizing• Conceptual combination• Idea generation• Idea evaluation• Implementation planning• Solution monitoring

Delivery Method

• Classroom Exercise

• Field exercises

• Group exercises

• Realistic, domain based, performance exercises

• Computer exercises

• Written exercises

• Self-paced exercises

• Imaginative exercises

Results: Impact on Dependent Variables

Sizable change was found for all four dependent variables, in the following order:

• Divergent thinking

• Problem solving

• Performance

• Attitudes and behavior

Results: Impact on Dependent Variables

• “The largest effect sizes were obtained in studies employing divergent thinking (∆=.75; SE=.11) and problem solving (∆=.84, SE=.13) criteria” (p. 369).

• “Studies applying performance criteria yielded smaller, albeit still sizable, effects (∆=.35; SE=.11)” (p. 369).

• “Studies employing attitudes and behavior criteria also produced sizable but somewhat weaker effects (∆=.24; SE=.13)” (p. 369).

External Validity Results:Training Effects Across People and

Settings• “Not only does creativity training appear useful

in various settings and for different age groups, the value of this training holds for populations who differ in their intellectual capabilities” (p. 370).

• “Studies that were based on a predominantly male sample yielded larger effects (∆=1.14; SE=.26 vs. ∆=.42; SE=.26) than studies based on a predominantly female sample” (p. 370).

Internal Validity Results: Does Lack of Rigor Inflate Training Effects

• Study quality: Educational level of author made little difference on the effect of training; peer reviewed studies showed larger effects sizes.

• Length of Follow-up: Studies using longer posttest intervals showed same sizable effects as shorter intervals.

• Author as Trainer: No difference found for studies in which the author was the trainer versus third-party studies.

Course Content: Examination of Training Models

“In the overall analysis, use of a cognitive framework in the development of training content

produced the only sizable positive correlation (r=.31) and regression weight (ß=.24). This general conclusion held across all criteria.”

(p. 376)

Course Content: What Aspects of Cognitive Models are Most

Effective

Program success was related to:

– Problem identification (r=.37)

– Idea generation (r=.21)

– Implementation planning (r=.19)

– Solution monitoring (r=.17)

– Conceptual combination (r=.16)

Course Content: Other Factors that Led to Positive Outcomes

• Aspect of training associated with positive outcomes included:

– Practice time

– Longer training time

– Lecture based instruction

– Social modeling– Cooperative learning– Case-based learning techniques

Conclusions

“Perhaps the most clear-cut conclusion to emerge from this study is that creativity training is effective.”

(p. 381)

Conclusions

“The results obtained in this study indicate that well-designed training can evidence substantial

external validity. Creativity training contributed to divergent thinking, problem

solving, performance, and attitudes and behavior for younger and older students and working adults, and for high achieving and more ‘run of the mill’ students”. (p. 382)

Conclusions

“Taken as a whole, these observations lead to a relatively unambiguous conclusion. Creativity

training works”. (p. 382)

(Scott, Leritz & Mumford, Creativity Research Journal, 2004, Vol 16, pp 361-388)

Recommendations:Best Practices for Creativity Training

• Training should be based on sound, valid, conceptions of the cognitive activities underlying the creative process.

• Training should be lengthy and relatively challenging.

• Articulation of creativity principles should be followed by application using material based on real-world cases.

• Presentation of material should be followed by exercises that allow participants to apply strategies