aosl research report - the impact of arts-based … · the impact of arts-based innovation training...

A u d i e n c e V i e w p o i n t s C o n s u l t i n g

13148RoundingRunCircle,Herndon,VA20171t:831-224-3085www.audienceviewpoints.com

TheImpactofArts-BasedInnovationTrainingontheCreativeThinkingSkills,CollaborativeBehaviorsandInnovationOutcomesofAdolescentsandAdults

AResearchStudyReport

Preparedby:

KateHaleyGoldmanStevenYalowitz,Ph.D.

ErinWilcox,M.A.

August3,2016

ThisresearchwasfundedbygrantDRL-1224111fromtheNationalScienceFoundation.HarveySeifter,PrincipalInvestigator©ArtofScienceLearning,2016.Allrightsreserved.

AudienceViewpointsConsultingArtofScienceLearningResearchReport 2

TableofContents

TableofContents 2

ExecutiveSummary 4

DiscussionandImplicationsforFutureResearch 11

IntroductionandProjectBackground 14

StudyDesign 16

Methods 20RecruitmentSurvey 22PreWorkshopSurvey 23PreWorkshopCreativeThinkingSkillsTest 23WorkshopObservations 24TeamCollaborationRatings 24PostWorkshopCreativeThinkingSkillsTest 25PostWorkshopSurvey 25TransferabilitySurvey 25TeamInnovationOutcomesAssessmentPanelRatings 25

Limitations 27

SamplingandCharacteristicsoftheSample 29HighSchoolStudents 29EarlyCareerSTEMProfessionals 32

Hypothesis1Findings:CreativeThinkingSkills 37DifferencesWithinGroups 38DifferencesBetweenGroups 39CriticalThinking 46CreativeThinkingSkillsSelf-Report 47

CreativeCompetenciesInventory(mini-ECCI) 47CreativeProblemSolvingProfile(CPSP) 48

TransferabilityofSkills 52DefinitionofInnovation 54

HighSchoolStudents 54EarlyCareerSTEMProfessionals 56

SelfPerceptionasInnovator 58HighSchoolStudents 58EarlyCareerSTEMProfessionals 64

UsingChallengeSkillsintheFuture 74

Hypothesis2Findings:CollaborationBehaviors 77ObservedCollaboration 77Self-ReportedCollaboration 78RelationshipBetweenObservedandSelf-ReportedCollaboration 79HighSchoolStudentObservedBehaviorFindings 79

ComparisonBetweenControlandTreatmentbyBehavior 80


ComparisonBetweenControlandTreatmentintheFinalSession 89Self-ReportedTeamCollaborationRatingsofHighSchoolStudents 91HighSchoolStudentBehaviorFindingsSummary 95EarlyCareerSTEMProfessionalBehaviorFindings 96

ComparisonBetweenControlandTreatmentbyBehavior 96ComparisonBetweenControlandTreatmentintheFinalSession 105

Self-ReportedTeamCollaborationRatingsofEarlyCareerSTEMProfessionals 107EarlyCareerSTEMProfessionalsBehaviorFindingsSummary 111

Hypothesis3Findings:TeamInnovationOutcomes 112HighSchoolStudentsFindings 113EarlyCareerSTEMProfessionalsFindings 114TeamInnovationOutcomesFindingsSummary 116

References 117

Appendices 118AppendixA:PreWorkshopSurvey 118AppendixB:PostWorkshopSurvey 123AppendixC:ObservationRating 127AppendixD:CollaborationRating 128AppendixE:TransferabilitySurvey 129AppendixF:PreWorkshopCreativitySkillsTaskWorcester 132AppendixG:PreWorkshopCreativitySkillsTaskSanDiego 134AppendixH:PostWorkshopCreativitySkillsTask 136AppendixI:CreativitySkillsTasksRubric 138AppendixJ:BusinessCase 141AppendixK:OutputScoringSheet 143AppendixL:Inter-raterReliabilityScoringforCreativitySkillsTest 146AppendixM:IdeaClusterCategoriesforCreativitySkillsTest 147


ExecutiveSummaryPROJECTBACKGROUND:TheArtofScienceLearningProject(AoSL)isaNationalScienceFoundation(NSF)-fundedinitiative,foundedanddirectedbyHarveySeifter,thatusestheartstosparkcreativityinscienceeducationandthedevelopmentofaninnovative21stCenturySTEM(Science,Technology,EngineeringandMath)workforce.In2007,Seifter,alongwithartist/scientistToddSilerandchoreographerLizLerman,ledanNSFsymposiumontherelationshipbetweenthearts,STEMlearningandworkforcedevelopment.In2008,SeifterandcolleaguesatNewYork’sLearningWorldsInstituteheldaseriesofroundtableswithscienceeducators,whichrevealedabroadlysharedbeliefintheconnectionbetweentheinvestigativenatureofscienceandthearts,andanappreciationforthepotentialofarts-basedlearningtofosterpassionforexplorationanddiscoveryinyounglearners.Thesemeetingsplayedanimportantroleindesigningaproposal,whichwassubsequentlyfundedbytheNationalScienceFoundation(DRL-0943769).In2011,Phase1oftheprojectconvened425scienceeducators,teachingartists,museumprofessionals,classroomteachers,businessleaders,policymakers,andacademicresearchersinregionalconferencesattheSmithsonianInstitution,IllinoisInstituteofTechnologyandCaliforniaInstituteofTelecommunicationsandInformationTechnology(Calit2).Thegoalsweretoexploretheconnectionbetweenthearts,innovationandeconomiccompetitiveness;createcommunitiesofpracticebysharingeducationalresources,curricula,andbestpracticesthatuseABLtostrengthenSTEMlearning;andexperiencefirst-handarts-basededucationaltechniquesthatdevelopcriticalandcollaborativethinkersfortheSTEMworkforce.AtthewritingofthisreporttheArtofScienceLearningprojectisinPhase2,fundedbytheNSF(DRL-1224111)todevelopanewarts-basedSTEMinnovationcurriculumforadolescentandadultlearners;threeyear-longarts-basedincubatorsforinnovationinSTEMlearningandpracticetotestandrefinethecurriculum;atravelingart/scienceexhibition;andpublicprogramsthatusetheproject’sactivitiesandoutcomestoadvancecivicengagementwithSTEM.Phase2alsoincludedresearchcomparingtheimpactofarts-infusedSTEMinnovationtrainingwithtraditionalproject-basedSTEMinnovationtraining,amulti-yearresearchprojectthatwasindependentlycarriedoutbyAudienceViewpointsConsulting.ThisreportcontainstheresultsofthisPhase2research. RESEARCHDESIGN:AoSL’sresearchcomponentwasdesignedtotesttheideathatintegratingtheartsintoSTEM-relatedinnovationtrainingwouldresultinenhancedcreativethinkingskills,moreextensivecollaboration,morerobustinnovationprocessesandimprovedinnovationoutcomes.Twocities,Worcester,MassachusettsandSanDiego,California,servedasthesitesfortheresearchstudy.HighschoolstudentswerethesamplepopulationinMassachusetts,andearlycareerSTEMprofessionalsinCalifornia.AtbothsitestheAoSLprojectteamhostedfiveweek-longinnovationtrainingsessions,witheachgroupmeetingforahalfdayperweek,totalingroughly20hoursofinvolvementforeachparticipantintheresearchproject.Thetrainingsessionsinvolvedproject-basedlearningfocusedonthefrontendofinnovation,withprojectsaddressinglocalSTEMchallenges(transportationalternativesinWorcester,waterresourcesinSanDiego).Overthecourseofthefiveweeks,teamsofparticipantscreatedsimpleprototypesandbusinesscasesfornewproducts,processesandservicesintendedtoaddressthesechallenges.Thetrainingcurriculum,groundedinbestpracticesderivedfromtheProductDevelopmentManagementAssociationBodyofKnowledge,includedthekeyconceptsofinnovation,STEMcontentspecifictoeachlocalchallenge,andcollaborativeprojectinnovationactivitiesandexercises.


Therewerethreemainhypothesesthatguidedthisresearch:1. Arts-basedinnovationtraining,comparedtotraditionalinnovationtraining,improvesan

individual'screativethinkingskillsincludingcriticalthinking,divergentthinking,problemidentification,convergentthinkingandproblemsolving.

2. Arts-basedinnovationtraining,comparedtotraditionalinnovationtraining,increasesindividualcollaborativebehaviorswithinateamcontext.

3. Arts-basedinnovationtraining,comparedtotraditionalinnovationtraining,enhancesthenovelty,impactandfeasibilityofteaminnovationoutcomes.

Inordertotestthesehypotheses,theresearchstudyusedaquasi-experimentaldesignwithapre-test,post-testintactgroupdesign,includingacontrolgroupforcomparisonpurposes.Intactgroupdesignmeansthatthesameparticipantsfilledoutthepre-testandpost-test,inordertocomparehowresponseschangedfromthebeginningtotheendoftheirparticipation.Individualswhoparticipatedinthestudyweregivenapre-recruitmentsurveyandbasedonthiswereassignedtoeitherthecontrolortreatmentgroup,andcounter-balancedbasedonrelatedvariablessuchasinterestandexperiencewiththeartsandsciences.Individualsweredistributedasevenlyaspossiblebetweenthecontrolandtreatmentgroupsbasedondemographicandpsychographicvariablescollectedduringthepre-test.METHODS:ArtofScienceLearningIncubatorsatEcoTarium(Worcester)andBalboaParkCulturalPartnership(SanDiego)servedashostsitesfortheresearchstudies,withArtofScienceLearningstaffembeddedinbothinstitutionsprovidinglocaladministrativesupport.Averysimilarrecruitmentapproachwasusedinbothcohorts(WorcesterandSanDiego)wherethelocalteammemberssentaninvitationonbehalfoftheresearcherproject.HighschoolstudentswererecruitedfortheWorcestercohort,andearlycareerSTEMprofessionalswererecruitedfortheSanDiegocohort.Bothgroupswererecruitedbasedonanumberofcriteria(age,experiencewithSTEM,experiencewiththearts,etc.).Atotalof16groupsparticipatedinthestudy:8studentgroups(4control,4treatment)inWorcester,and8earlycareerSTEMprofessionaladultgroups(4control,4treatment)inSanDiego.Eachgroupincluded7to10individuals.Bothcontrolandtreatmenttrainingswereheldatseparatetimes,andwhileparticipantsknewthattherewasanothersimilargroupmeetingthesameday,theywereunawarethatthetrainingvaried.Controlandtreatmentgroupsbothusedhands-onproject-basedlearningandanapproachtoinnovationgroundedinBestPracticesfromtheProductDevelopmentManagementAssociation,asarticulatedinthePDMA2014BodyofKnowledge(Kahn,2013).Thetreatmentcurriculumreplaced9hoursofthetraditionalinnovationpedagogyusedinthecontrolcurriculumwith9hoursofarts-basedactivitiesdesignedtoachievethesamelearningobjectives.Inthismanneronlytheapproachwasvaried,toprovideforthecleanestcomparisonofthetwoapproaches.Thereweresevenmethodsused,inordertotriangulatetheresearchfindings:1)arecruitmentsurveyfromthoseinterestedtodetermineeligibilityforparticipation,2)pre-workshopsurvey,3)post-workshopsurvey,4)creativethinkingskillsassessments,5)observationsofgroupsduringweeklymeetings,6)afollow-uptransferabilityofskillslearnedsurveywithasubsetofparticipants,and7)scoringoftheteam’sinnovationproducts,processesandservicesbyanexpertpanelofjudges.Methods1and2wereconductedbeforethetrainingbegan;method3wasconductedintheweeksfollowingthetraining;4and5includeddatacollectedduringthe5-weektrainingperiod;andmethods6and7wereconductedseveralmonthsfollowingthetraining.


MAINFINDINGS(basedonthehypotheses):Hypothesis1:Arts-basedinnovationtraining,comparedtotraditionalinnovationtraining,improvesanindividual'screativethinkingskillsincludingcriticalthinking,divergentthinking,problemidentification,convergentthinkingandproblemsolving.MappingChangesinCreativeThinkingAscreativityisacomplexconstruct,thereweremultiplemeasurementsofdifferentattributesofcreativitywithintheresearch,includingvalidatedscalesfromotherstudiesaswellasinstrumentsanditemscreatedforthisproject.Scalesfromotherstudiesaskedparticipantstoratetheirpersonalcapacitiestowardsavarietyofcreativeprocesses,andtheirpreferencesforcreativityatavarietyofstagesofdevelopment.TheseincludedtheECCIscale(Epstein,Schmidt,&Warfel,2008),twocriticalthinkingscalesandtheCreativeProblemSolvingProfile(CPSP)developedbyBasadur,Graen,andWakabayashi(1990),whichmeasuresindividualstrengthwithinfourdifferentcomponentsofthecreativityprocess:generation,conceptualization,optimization,andimplementation.AcreativityskillsinstrumentcreatedforthisprojectaskedparticipantstoidentifyproblemsrelatedtoagivenInnovationChallenge,selectonetoworkon,generatepossiblesolutionstotheselectedproblem,selectonesolution,andexplaintheirchoices(seeAppendicesFthroughHforthefullexercises).Thesameexercisewasgiventwice:onceintheopening15minutesoftheinitialsession,andagainduringtheclosing15minutesofthefinalsessionfiveweekslater.AdifferentInnovationChallengewasusedfortheseconduseoftheexercisetopreventanypracticeeffects.Aproject-developedparticipanttransferabilityofskillssurveymeasuredtheextenttowhichengaginginthefive-weekresearchchallengehadresidualimpactfourmonthslater.Theresearchteamwasinterestedinwhetherparticipantswereabletoapplywhattheydidduringthetrainingtotheirownsubsequentexperiences,includingschool,extracurricularactivitiesandhomeorpersonallives,andtowhatextentparticipantsexpectedthatimpacttocontinueorgrowinthefuture.HighSchoolStudentsOverall,creativecompetencies,asmeasuredbytheECCIscale,significantlyincreasedinthehighschooltreatmentgroup,anddecreased(thoughnotsignificantly)withinthecontrolgroup.TherewerenosignificantdifferencesbetweentreatmentandcontrolgroupchangescoresforhighschoolstudentsontheCPSPscale.Someofthemoststrikingfindingswerewithinthemetricsfromtheoutcomesofthecreativityskillstest–7ofthe16creativeskillsmeasuresshowedastatisticallysignificantincreasefromthepre-testtothepost-testforthetreatmentgroup.Thesedifferenceswerewithinbothconvergentanddivergentskills,withstrongerevidenceforanincreaseindivergentthinkingskills.Forthemeasuresspecificallyaboutcriticalthinking,statisticallysignificantdifferenceswerefoundbetweenthecontrolandtreatmentgroupsforbothofthemaincriticalthinkingscalesused,withthetreatmentgroupscoringasmuchasthree-quartersofapointhigher.Inthiscase,therewerenodifferencesbetweenpre-andpost-testmeasuresforthetreatmentgroup.Thestudentswhowereinthecontrolgroupscoredsignificantlyloweronthepost-testcomparedtothepre-test.Thecombinationoflackofchangeinthetreatmentgroupandadecreaseinscoreswithinthecontrolgroupresultedinstatisticallysignificantdifferencesbetweenthegroups,butnoevidenceofgainwithineitherofthehighschoolgroups.Significantdifferencesalsoemergedaroundskilltransferability.Highschoolstudentsinthetreatmentgroupweremorelikelytoreportapositiveimpactandanticipatefutureimpactfromtheseexperiences(comparedtothecontrolgroup),withsomeofthedifferencesbeingquitelarge.


EarlyCareerSTEMProfessionalsTherewasastatisticallysignificantdecreaseincreativityinbothtreatmentandcontrolgroupsinearlycareerSTEMprofessionalsontheECCIscale.TherewerenosignificantdifferencesbetweentreatmentandcontrolgroupchangescoresforearlycareerSTEMprofessionalsontheCPSPscale.BoththeearlycareerSTEMprofessionalstreatmentandthecontrolgroupsshowedsomeincreasesincreativeskillsfrompretesttoposttest,includingwithinmeasuresofconvergentanddivergentthinking.Thetwogroupsshowedgainswithindifferentskills.Whencomparedagainstoneanother,onestatisticallysignificantdifferencebetweenthetreatmentandcontrolgroupsemerged,intheabilitytoidentifyandclearlyframeproblemsarisingfromagivenchallenge.NodifferenceswerefoundbetweentheearlycareerSTEMprofessionalsgroupsinthecriticalthinkingscale.Thetreatmentandthecontrolgroupsbothhadslightincreasesfromthepretoposttests,andnodifferenceswerefoundbetweenthetwogroups.Therangeofscoreswaslarge,meaningthatboththearts-basedandthetraditionalinnovationtraininghaddifferentiatingeffectsoncriticalthinkingforadults,inthatsomebenefitedgreatlybutotherslostground.Thereislikelysomeothervariableorsetofvariablesthatdetermineshowthetrainingwillimpactcriticalthinkingskills;however,analysisofthevariablestodatehasnotuncoveredspecificleadsonwhatthoseinfluencesmightbe.Nosignificantdifferencesbetweencontrolandtreatmentwerefoundaroundskilltransferability.Hypothesis2:Arts-basedinnovationtraining,comparedtotraditionalinnovationtraining,increasesindividualcollaborativebehaviorswithinateamcontext.AssessingIndividualCollaborativeBehaviorResearchersobservedthebehaviorsofindividualswithintheirgroupsduringsubstantialpartsofeachofthesessionstheywereworkingtogetheroverthefive-weekperiod,trackingchangesintheprevalenceofspecificbehaviorsofindividualsineachgroupovertime.Inanattempttotriangulatearealisticdepictionofanindividual’scollaborationandparticipationintheInnovationChallenge,attheendofeachworkshopsessionparticipantswerealsoaskedtoratethemselves,andeachindividualontheirteams,onaseriesofbehaviorsthatalignedwiththebehaviorsrecordedbydatacollectorsintheworkshopobservationsheet.HighSchoolStudentsBasedonobservationaldata,comparisonsweremadeoneachbehavioroverthefive-weekperiodforboththecontrolandtreatmentgroups.Inlookingateachgroupindividually,bothtreatmentandcontrolgroupsshowedsimilar,andstatisticallysignificant,increasesintrustinmovingtowardsasolution,beingtransparentincommunication,theabilitytodisagreeproductively,creatingacultureofmutualresponsibilityandproductivelymanagingdisruption.Controlgroupsshowedstatisticallysignificantincreasesoverthefive-weekperiodinsharingleadership,beingtransparentincommunication,definingacommonpurpose,andcreatingacultureofmutualaccountability.Treatmentgroupsshowedstatisticallysignificantincreasesoverthefive-weekperiodinemotionallyintelligentbehavior,empathiclistening,andtheabilitytodisagreeproductively.Whencomparingthetwogroupsdirectly,therewere8ofthe11behaviorswherethefrequencyandpatternsofthebehaviorsdifferedsignificantlybetweencontrolandtreatment.In6ofthese8behaviors,thetreatmentgroupshowedthestrongerperformance:sharesleadership,trustinmovingtowardasolution,transparentincommunication,emotionallyintelligentbehavior,disagreeproductivelyanddefiningacommonpurpose.Intheremaining2ofthese8behaviors(creatingacultureofmutualaccountabilityandproductivelymanagesdisruption)thecontrolgroupshowedmarginally,


butstatisticallysignificant,strongerperformance.Anadditionalanalysisallowedforacomparisonbetweenthecontrolandtreatmentgroupsduringtheirfinal(R5)sessionswhenparticipantswerecompletingtheircourseofstudyandteamswerefinishingtheirworkonthechallengeandmakingalloftheirfinaldecisionswithrespecttobusinesscases(seeAppendixJ)andpresentations.Thus,R5datagaveasenseofthecumulativeimpactofthefulltwenty-hourinterventiononcollaborativebehaviorofcontrolandtreatmentgroups.Whencomparingthetwogroupsdirectly,statisticallysignificantdifferenceswereseenwithrespecttothefrequencyoffivebehaviorsduringthissession:sharesleadership,emotionallyintelligentbehaviors,mutualrespect,abilitytodisagreeproductively,anddefiningacommonpurpose.Alloftheseshowedahigherlevelofoccurrenceforthetreatmentgroup.Self-reportedteamcollaborationratingsweremarkedlydifferent;onlytwoitems(mutualrespectandtrust)showedasignificantchangefrompre-testtopost-test,anditwasthecontrolgroup,ratherthanthetreatmentgroup,thatshowedasignificantincreaseinbothcases.Treatmentgroupsshowedstatisticallysignificantgreaterincreasesinsharingleadership,emotionallyintelligentbehavioranddefiningacommonpurpose,comparedwiththecontrolgroups.EarlyCareerSTEMProfessionalsTheobservationaldatarevealsignificantpre/postincreasesinsevencollaborativebehaviorsamongthetreatmentgroups:sharingleadership,activefollowing,emotionallyintelligentbehavior,empathiclistening,mutualrespect,trustinmovingtowardsasolution,andtransparencyincommunication.Onlyoneofthesebehaviors(emotionallyintelligentbehavior)alsosawanincreaseamongthecontrolgroupoverthefiveweeks.Whencomparingthetwogroupsdirectly,therewere7ofthe11behaviorswherethefrequencyandpatternsofthebehaviorsdifferedsignificantlybetweencontrolandtreatment.In4ofthese7behaviors,thetreatmentgroupshowedtheunambiguouslystrongerperformance.Thesebehaviorswereactivefollower,mutualrespect,trustinmovingtowardasolution,andtransparentincommunication.In2ofthebehaviors,sharingleadershipandempathiclistening,thecontrolgroupshowedamarginally,butstatisticallysignificant,strongerperformance.Inemotionallyintelligentbehavior,thetreatmentgroupshowedamarginally,butstatisticallysignificant,strongerperformance.Incomparingjustthelastsession,therewerestatisticallysignificantbehavioraldifferencesfor2ofthe11observedbehaviorswereobserved:mutualrespectandtrustinmovingtowardsasolution.Forbothofthese,thetreatmentgrouphadasignificantlyhigheroccurrenceofthesebehaviors. Onceagain,therewerestrikingdifferencesbetweenobservationaldataandself-report;therewerenostatisticallysignificantdifferencesfortheself-reportedteamcollaborationmeasuresfortheearlycareerSTEMprofessionals.Hypothesis3:Arts-basedinnovationtraining,comparedtotraditionalinnovationtraining,enhancesthenovelty,impactandfeasibilityofteaminnovationoutcomesJudgingInnovationOutcomesApanelofthreenationalexperts,drawnfromtheselectioncommitteeoftheProductDevelopmentManagementAssociation(PDMA)’sOutstandingCorporateInnovatorAwards,developedanassessment


rubricidentifyingandweightingsevenmeasurestogaugethequalityoftheinnovationoutputs,andsubsequentlyappliedtherubrictothenewproduct,processandserviceconceptsdevelopedbytheteams.PanelistsassembledattheUniversityofIndiana’sKellySchoolofBusinesstoreviewbusinesscasescreatedbytheteams(workingonatemplatedevelopedbyHarveySeifter),PowerPointpresentationscreatedbyeachteamaboutitsinnovation,pre-recordedvideosofeachteam’sconceptpresentation,andpre-recordedvideosofeachteam’sresponsestoastandardizedsetofquestions.Scoringwasdonewithoutpanelistsknowingwhichofthe16groupswerecontrolortreatmentgroups.HighSchoolStudentsTreatmentoutperformedcontrolonallsevenindividualitemsscored.Fourofthesedifferenceswerestatisticallysignificant:insightintochallenge,clarityandrelevanceoftheproblem,problemsolvingstrategy,andthepotentialimpactoftheirproposal.Whilethedifferencesbetweencontrolandtreatmentontheotherthreeitemsscoreddidnotreachstatisticalsignificance,thetreatmentgroupdidhavehigherratingsthanthecontrolgrouponeach.Similarly,inthetotalweightedteaminnovationoutcomescore,whichusedanaverageweightedtotalscoreacrossallitemsforthecontrolgroupcomparedtotheaverageweightedtotalscoreacrossallitemsforthetreatmentgroup,thetreatmentgrouphadhigherratingsbutthedifferencedidnotreachstatisticalsignificance.EarlyCareerSTEMProfessionalsNoneofthedifferencesbetweencontrolandtreatmentonthesevenindividualitemsscoredwerestatisticallysignificantfortheearlycareerSTEMprofessionals.Similarly,therewasnostatisticallysignificantdifferencebetweenthecontrolandtreatmentgroupsinthetotalweightedscoreacrossallitems.Hypothesis3SummaryItwasaveryimportantresultthattheexpertpanelistsratedthehighschoolproducts,processesandservicesofthetreatmentteamssignificantlyhigherthatthoseofthecontrolteamsintermsofinsight,clarity,problemsolvingstrategyandpotentialimpact.Itispossiblethatthislackoffindingsfromtheadultteamsmayresultfromusingacurriculumthatwasdevelopedspecificallyforadolescents.Furtherstudytodeterminewhetheradultfindingswouldchangewiththesubstitutionofacurriculumspecificallydesignedforusewithadultswouldbeveryuseful.RESEARCHCONCLUSIONS:Thestudylookedtoidentifydifferencesincreativityandcollaborationwhenusinganarts-basedapproachtograpplingwithlocalissuesandchallenges.Asnotedinthefindings,therewereanumberofpositivefindingsfromthestudy:

• Highschooltreatmentgroupsshowedalargenumberofstatisticallysignificantpositivedifferencesincreativethinkingskillsfrompretesttoposttest.Forthecontrolgroups,therewerenogainsonanyvariableafterthetraining.[Hypothesis1]

• Highschooltreatmentgroupsshowedstatisticallysignificantgainsinfouroffivedivergentthinkingskillsfrompretesttoposttest.Forthecontrolgroups,therewerenosuchgains.[Hypothesis1a]


• Highschooltreatmentgroupsshowedstatisticallysignificantgainsinthreeofsixconvergentthinkingskillsfrompretesttoposttest.Forthecontrolgroups,therewerenosuchgains.[Hypothesis1b]

• Highschooltreatmentgroupsshowedastatisticallysignificantpositivegainincriticalthinking

skillsfrompretesttoposttest.Thecontrolgroupsshowednosuchgain.[Hypothesis1c]

• Highschooltreatmentgroupsshowedsignificantlystrongerperformancethancontrolgroupsinsharingleadership,trustinmovingtowardasolution,transparencyincommunication,emotionallyintelligentbehavior,productivedisagreement,anddefiningacommonpurpose,basedonobservationaldata.[Hypothesis2]

• Highschoolstudentsperceivedtheirowncollaborativebehaviorshavingpositiveincreasesoverthetrainingforallofthemeasures.[Hypothesis2]

• AdultearlycareerSTEMprofessionalgroupsshowedsignificantlystrongerperformancethancontrolgroupsinemotionallyintelligentbehavior,mutualrespect,activefollowing,trustinmovingtowardasolutionandtransparencyincommunication,basedonobservationaldata.[Hypothesis2]

• EarlySTEMcareerprofessionalsperceivedtheirowncollaborativebehaviorsashavingpositiveincreasesoverthetrainingforalmostallofthemeasures.[Hypothesis2]

• Highschooltreatmentgroupsdevelopedsignificantlystrongerfinalinnovationoutputsthanthecontrolgroups.Externaljudgesfoundlargeandsignificantpositivedifferencesbetweencontrolandtreatmentgroupsininsightsintothechallenge,analyticclarity,problemsolvingstrategyandpotentialimpact.[Hypothesis3]

• Highschoolstudenttreatmentgroupsreportedasignificantlygreaterincidenceofapplyingtheirinnovationlearningexperiencestowork,school,volunteerandextracurricularactivitiesthanthehighschoolstudentcontrolgroups.


DiscussionandImplicationsforFutureResearchTherewasatrendofastrongeroverallimpactofthetrainingonstudentsratherthanadults(especiallyinthecreativeandcollaborativethinkingself-reportmeasures),withthenotableexceptionofobservedcollaborativebehavior.Thereareseveralpossibleexplanationsforthistrend.Itmaybethatanarts-basedapproachtoinnovationtrainingismoreeffectiveforpeoplewhoarestillinschool,comparedtothosewhohavealreadystartedtheirSTEMcareers.Itisalsopossiblethatthedifferentagesandconsequentexperiencesoftheparticipantshadsomethingtodowiththedifferenceinresults.Athirdpossibleexplanationliesinthedecisionmadebytheresearcherstousethecurriculumdesignedforthehighschoolstudentswiththeadultparticipantsaswell,ratherthanmodifyingtheadultcurriculumtoreflectdifferencesbetweenthetwopopulations.Thisdecisionwasimportanttotheresearcherssincehavingverysimilar,ifnotidentical,approacheswasnecessaryinordertomakemoredirectcomparisonsbetweenthetwocohorts,aswellascombinethemasappropriatetotheanalyses.Ifthecurriculahadbeendifferentthenitwouldhavebeenimpossibletoknowwhetheranyvariationintheresultswereduetothedifferentaudiencesorthedifferentpedagogicalapproachesbeingused.Itwillbeimportanttotest,infuturestudies,whetherornotusinganarts-basedinnovationtrainingcurriculumspecificallydesignedforadultlearnerswouldleadtoastrongerimpactonthoselearners.ItisworthnotingthattherewereanumberofpositiveresultsfromthetreatmentapproachfortheearlycareerSTEMprofessionalswithrespecttotheobservedcollaborativebehaviors,whilethestrongerimpactwithstudentshappenedwithindividualcreativethinkingskillsandteaminnovationoutcomes.Itmaybethatthearts-basedapproachyieldedpositivedifferencesinhowtheadultsapproachedgroupinteractions,withouttranslatingintoachangeintheindividualsthemselvesorthesolutionstheycameupwith.Itwouldbeinterestingtoseeifanapproachtotheadultcurriculumthatallowsmorescopeforcollaborativeactivitythanwasincludedinthehighschoolcurriculum,mayproveeffectiveinyieldingpositivechangestoindividualcreativethinkingskillsorteaminnovationoutcomes.Itisalsopossiblethatthelackofdifferenceamongtheadultsinmuchofthevalidatedself-reportedmeasures(CPSP,CreativeProcesses)couldbearesultofthescalesbeingmoreusefulinothersituationsandthusnotasvalidforthistypeofapproach.Meanwhile,theobservationalmeasuresandrelatedweeklyself-reportedteamcollaborationmeasuresyieldedimpactsbetweencontrolandtreatmentconditions.Giventhattheseitemswerespecificallycreatedforthisstudy,itwouldbeexpectedthattheywouldyieldsomesignificantdifferences.Whiletheseinstrumentswerecarefullyplannedandcreated,theydidnotgothroughvalidityandreliabilitytesting,sofurthervalidationoftheinstrumentswouldbeusefultoseewhethertheseobservationalmeasureswouldbehelpfulinotherstudiesrelatedtocreativity,collaboration,andarts-basedapproachesinSTEM.Oneinterestingdifferenceisthatinmanycases,observations,activitytestsandteamoutcomesshoweddifferences,whilepreviouslyvalidatedself-perceivedmeasuresdidnotshowthesamedifferences.Asnotedinthereportandfindingsabove,thereweretimeswherethetreatmentorcontrolgroups(andinsomecases,boththetreatmentandcontrolgroups)showedadecreaseinoutcomemeasures,includingforbothcreativityandcollaboration.Theresearchersdonothaveasingleexplanationforthis;rather,therearesomedifferentpossibilities.Itcouldbethatthereissomethinginherentinthemeasuresthatimpactedthefindingsinthisparticularstudy,(i.e.,thestructuredlearningofinnovationprocessesthatformedthecoreofboththecontrolandtreatmentcurricula,ortheparticulararts-basedapproachusedinthetreatmentcurriculum).Therecouldalsobesomeunmeasureddifferencesunevenlydistributedbetweenthecontrolandtreatmentgroupsthatcontributedtothesedifferences,althoughatthispointthereisnotanyspecificevidencetowardswhattheymightbe.Variabilitywas


strongintermsofimpact,withhighstandarddeviationsatsomepoints.Thissuggeststherewasanunderlyingvariablenotmeasuredinpre-art,pre-scienceorcreativityscoresthatmighthelpdeterminewhetherthetrainingwouldhaveimpact.Futureresearchcoulddelveintothisarea.Additionally,theremaybesomethingaboutthetasksthemselvesthataccountforthedifferences.Withoutspecificevidenceoranindicationforwhichfactorsmayhaveinfluencedthisdecreaseinthetreatmentgroup,itisdifficulttospeculateaboutwhythismaybethecase.Furtherresearchcouldduplicatethestudytoseewhetherornotsimilarresultsforthetreatmentgrouparefound,allowingforabetterunderstandingofthisphenomenon.InlookingatpotentialrealworldapplicationsofthisresearchtothelearningandpracticeofSTEMinnovation,itseemsthatdifferentapproachestotheintegrationofarts-basedlearningmayworkbestfordifferentaudiences.Forhighschoolstudents,thearts-basedapproachwasclearlymoreeffectiveinfosteringcreativethinkingandproblemsolving,suggestingthatitmaybemoreeffectivethanatraditionalapproachforstudents.ThishaspotentiallyfarreachingimplicationsforhowadolescentsmightbestexperienceSTEMandinnovationlearninginawiderangeofformalandinformalsettings.Furtherresearchcouldinvestigatewhichspecificfactorswouldbemorelikelytoleadtopositiveoutcomes,andpotentialstrategiesforoptimizingtheimpactsofarts-basedlearning.Foradults,outcomessuggestthatanarts-basedapproachmaybemosteffectivewhenusedtoincreasecollaborationorstrengthenteamsandcollaborativecultureinaprofessionalsetting.Itwouldalsobevaluabletoexplorethepossibilitythatbuildingincreasedopportunitiesforcollaborativeexplorationintoanadultarts-basedcurriculummayyieldgreaterimpactonindividualcreativethinkingskillsandteaminnovationoutcomes.Itispossiblethattheconditionsthatneedtobepresenttopositivelyimpactadultcreativethinkingskillsandinnovationoutcomesmaybedifferentfromthoseneededforstudents.Theremayalsohavebeensomedifferentkindsofcreativethinkingskillsthatcouldbeenhancedwithanarts-basedapproach.Theseareimportantquestionsforfurtherinvestigation.InboththeadolescentandearlycareerSTEMprofessionalcases,largerstudieswithmorethanonetreatmentgroupforeachaudiencewouldbeinvaluable;thisseemsanobviousnextstepinbuildingonthisresearch.Giventhedearthofresearchinthisarea,andtheuniquenessofthearts-basedapproachtothelearningandpracticeofSTEMinnovation,itisnotsurprisingthatevenwiththecompellingandpromisingfindingsinthisstudythereareadecentnumberofunansweredquestions.Whileaquasi-experimentalapproachwasabletobeusedforthecurrentstudy,anexperimentalstudywouldgoevenfartherinansweringsomeofthemorefundamentalquestionsstillremainingaroundcreativityandcollaboration,manyofwhichhavebeentouchedonabove. Basedontheoutcomesofthisstudy,westronglyrecommendfutureresearchintothefollowingquestionsasparticularlyusefultotheadvancementofknowledgeinthisfield:• Doesarts-basedlearningfosterengagementinadolescentandadultlearners?Forbothcohorts,

theattritionratewasconsiderablyhigherforthecontrolgroupthanforthetreatmentgroup.Whiledatawerenotcollectedduringthisstudyaboutwhyparticipantswerenolongercontinuingwiththeprogram,onepossibleexplanationisthatthearts-basedapproachengenderedagreaterlevelofengagementthanthetraditionalapproach.Thiswouldbeinterestingtopursueinfurtherstudies.


• Howcanarts-basedlearningbemosteffectivelyintegratedintoinnovationtraining?Weobservedmanypowerfulpositiveimpactsofthistypeofinnovationtraining,whencomparedwithamoretraditionalpedagogicalapproach.o Arethereparticulararts-basedactivitiesandapplicationsthatdrovetheimpactsoncreative

thinkingskills,collaborativebehaviorsandinnovationoutcomesthatweobservedandmeasured?Mighttheuseofothersuchactivitiesfurtherstrengthenorbroadenthoseimpacts?

o Whichtypesandwhatamountsofarts-basedlearning,andinwhatsequence,wouldoptimizeimpact?

• Canarts-basedlearningservetoneutralizethepotentialnegativeimpactsoftraditionalhigh

schoolinnovationtrainingoncreativity?Thehighschoolstudenttreatmentgrouphad7creativethinkingskillvariablesthatshowedasignificantincreasefrompretopost,andnoneoftheseweresignificantlydifferentforthecontrolgroup.Likewise,thehighschoolstudentcontrolgrouphad5creativethinkingskillvariablesthatshowedasignificantdecreasefrompretopost,yettherewasnosignificantdecreaseforthetreatmentgroup.Theseresultsstronglysuggestthatarts-basedlearningmayhavethecapacitytoovercomeandneutralizewhatappearstobethenegativeimpactoftraditionalinnovationtrainingonadolescentcreativethinking.Boththequestionofwhethertraditionalinnovationtrainingdepressescreativityinadultlearners,andthepotentialofarts-basedlearningtoovercomeanynegativeimpact,bearsfurtherinvestigation.

• Doestheuseofarts-basedlearningenhancethecognitivegainsofadolescentoradultlearners?Thisstudydidnotexaminetheextenttowhichthearts-basedapproachtoinnovationtrainingincreasesknowledgeandunderstandingofSTEMcontent,bothwithrespecttolearningingeneralandforspecifictopics.Severalofourfindings,however,suggestthatthismaybeapossibility.Thisisanimportanttopictopursueinfuturestudies.

• Doesarts-basedlearningfostergreaterpost-learningimpact,agencyandoptimisminadolescentlearners?Inasurveyfourmonthsaftertheintervention,thehighschoolstudenttreatmentgroupreportedasignificantlygreaterincidenceofapplyingtheirinnovationlearningexperiencestowork,school,volunteerandextracurricularactivitiesthanthehighschoolstudentcontrolgroup.Thetreatmentgroupalsoshowedamuchgreateroptimismthattheywouldbelikelytoapplytheirinnovationlearningexperiencestofuturework,school,volunteerandextracurricularactivitiesthanthehighschoolcontrolgroup.Thesefindingssuggestthevalueofinvestigatingtheimpactofarts-basedlearningonadolescentagencyandoptimism.

• Isself-reportanaccuratewaytoassesscollaborationingroups?Althoughneithercontrolnortreatmentgroupsratedthemselvesashavingimprovedonanyoftheself-reportedteamratingmeasures,observersrecordedthetreatmentgroupsashavingengagedinmanyofthesebehaviorssignificantlymorebythelastsession.Thesefindingssuggesttheimportanceofincludingdirectobservationinassessmentsofthenatureandextentofgroupcollaboration,ratherthansimplyrelyingonself-reportfromparticipants.


IntroductionandProjectBackgroundTheArtofScienceLearningPhase2isafour-yearNationalScienceFoundation-fundedinitiative(Grant#1224111)thatusestheartstosparkcreativityinscienceeducationandthedevelopmentofaninnovative21stCenturySTEMworkforce.TheinitiativeisbuiltonArtofScienceLearning’sPhase1,aswellasdecadesofworkbyProjectDirectorHarveySeifterandcolleagues,exploringtheimpactofartisticskills,processesandexperiencesoncreativity,innovationandlearning(aneducationalmethodologyknownas“arts-basedlearning”).Thegoaloftheproject'sPhase2developmentactivitieswastoexperimentwitharts-basedlearninginavarietyof"innovationincubator"modelsincitiesaroundthecountry.Modeledonbusiness"incubators"or"accelerators"thataredesignedtofosterandaccelerateinnovationandcreativity,theseSTEMincubatorsgeneratecollaborationsofdifferentprofessionalsandthepublicaroundSTEMeducationandotherSTEM-relatedtopicsoflocalinterestthatcanbeexploredwiththehelpofcreativelearningmethodologies.Theseincludeinnovativemethodstogeneratecreativeideas,ideasfortransformingoneSTEMideatoothers,drawingonvisualandgraphicalideas,improvisation,narrativewriting,andtheprocessofusinginnovativevisualdisplaysofinformationforcreatingvisualroadmaps.TheArtofScienceLearning,incollaborationwithBalboaParkCulturalPartnership,Phase2’snationaladministrativesponsor,andseveralinformalscienceeducationandotherculturalandbusinessorganizationsinSanDiego,Chicago(notablytheMuseumofScienceandIndustry),andWorcester,MA(principallyEcoTariumandClarkUniversity)implementedaresearchanddevelopmentprojecttoinvestigatearangeofpossibleapproachesforstimulatingthedevelopmentof21stCenturycreativityskillsandinnovativeprocessesattheinterfacebetweeninformalSTEMlearningandmethodsforcreativethinking.Theprojectgoalsincludeddevelopingnewwaystousearts-basedlearningtoenhanceSTEMinnovationskillsamongarangeoflearners,developingimpactfulnewmodelsusingarts-basedlearningtostrengtheninformalSTEMlearningandadvanceunderstandingofthepotentialimpactsofarts-basedlearningonthepublic'sunderstandingofandengagementwithSTEM.Aparticularfocuswasonstrengtheningthe21stCenturyworkforceskillsofhighschoolstudentsandearlycareerSTEMprofessionals,andcreatingdeliverablesandresourcesforthefieldtofostertheintegrationoftheartsandSTEMfields.Activitieshaveincludedthefollowing:theformationandcollaborativeprocessesoftwoincubatorsites,anindependentresearchstudy(thisreport),thedevelopmentofacomprehensivearts-basedinnovationprocesscurriculumforSTEMlearners,professionaldevelopmentbasedonthecurriculum,publicengagementeventsandexhibits,aprojectwebsiteandtoolsforsocialnetworking,andprojectevaluation.Anationaladvisorycouncilincludedprofessionalsineducation,science,creativity,andbusiness.Formoreinformationabouttheprojectseethefollowing:

• ArtofSciencelearningwebsite:http://www.artofsciencelearning.org/• NSF’swebsiteabouttheprojectat

http://www.nsf.gov/awardsearch/showAward?AWD_ID=1224111&HistoricalAwards=false• Theprojectsiteontheinformalscience.orgsiteathttp://informalscience.org/projects/ic-000-

000-000-099/Integrating_Informal_STEM_and_Arts-Based_Learning_to_Foster_Innovation

Asmentionedabove,theprojectincludedasummativeevaluationoftheincubatorsbySlover-Linnett(aprofessionalevaluationfirm)thatfocusedonthegoalsandobjectivesoftheproject.ThecurrentreportsummarizestheseparateresearcheffortconductedbyAudienceViewpointsConsulting(AVC)that


lookedintobetterunderstandingthetheorybehindtheintersectionofartsandsciencesincollaborativeprocessesfocusedonsolvingreal-worldproblems.AVCwascontractedtoconductindependentresearchstudiesatsitesinWorcester,Massachusetts(attheEcoTarium)andSanDiego,California(attheBalboaParkCulturalPartnership).ThemaingoaloftheresearchwastobetterunderstandtheaffordancesofusingacurriculumthatintegratedtheartsintoSTEMinnovation,includingtheuseofarts-basedprocessesandpracticesintryingtosolvespecificproblemsor‘challenges’identifiedinthecommunity.TheresearchfocusedonbothhighschoolstudentsandearlycareerSTEMprofessionalsinordertobetterunderstandtheconditionsandfactorsrelatedtospecificoutcomes.HighschoolstudentswereincludedintheresearchsincetheyareapopulationthatislearningaboutandbecomingmoreinterestedinSTEM,andtheseareformativeyearsforthinkingaboutandengaginginthesetopics.TheearlycareerSTEMprofessionalswereincludedinordertogatherinformationaboutthepotentialforanarts-basedapproachtobeusedwiththosejustbeginningorcontinuingtoengageinSTEM-basedfields.Includingbothwouldallowforabetterunderstandingofthepotentialofengaginganarts-basedapproachtocollaborativeproblemsolvingindifferentaudiences.


StudyDesignAoSL’sresearchcomponentwasdesignedtotesttheideathatintegratingtheartsintoSTEM-relatedinnovationtrainingwouldresultinenhancedcreativethinkingskills,moreextensivecollaboration,morerobustinnovationprocessesandimprovedinnovationoutcomes.Twocities,Worcester,MassachusettsandSanDiego,California,servedasthesitesfortheresearchstudy.HighschoolstudentswerethesamplepopulationinMassachusetts,andearlycareerSTEMprofessionalsinCalifornia.AtbothsitestheAoSLprojectteamhostedfiveweek-longinnovationtrainingsessions,witheachgroupmeetingforahalfdayperweek,totalingroughly20hoursofinvolvementintheproject.Thetrainingsessionsinvolvedproject-basedlearningfocusedonthefrontendofinnovation,withprojectsaddressinglocalSTEMchallenges(transportationalternativesinWorcester,waterresourcesinSanDiego).Overthecourseofthefiveweeks,teamsofparticipantscreatedsimpleprototypesandbusinesscasesfornewproducts,processesandservicestoaddressthesechallenges.Thetrainingcurriculum,groundedinbestpracticesderivedfromtheProductDevelopmentManagementAssociationBodyofKnowledge,includedkeyconceptsofinnovation,STEMcontentspecifictoeachlocalchallengeandcollaborativeprojectinnovationactivitiesandexercises.Thedecisionwasmadetousethecurriculumdesignedforusewithadolescentsforbothgroups,inordertoallowforacomparisonbetweenthehighschoolstudentsandearlycareerSTEMprofessionals,aswellascombinethemasappropriatetotheanalyses.Withoutusingthesamecurriculum,itwouldhavebeenimpossibletodeterminewhetheranypositiveresultsweregeneralizableacrossdifferentaudiences.Belowisthebasicscheduleforeachworkshopday,comparingactivitiesanddiscussionbetweenthecontrolgroupandthetreatmentgroup(seeTable1).TheworkshopstaffutilizedthesamecurriculumandmaintainedthesameschedulefortheWorcestercohortandtheSanDiegocohortinordertocomparedifferencesbetweenthegroupsofadultsandstudents.Thestudyusedaquasi-experimentaldesignwithpre-test,post-testintactgroupdesign,inordertoallowcontrolforothervariables(seeFigure1belowforthedesignandmethodsforthecontrolandtreatmentgroups).Thecontrolgroupswereincludedtoprovideacomparisontotreatmentgroupparticipants,whoexperiencedthearts-basedactivities.BothgroupsexperiencedaSTEMinnovationworkshop;however,thetreatmentgroupalsoexperiencedthearts-basedactivities,whichallowedforisolatingandunderstandingtheaddedvalueofthearts-basedactivitiestotheworkshopexperience.Thestudyalsoincorporatedmixedmethods;whilethefocuswasonquantitativedata,criticalqualitativedatawerecapturedtobetterunderstandtheindividualexperienceofparticipatingintheInnovationChallenges.Therewerethreemainhypotheses,whichincludedsub-hypotheses:

1. Artsbasedlearninginfluencesindividuals’creativethinkingskills.a. Arts-basedinnovationtrainingincreasesanindividual’sabilitytoemployconvergent

thinkingcomparedtotraditionalinnovationtraining.b. Arts-basedinnovationtrainingincreasesanindividual’sabilitytoemploydivergent

thinkingcomparedtotraditionalinnovationtraining.c. Individualswhoparticipateinarts-basedinnovationtraininghaveagreaterincreasein

criticalthinkingskillsthanthosewhoparticipateintraditionalinnovationtraining.


2. Artsbasedlearninginfluencesinnovation-relatedprocessbehaviorswithintheteamcontext.a. Artsbasedinnovationtrainingincreasesanindividual’scollaborativebehaviorsand

collaborativeskills,ascomparedwithtraditionalinnovationtraining.

3. Artsbasedlearninginfluencesthequalityofthefinalproductdevelopedbytheteam.a. Artsbasedinnovationtraining,comparedtotraditionalinnovationtraining,willresultin

ateamproducingamoreinnovativeandfeasibleproduct.

BothhighschoolstudentsandtheearlycareerSTEMprofessionalsmaintainedthesameschedule,whilethereweredifferencesintheactivityflowsofthecurriculumbetweenthecontrolandtreatmentgrouptoaccountforreplacementof9hoursofthetraditionalinnovationpedagogyusedinthecontrolcurriculumwith9hoursofarts-basedactivitiesdesignedtoachievethesamelearningobjectivesusedinthetreatment(seeTable1).Table1:ResearchCurriculumSchedule

Morning-TreatmentGroup Afternoon-ControlGroup

R1 1.Welcomeandoverview9:00am-9:15am

1.Welcomeandoverview1:30pm-1:45pm

2.CreativitySkillsTest9:15am-9:30am 2.CreativitySkillsTest1:45pm-2:00pm3.IntroductiontoChallenge9:30am-10:30am

3.IntroductiontoChallenge2:00pm-3:00pm

4.IntroductiontoInnovation10:30am-10:50am

4.IntroductiontoInnovation3:00pm-4:15pm

5.Metaphormingactivity11:05am-12:50pm

5.InnovationTools&TheirUse4:15pm-5:15pm

6.Wrapup12:50pm-1:00pm 6.Wrapup5:15pm-5:30pmR2 1.Welcomeandreview9:00am-9:15am 1.Welcomeandreview1:30pm-1:45pm

2.IntroductiontoOpportunity9:15am-9:30am

2.IntroductiontoOpportunity1:45pm-2:00pm

3.InformationGathering/Enrichment#19:30am-10:15am

3.ExpertPanel2:00pm-2:30pm

4.ExpertPanel10:15am-10:45am

4.Q&AwithExpertPanel2:30pm-3:00pm

5.Q&AwithExpertPanel11:00am-11:30am

5.BrainstormingThreeOpportunitySources3:00pm-3:30pm

6.Problem/OpportunityWorkshop11:30am-11:45am

6.IdentifyingOpportunitiesandBuildingSharedGoals3:45pm-4:40pm

7.Problem/OpportunitySelectionWorkshop11:45am-12:50pm

7.DescribeWhatProblems/SolutionsYourTeamSelected4:40pm-5:10pm

8.Wrapup12:50pm-1:00pm 8.Wrapup5:10pm-5:30pm


R31.Welcomeandreview9:00am-9:05am 1.Welcomeandreview1:30pm-1:45pm

2.OpportunitySelection9:05am-9:50am 2.Vision2025Exercise1:45pm-2:45pm

3.IdeationCycle9:50am-10:55am 3.SolutionSelection2:45pm-3:30pm4.IdeationCycle,parttwo

10:55am-11:40am4.ResearchingProblem/Solution

Statements3:45pm-4:45pm5.IdeaEnrichmentandRefinement

11:40am-12:55pm5.ScoringEnhancedSolutions

4:45pm-5:10pm

6.Wrapup12:55pm-1:00pm 6.Wrapup5:10pm-5:30pmR4

1.WelcomeandReview9:00am-9:05am 1.Welcomeandreview1:30pm-1:45pm2.ConceptSelection9:05am-9:40am

2.PrioritizeProduct/ServiceConceptsfromsolutions1:45pm-3:15pm

3.RehearsingIdeaSession9:40am-11:30am

3.ConceptFeasibilityandMarketEnhancement3:30pm-4:30pm

4.CreatingBusinessCases11:30am-12:55pm

4.BeginBusinessCaseDevelopment4:30pm-5:15pm

5.Wrapup12:55pm-1:00pm

5.Wrapup5:15pm-5:30pm

R51.WelcomeandReview9:00am-9:05am 1.Welcomeandreview1:30pm-1:35pm2.CompletingBusinessCases

9:05am-9:45am2.Iteration&BusinessCaseReview

Discussion1:35pm-1:50pm

3.PresentationDevelopment9:45am-10:40am

3.WriteClearandComprehensiveConcept

Statements1:50pm-2:10pm4.CastingandRehearsingthe

Presentations10:40am-11:45am4.Sub-groupsWorkonBusinessCasesandGavePresentations2:10pm-4:15pm

5.GatePresentations11:45am-12:45pm 5.GatePresentations4:15pm-5:15pm6.CreativitySkillsTest12:45pm-1:00pm

6.CreativitySkillsTest5:15pm-5:30pm

Note:Therewasa15minutebreakroughlyinthemiddleofeachfour-hoursession,forbothcontrolandtreatmentgroups.


Figure1:StudyDesignandMethodsVisualRepresentation

HighSchoolStudents

Control Treatment

RecruitmentPhase• PreSurveyInterventionWeek1• CreativitySkillsPreTest

• ResearcherObservation

• CollaborationRatings

Weeks2-4• ResearcherObservation


Week5• ResearcherObservation


• FinalPresentation

• CreativitySkillsPostTest

PostIntervention• PostSurvey• TransferabilityStudy

EarlySTEMCareer

Professionals

Control Treatment


MethodsThestudyusedanexperimentaldesignwithtreatmentandcontrolgroups;controlgroupswereincludedtoprovideacomparisontotreatmentgroupparticipantswhoexperiencedthearts-basedactivities.BothgroupsexperiencedaSTEMinnovationworkshop;however,thetreatmentgroupalsoexperiencedthecreativethinkingactivities,allowingforanunderstandingoftheaddedvalueofthearts-basedactivitiestotheworkshopexperience.Thestudyalsoincorporatedmixedmethods;whilethefocuswasonquantitativedata,criticalqualitativedatawerealsocapturedtobetterunderstandtheindividualexperienceofparticipatingintheInnovationChallenge(seeTable2).Datawerecollectedattwolocations:theEcoTariuminWorcester,Massachusetts,andBalboaParkinSanDiego,California.Furthermore,thereweretwomainaudiencesthestudyfocusedon:1)highschoolstudents,and2)adultswhowereatanearlystageintheirSTEMcareers(earlycareerSTEMprofessionals).AllparticipantsintheWorcestercohortwerehighschoolstudents,andalloftheparticipantsintheSanDiegocohortwereearlycareerSTEMadults.Atotalof65individualsparticipatedinthecompletecourseof5Worcesterworkshops,andatotalof69individualsparticipatedinthecompletecourseof5SanDiegoworkshops.WhileparticipantswereawarethattheywereparticipatinginanNSF-fundedstudy,theywerenotawarethatitwasanexperimentalstudyorthatitwaslookingatarts-basedapproachestoinnovation.InlookingatattendanceratesfortheWorcestercohort,therewasaninterestingeffect-thetreatmentgroupsaw32ofthe34initialparticipantscompletethefivesessions,anattritionrateof6%;whilethecontrolgroupsaw33oftheinitial38participantscompletethefivesessions,anattritionrateof13%.TheSanDiegogroupssawasimilarpatternofattrition:37ofthe40initialparticipantsinthetreatmentgroupcompletedallfivesessions,anattritionrateof8%;whilethecontrolgroupsaw32oftheinitial36participantscompletethefivesessions,anattritionrateof11%.Forbothcohorts,theattritionratewasconsiderablyhigherforthecontrolgroupthanforthetreatmentgroup,thoughthisoccurredtoagreaterdegreeinWorcester.Ineachcohort,thegreaterlevelofattritionamongthecontrolgroupswasmanifestbyweektwo.Whiledatawerenotcollectedduringthisstudyaboutwhyparticipantswerenolongercontinuingwiththeprogram,onepossibleexplanationisthatthecontrolandtreatmentapproacheswereengenderingadifferentlevelofengagement.Multiplemethodswereusedaspartofthislargeresearchplan,including:

1. RecruitmentSurvey2. Pre-andpost-interventionwrittenquestionnaires;3. Creativethinkingskillsassessment;4. Observations;5. Follow-uptransferabilitysurveyswithasubsetofparticipants;and6. Scoringofinnovationteamproducts.

SeetheAppendicesofthereportfortherelatedinstruments.


Table2:SummaryofMethods

Method WhenAdministered SourceofApproach,InstrumentorItems

RecruitmentSurvey(DemographicVariables)

BeforeParticipation(Pre)

Researcher-generatedbasedonprojectneeds.

PreWorkshopSurvey(CurrentlyLevelofArtsandSTEMparticipation)

BeforeParticipation(Pre)

ModifiedfromSlover-Linnett’sevaluationoftheAoSLincubators.

CreativeThinkingSkillsTestSurvey

BeforeandAfterParticipation(Pre/Post)

AnactivitydevelopedbyHarveySeifter,modifiedfromRuncoandBasadur’s(1993)articleAssessingIdeationalandEvaluativeSkillsandCreativeStylesandAttitudes,andscoredonarubricdevelopedbyHarveySeifterandAVC.

Slover-LinnettCreativeProcessPerceptionsSurvey


ScaledevelopedbySlover-Linnett(AoSLevaluator)tolookatperceptionsofthecreativeprocess.Itworkedwellontheevaluationpretest,providingasignificantrangeofopinions.

Basadur’sCreativeProblemSolvingProfile(CPSP)Survey


AmodifiedversionofBasadur,Graen,andWakabayashi’s(1990)CreativeProblemSolvingProfile(CPSP)Inventory,whichmeasurespreferencefordifferentstagesinthecreativeprocess,differentiatinghowindividualsprefertogainanduseknowledge.Theyproposefourstagestoanindividual’screativeprocess.

CriticalThinkingSurvey


DrawnfromBasadurandFinkbeiner’s(1985)scaleonpreferenceforideationandtendencyforprematurecriticalevaluationofideas,thisscalewasusedinRuncoandBasadur’s(1993)studyonideationandevaluativethinkingchangeduringtraining.


CreativeCompetenciesSurvey


AdaptedfromanabridgedversionoftheEpsteinCreativityCompetenciesInventoryforIndividuals(ECCI-i),developedbyEpstein,Schmidt,andWarfel(2008).Therearefourmeasurable,trainablecompetencies:capturing(preservingnewideasastheyoccur),challenging(takingondifficulttasks),broadening(seekingknowledgeandskillsoutsideone'scurrentareasofexpertise),andsurrounding(seekingoutnewstimuliorcombinationsofstimuli).

Self-ratedCommonPurposeandCollaborationSurvey

DuringParticipation(EachWeek)

Eleventraitsofcollaborativebehavior,identifiedbytheprojectdirector,weredrawnfromthecollectiveworkofHackman(2002),KatzenbachandSmith(1993),Sawyer(2007),SengeandScharmer(2001),ThotaandMunir(2012),MossKanter,HeifetzandLinsky(2002).Individualswereaskedtoratetheirteameachweek;thesametraitswereincludedfortheObservedBehaviorsmethod.

ObservedBehaviors DuringParticipation(EachWeek)

Eleventraitsofcollaborativebehavior,identifiedbytheprojectdirector,drawingfromthecollectiveworkofHackman(2002),KatzenbachandSmith(1993),Sawyer(2007),SengeandScharmer(2001),ThotaandMunir(2012),MossKanter,HeifetzandLinsky(2002).Theresearcherscountedinstancesofthevariousbehaviorsduringcollaborativetaskseachweek;thesametraitswereincludedfortheSelf-ratedCommonPurposeandCollaborationSurvey.

TeamInnovationOutcomesAssessmentPanel

AfterParticipation(Post)

ExpertpanelistsfromtheProductDevelopmentManagementAssociation’sselectioncommitteeofOutstandingCorporateInnovationAwardsviewedteams’finalproducts(businesscase,PowerPointpresentations,videoofconceptpresentation,andvideoofresponsestoquestions.Thepaneldevelopedarubricwithsevenelements,ratedeachteamindependentlyontheseelements,thencalculatedafinalscoreforeachteam.

RecruitmentSurveyAstheteamwasattemptingtorecruitindividualswhometadefinedsetofcriteria,potentialrecruitswereaskedtofilloutaquestionnairetofindoutmoreabouttheirbackgroundandrelevantexperiences.ThequestionnaireaskedaboutinvolvementwithSTEMactivities,currentemploymentstatus,currentstudentstatus,genderandage.ThequestionnairealsoaskedhowparticipantsfoundoutaboutthisInnovationChallengeresearchproject.Thequestionnairecapturedthecontactinformationfor


participantsinordertoemailinformationandaninvitationtothepreworkshopsurvey.ThisquestionnairewaspostedonapublicwebpageusingtheSurveyMonkeyweb-basedplatformtoprovideaccesstothepotentialrecruits.

PreWorkshopSurveyOnceacceptedintotheInnovationChallenge(i.e.,theworkshop),participantswereaskedtocompleteasurveypriortotheChallengestart,inordertolearnmoreabouttheparticipantandsothatresponsestothispre-surveycouldbecomparedtotheresponsestothepost-surveyaftercompletingtheInnovationChallenge.Thequestionswerebrokenupintosectionsthatdirectlycorrelatedtothemainhypothesestestedwithinthestudy.AlinktothesurveywassentviaemailtoparticipantsandthesurveywashostedonaprivatewebpageontheSurveyMonkeyweb-basedplatform.SeeAppendixA.Question1collectedinformationaboutanindividual’spriorconnectionswithartandscience.Thisinformationwasmostlyusedtosortindividualsintoteamssothattheteamshadsimilaraveragescoresforartandscienceexperience,toreducebiasbetweenteams.Question2focusedoncreativethinking.ThequestionwasadaptedfromanabridgedversionoftheEpsteinCreativityCompetenciesInventoryforIndividuals(ECCI-i),developedbyEpstein,Schmidt,andWarfel(2008).Slover-Linettusedthisscalewithsomesuccesswithinthepre-surveyforincubatorparticipants.Question3wasadaptedfromtheNorton(1975)ToleranceforAmbiguityScale.TheAVCresearchteamtookitemsfromthe“problem-solving”and“philosophy”subscales;thereweresixothersubscales.Toleranceforambiguitymaybeasignificantindependentvariableforparticipants,giventhenatureofthecontrolandtreatmentcurriculum.Question4wasdevelopedbySlover-LinnetttolookatperceptionsoftheCreativeProcess.ItseemedtofunctionwellontheirpretestwithasignificantrangeofopinionsQuestion5wasdesignedtolearnmoreaboutcriticalthinkingskills.DrawnfromBasadurandFinkbeiner’s1985scaleonpreferenceforideationandtendencyforprematurecriticalevaluationofideas,thisscalewasusedinRuncoandBasadur’s(1993)studyonideationandevaluativethinkingchangeduringtraining.Question6wasdrawnfromamodifiedversionofBasadur,Graen,andWakabayashi’s(1990)CreativeProblemSolvingProfile(CPSP)Inventory,whichmeasurespreferencefordifferentstagesinthecreativeprocess,differentiatinghowindividualsprefertogainanduseknowledge.Participantswerethenaskediftheyidentifiedasaninnovator,andpromptedtogivetheirownpersonaldefinitionofinnovation.Thesurveyconcludedwithafewadditionaldemographicquestions,likeethnicoriginandhighestlevelofeducationcompleted.

PreWorkshopCreativeThinkingSkillsTestParticipantsintheInnovationChallengewereaskedtocompletean“InnovationWarm-up”exercise,atthestartofthefirstdayoftheworkshop.ThetaskincludeddefiningproblemsandchoosingaproblemrelatedtotheInnovationChallengeasdescribedbythepromotionalmaterial,thendefiningsolutions


andchoosingasolutiontothepreviouslychosenproblem.Participantshad15minutestocompletetheskillstest.Thistestwastakenwithpaperandpencil.SeeAppendixFandAppendixG.ThesetaskswerederivedfromconceptssimilartothechallengesnotedinRuncoandBasadur’s(1993)articleonAssessingIdeationalandEvaluativeSkillsandCreativeStylesandAttitudes.Thesolutionswerescoredinacomplexrubricincludingthequantityofproblemsandsolutionsidentified,thebreadthofproblemsandsolutions,andthemotivationsforchoosingaparticularproblemtofocuson.TherubriccanbeseeninAppendixI.

WorkshopObservationsDatacollectorsobservedparticipantseachweekoftheInnovationChallenge.Thepurposeofthisapproachwastodocumentexamplesofindividualcollaborativebehaviorsinteamsettings,acrossthefiveresearchsessionsorweeks.AVCscheduledatleasttwodatacollectorsforeachworkshopweek;AVCteammemberscollectedsomedata,butalsohiredlocalindividualswhohadexperienceobservingbehaviorinanexperimentsetting.Duetotheconstraintsofthenumberofgroupsineachworkshopandthenumberofdatacollectorsitwasimpossibletoobserveeachgroupduringallactivitiesandgroupdiscussionperiods.Itwasthereforenecessaryfordatacollectorstorotatebetweengroupsduringtheworkshopsessions.SeeAppendixC.Datacollectorsobservedtheindividualsononeteamforaperiodoftime,usuallybetween10and30minutes,duringagroupactivityordiscussionperiod.Sincethebehaviorsfocusedoncollaboration,participantswerenotobservedduringpresentationsbyfacultyorwhenindividualswereworkingontheirown.TheAVCandArtofScienceLearningteamsdevelopedalistofbehaviorstobeobservedwhichcorrespondedtothemainhypothesestestedwithinthestudy.Datacollectorsrecordedbehaviorsdisplayedbyindividualsontheteamduringtheobservationperiod,andadditionallyrecordedanyunusualornoteworthyactivityoutsideofthelistofbehaviorstoobserve.Observationswererecordedonpaperdatasheetswithpenorpencil.Attheendofthepredeterminedobservationperiodthedatacollectorsmovedontothenextteamandbeganwithanewobservationperiod.Eachsessionwasdiscussedatlengthbeforethesessionstodeterminewhentheparticipantswouldbeobserved,forhowlong,andwhendatacollectorswouldswitchbetweenteams.Withtwodatacollectorsandfourteams,switchingoncewithinanobservationperiodwouldmeanthatallfourteamswereobservedduringanactivityorobservationperiod.

TeamCollaborationRatingsAttheendofeachworkshopsessionparticipantswereaskedtoratetheirteamonaseriesofbehaviorsthatdirectlyalignedwiththecollaborativebehaviorsrecordedbydatacollectorsintheworkshopobservationsheet.Thiswasanattempttotriangulatearealisticdepictionofanindividual’scollaborationandparticipationintheInnovationChallenge.Individualswereaskedtoratetheirteam’sperformancebasedonaseriesofbehaviorsonascalefrom1to7points,andthenwereaskedtogivetheirteamanoverallratingbetween1and7points.ThesurveysweretakenoniPadsthroughtheSurveyMonkeyweb-basedplatform.Therewere4to8iPadsavailableduringtheworkshops,soindividualstookturnsfillingoutthesurveybeforesubmittingtheirsurveyandpassingtheiPadontothenextteammember.


PostWorkshopCreativeThinkingSkillsTestJustbeforethecloseofthelastdayoftheInnovationChallenge,participantswereaskedtoagaincompletetheexercisetheyhaddoneontheirfirstdayoftheworkshopthistimedescribedasan“InnovationWrapUp.”Theformatoftheexercisewasexactlythesameasthefirstsession:theparticipantsinbothcohortswereaskedtoaddressaccesstoaffordableandhealthyfood,ratherthanthetopicsthesecohortshadbeenworkingonoverthepreviousweeks.SeeAppendixH.

PostWorkshopSurveyApproximatelyoneweekaftercompletingtheInnovationChallenge,participantsweresentalinktoaPostWorkshopSurvey,whichwashostedonaprivatewebpagethroughtheSurveyMonkeyweb-basedplatform.ThissurveyhadtheexactsamequestionsasthePreWorkshopSurvey,Questions2to6,inordertocomparetheresponsesofindividualsbeforeandaftertheworkshopexperience,andtocomparetheresponsesbetweenthecontrolandtreatmentgroups.SeeAppendixB.Again,likethePreWorkshopSurvey,individualswereaskediftheyself-identifiedasaninnovator;however,thistimeindividualswerealsoaskediftheirdefinitionofinnovationhadchangedsinceparticipatingintheworkshop.Finally,participantswerealsoaskedtorateeachindividualmemberoftheteam,onascalefrom1to7,basedontheircontributionsoverthefivesessions.

TransferabilitySurveyAboutfourmonthsaftercompletingtheInnovationChallengeAVCsentafollowupwebsurveytoallparticipantstoexplorethelong-termeffectsoftheirexperience.ThesurveywashostedonaprivatewebpagethroughtheSurveyMonkeyweb-basedplatform.Participantswereaskedwhethertheyhadbeenthinkingabouttheworkshopsinceitscompletion,iftheworkshophadchangedhowtheythinkaboutthecreativethinkingprocess,andwhetherindividualsthoughttheywouldbeabletousesomeofthetechniqueslearnedduringtheworkshopsintheirprofessionalandpersonallife.CompletionoftheTransferabilitysurveywasnotmandatory;therefore,thetotalnumberofindividualswhoparticipatedinthismethodislowerthanthetotalnumberofindividualswhoparticipatedintheworkshops.SeeAppendixE.

TeamInnovationOutcomesAssessmentPanelRatingsThereviewofthefinalteaminnovationoutcomeswasconductedbyapanelcomposedofthreemembersoftheselectioncommitteeoftheProductDevelopmentManagementAssociation’s(PDMA)OutstandingCorporateInnovationAwards,includingthatcommittee’sfoundingandcurrentChairs.Thepanel,consultingwiththeprojectandresearchteams,developedanassessmentrubricidentifyingandweightingsevenmeasurestogaugethequalityoftheinnovationoutcomes,includingteaminnovationoutputsandworkproducts.Theysubsequentlyappliedtherubrictoeachteam’sproduct,processandservicesolutionconceptsdevelopedbytheteams.ThepanelistsmetinpersonovertwodaysattheKelleySchoolofBusinessatIndianaUniversitytoreviewanddiscussthePowerPointpresentationscreatedbyeachteamaboutitsinnovation,pre-recordedvideosofeachteam’s5-minuteconceptpresentation,andpre-recordedvideosofeachteam’sresponsestoastandardizedsetofquestions.Aheadofthepanelmeeting(butafterthepanel’scompletionofworkontheassessmentrubric),thepanelistsalsoreceivedthebusinesscases(seeAppendixJ)createdbyeachoftheteamsfortheiradvancereview;panelistswereaskednottodiscussthesematerialswiththeircolleaguesaheadofthemeeting.Takentogether,theseelements(businesscase,PowerPointpresentations,videoof


conceptpresentation,andvideoofresponsestoquestions)formedthebasisforthepanel’soverallassessmentofeachteam’sinnovationoutcomes. Thereviewersdidnotknowwhichofthe16groupswereinthecontrolortreatmentcondition,andtheorderofteamswithineachcondition(i.e.,controlandtreatment)wasrandomized.SeeAppendixK.


Limitations1.Numberofresearchsites:Initiallyitwasintendedtohavethreeresearchsites(Chicago,Worcester,andSanDiego),andhaveeachsiteruna5-weekInnovationChallenge.Unlikeinthefinaldesign,eachsitewouldhaveacohortofhighschoolstudentsandofearlySTEMcareerprofessionals,thoughthesetwogroupswouldnotbemixed.Forexample,thehighschoolcontrolgroupwouldmeetSaturdaymorning,theearlySTEMcareergroupSaturdayafternoon,thehighschooltreatmentgroupSundaymorning,andtheearlySTEMcareertreatmentgroupSundayafternoon.RecruitmentfortheChicagoincubatorwaslow,especiallyamongSTEMprofessionals,andwewereconcernedthatthegroupsrecruitedwouldnotresultinsamplesizeslargeenoughtoanalyze.TheChicagoInnovationChallengewasindefinitelypostponed,andwemadecorrespondingchangestothesamplemake-upinWorcesterandSanDiego.Thechangesallowedtheteamtoreducethesamplesize,asweonlyhadonepopulationtypepersite.Whilethisconflatesthesitewiththepopulationtype,itallowedrecruitmenttobesuccessfulwithinthosesites.2.Datacollectionlocations/needforlocaldatacollectors:DuetothetimeandfinancialfactorsofhavingmultipleAVCemployeestravelingfor10weekends,AVCrecruitedlocaldatacollectorsinWorcester,Massachusetts,andSanDiego,CaliforniatocollectobservationdataduringtheInnovationChallenge.AVCfoundlocaldatacollectorswhocamewithgoodrecommendationsfrompeersinthefieldandwhowereexperiencedinobservingbehaviorsininformallearningenvironments.AnAVCteammembertrainedtwolocaldatacollectorsduringthefirsttwoweeksinSanDiegoandthefirstthreeweeksinWorcesteronusingtheobservationandcollaborationratinginstrumentsandansweringbasicquestionsabouttheflowoftheworkday.AnAVCteammemberwasavailablebyphoneandtextmessageduringthelastweeksoftheinterventionincaseofemergencyorneedforclarification.Therefore,includingthreeAVCteammemberswhocollectedobservationdata,therewereatotalofsevendatacollectorsforoneinstrument.Itispossiblethathavingmoredatacollectorscouldintroducemoreerrorintothedataduetodifferentdefinitionsandinterpretationsofbehaviors.Tomitigatediscrepanciesalldatacollectorswereencouragedtocomparenotesthroughoutthedayanddiscussquestionsaboutwhethercertaingesturesorlanguageusedbyparticipantsdidindeedqualifyasspecificbehaviorsaccordingtotheinstrument.3.Placingpeoplepurposefullyintogroups:Puttingpeopleintothecontrolandtreatmentgroups,whilebeingbeneficialforspreadingthetwoconstructs(artandscience)out,mayhavehadsomeresidualeffectonthefindings.Amainpurposeofthestudywastocomparethetreatmentandcontrolgroupswithineachofthetwocohorts(highschoolstudents,earlycareerSTEMprofessionals),toseeifbeingexposedtothearts-basedapproachmadeadifferenceintheoutcomes.Giventhatcomparison,itwasimportanttocounterbalancethecontrolandtreatmentgroupssothatthecharacteristicsofeachgroupdidnotskewinacertaindirectionthatcouldimpacttheresults.Asresearchers,itwasalsounclearhowdiversethegroupswouldbe;iftheywereverydiversethenitwouldbeimportanttospreadthatdiversityevenlyacrossthecontrolandtreatmentgroups,andthenalsowithinthefourgroupsineachofthesecohorts.Studentswerecounterbalancedbyaccountingforan“artscore,”a“sciencescore,”anda“creativityscore;”theaveragescoresforeachwerematchedascloselyaspossible,sothesethreemainscoresmainlydrovewhereindividualswereplaced(whichcohort,thenwhichgroupwithineachcohort).Whilethisservedthepurposeofcounterbalancingtheeffectofthesevariablesontheresults,theremayhavebeensomeresidualeffecttomakingsurethegroupswerespreadevenlyalongthese


variables.Saidanotherway,theremayhavebeenanintangiblebenefittohavinggroupsbemoresimilartoeachotherthataffectedtheresults.4.Attritionduringthedatacollection:TherewasastrictnoabsencepolicyforparticipationintheInnovationChallengeworkshop.Anyparticipantwhomissedoneweekoftheworkshopwasnotallowedtoreturnforanyfollowingworkshopday.Theprogramfacilitatorsfeltthatparticipantswhomissedaworkshopdaywouldfalltoofarbehindincontentunderstandingandgroupcollaborationandcouldnotbereintegratedtothegroup.ItispossiblethatparticipantswhofelttheInnovationChallengewasnotmeetingtheirexpectationsdecidedtodropoutoftheprogram.ThoseindividualswhodidnotperceivetheInnovationChallengetobebeneficialforthemwouldhavelikelyshowednochangeintheirbehaviorsandskillsthroughoutallmethodsofdatacollection,PreSurveytoPostSurvey,CollaborationRatings,Observations,etc.Thiscouldpossiblyskewthedataasweonlyhaveresponsesandratesofchangeforparticipantswhocompletedtheentireintervention.Thisisaconsequenceofthenatureoftheinterventiondesignedasaworkshop,notasamandatoryexperimentaldesignwhereparticipantsmustcompleteallworkshopdays.5.Usingadolescentcurriculumforbothgroups.InordertobeconsistentandallowforacomparisonbetweenthehighschoolstudentsandadultswhowereatanearlystageintheirSTEMcareer,thesamecurricularapproachwasusedforbothcohorts.Thecurriculumusedinthisstudywasspecificallydesignedforusewithadolescents,andwhiletherewasthepossibilitythatemployingastudent-focusedcurriculumcouldbeexperienceddifferentlybytheearlycareerSTEMprofessionalsitwasdeemednecessarybytheresearchersinordertocomparetheutilityoftheapproachwithbothcohorts.ItispossiblethatsomeofthelackoffindingsamongtheearlySTEMprofessionalsmayresultfromthecurriculumhavingbeendevelopedmorefortheadolescents.Furtherstudytodeterminewhetheradultfindingswouldchangewiththesubstitutionofacurriculumspecificallydesignedforusewithadultswouldbevaluable.


SamplingandCharacteristicsoftheSampleIntheoriginalplansfortheInnovationChallengeworkshopseachlocationwouldhaveacohortofhighschoolstudentparticipantsandacohortofearlycareerSTEMprofessionalparticipants.BecauseofdifficultiesinrecruitmentofearlycareerSTEMprofessionalparticipants,inWorcestertheprojectdirectorsandAVCdecidedtocanceltheadultworkshopinWorcesterandrunonlythehighschoolstudentprogram,thenrunonlytheearlycareerSTEMprofessionalsprograminSanDiego.

HighSchoolStudentsTheArtofScienceLearningteamatthehostsitetheEcoTariuminWorcester,ledbyAoSL’sWorcesterIncubatorDirectorJoyceKressler,conductedrecruitmentofparticipants,reachingouttocontactsinthecommunitytoraiseawarenessabouttheworkshop.Projectcommunitypartnersincludedregionalpublicandprivateschools,nonprofitorganizations,collegesanduniversities,informallearningcenters,parentorganizations,businessesandeducators.Altogether,AoSLcontactedmorethan15highschools(forexample,WorcesterTechnicalHighSchoolandDohertyHighSchool),morethan10communitycollegesanduniversities(forexample,WorcesterStateUniversityandUniversityofMassachusettsMedicalSchool),6non-profitorganizations(forexample,theBoysandGirlsClubofAmerica),7STEM-relatedcompanies(forexample,Intel),andmorethan10relevantprofessionalorganizations(forexample,MassachusettsBiomedicalInitiatives).Informationandinvitationstoregisterfortheworkshopweresentviaemailtoacontactateachschoolorinstitution,andtheAoSLteamattheEcoTariumalsomadephonecallstocertainpartnerswhereapplicable.Additionally,theEcoTariumpromotedtheWorcesterInnovationChallengethroughapostingontheirFacebookpagethatwaspaidforbyfundsfromtheArtofScienceLearninggrant.ProspectiveparticipantsintheInnovationChallengeworkshopwereaskedtofilloutashortquestionnaireabouttheirexperiencewithSTEM(Science,Technology,Engineering,Math)subjectsandtheircurrentinvolvementwithcreativeorartisticpursuits.RecruitersfocusedtheireffortstofindparticipantswhohadsomelevelofinterestorparticipationintheSTEMfields,butitwasnotnecessarythataparticipantbeactivelyinvolvedinanycreativeorartisticendeavor.Thequestionnairealsocollectedinformationaboutthegradelevelandhighschooltheparticipantattended,informationthatwouldbeusedtogroupparticipantsintoteamssothatpeoplewhoalreadykneweachotherwouldnotbeonthesameteams.Recruitmentfocusedonhighschoolstudentsin10thand11thgrades;however,studentsin9thand12thgradeswereacceptedtotheWorcesterInnovationChallenge.ParticipantswerealsorequiredtosubmittwoelectronicconsentformsinordertoparticipateintheWorcesterInnovationChallenge:aformconsentingforthemselves,andasecondformwiththeassentoftheirparentorguardian.ParticipantswhodidnotsubmitthesetwoseparateformswerenotallowedtocompletethePreWorkshopSurveyperIRBregulationsandthereforecouldnotcontinueintherecruitmentprocess.BytheendoftherecruitmentperiodthereweremorestudentswhohadregisteredfortheWorcesterInnovationChallengeworkshopthancouldparticipate;atotalof190applicationswerereceived,someincomplete,andonly88participantscouldbeacceptedtotheprogram.Participantsweretakenonafirstcomefirstservedbasis,thendividedbytheirschoolsandgradeintoeighttemporaryteamsofelevenstudentswiththeunderstandingthatafewstudentsmightnotshowthefirstday,orwoulddropoutafterthefirstworkshopsession.Thetemporaryteamswereconfiguredinawaythatnoteamwouldhavemorethanoneparticipantfromthesameschoolandgrade;however,whileteamsdidhavemore


thanoneparticipantfromthesameschool,theywerefromdifferentgrades.Considerationwasalsogiventoparticipantswhohadrequestedparticipatingineitherthemorningorafternoonworkshopsduetoschedulingortransportationconflicts.Otherthanthesestipulations,participantswererandomlyassignedtothecontrolgroup(afternoonsession)ortreatmentgroup(morningsession).ParticipantswhoattendedaschoolwithalreadyconsiderablerepresentationontheteamswerecontactedandinformedthattheywouldbeunabletoparticipateintheWorcesterInnovationChallenge.ParticipantswerethensortedbasedontheirresponsestothePreWorkshopsurveyabouttheirexperiencewithartandscienceeducationandtheir“creativityscore”(acompositescoreofspecificquestionsfromthePreWorkshopSurvey).Teamswereconfiguredtohaveverysimilaroverallaveragesoftheirscoresforart,scienceandcreativity,withoutanyoverlappingofhighschoolandgradelevel.Itwascriticaltomaintaintheseaveragescores,whichdidnothaveastatisticallysignificantdifference,inorderfortheteamstobecomparablebetweencontrolandtreatmentgroups.Onthefirstdayoftheworkshopparticipantswereresortedintogroupsifindividualsonthesameteamhadpersonalrelationshipswitheachother,whichcouldnothavebeenforeseenbytheplanningteam.Teamswerealsoresortedintheeventthatmultipleparticipantstemporarilyplacedonacertainteamdecidednottoparticipate,toevenouttheaveragesizeofeachteam.However,onceindividualswereofficiallyplacedonateamduringthefirsthouroftheWorcesterInnovationChallenge,individualswerenotallowedtoswitchtoadifferentteamatanypointduringthelengthoftheChallenge.TheInnovationChallengeworkshopsforthehighschoolstudentcohortwereheldonconsecutiveSundaysfromOctober26th,2014throughNovember23rd,2014.Eachhalfdaysessionranforaperiodoffourhours;themorningsessionwasheldfrom9:00amto1:00pmandtheafternoonsessionwasheldfrom1:30pmto5:30pm.ParticipantsintheWorcesterworkshopwereofferedastipendof$250,whichtheywouldreceiveafterattendingallfiveworkshopdatesandcompletingthepostworkshopsurvey.TheworkshopswereheldattheEcoTarium,asciencemuseumandnaturecenterinWorcester,Massachusetts.ByweekfiveoftheWorcesterInnovationChallengeatotalof65individualsparticipatedinthecohort.ThismeansthattherewassomeattritionthroughoutthefiveweeksoftheChallenge;however,themajorityofindividualswhodroppedoutoftheprogramdidsobetweenweekoneandweektwo.Asnotedabove,theattritionratewasconsiderablyhigherforthecontrolgroupthanforthetreatmentgroup.Thetreatmentgroupsaw32ofthe34initialparticipantscompletethefivesessions,withanattritionrateof6%;whilethecontrolgroupsaw33oftheinitial38participantscompletethefivesessions,withanattritionrateof13%.Thereweremorefemaleparticipants(68%)thanmaleparticipants(32%),withallparticipantsintheWorcesterInnovationChallengebetweentheagesof15and18yearsold(seeTable3).Almostalloftheparticipantswerefull-timestudents(95%)andcamefrommorethan20differenthighschoolsinthearea.Themajorityofparticipantswerenotemployed(85%),thoughsomewereemployedpart-time(12%).ThelargestsinglegroupofparticipantsreportedlyidentifiedwiththeCaucasianorWhiteethniccategory(41%),withthenextmostcommonlyidentifiedcategoriesbeingAsian(15%)andAfricanAmericanorBlack(14%).AsmallerpercentageofparticipantsidentifiedasHispanicorLatino(8%)andAmericanIndianorAlaskaNative(1%).Someparticipantspreferrednottoanswer(11%)andsomechose“Other”andwroteinhowtheyidentifiedthemselves.Forexample,someparticipantswrote:

NativeAmerica,Portuguese,AfricanAmerican,FrenchCanadian,etc.Arabic


Indian EuropeanandAfrican IamhalfCaucasianandhalfPakistani

Table3:DemographicsforHighSchoolStudentParticipantsCharacteristic Control(n=33) Treatment(n=32) Total(n=65)Gender

Male 27% 38% 32%Female 73% 62% 68%

AgeCategory 15to18 100% 100% 100%

EthnicCategory CaucasianorWhite 42% 42% 41%AfricanAmericanorBlack 21% 6% 14%Asian 6% 25% 15%HispanicorLatino 6% 9% 8%AmericanIndianorAlaskaNative 0% 3% 1%Other 9% 12% 11%Prefernottoanswer 15% 3% 9%

CurrentLevelofEducation Stillenrolledinhighschool 85% 97% 91%GED 3% 0% 1%Didnotanswer 12% 3% 8%

StudentStatus Part-timestudent 0% 6% 3%Full-timestudent 97% 94% 95%Didnotanswer 3% 0% 1%

GradeLevel 9th 6% 0% 3%10th 48% 44% 46%11th 39% 31% 35%12th 6% 25% 15%

EmploymentStatus Notemployed 88% 81% 85%Employedpart-time 9% 16% 12%Didnotanswer 3% 3% 3%

(Table3continuedbelow)


Table3:DemographicsforHighSchoolStudentParticipantscontinuedCharacteristic Control(n=33) Treatment(n=32) Total(n=65)SchoolAttendance

WorcesterTechnicalHighSchool 18% 19% 18%UniversityParkCampusSchool 18% 3% 10%DohertyMemorialHighSchool 3% 16% 9%NorthHighSchool 6% 9% 8%BancroftSchool 3% 12% 8%SouthHighCommunitySchool 9% 3% 6%BurncoatHighSchool 9% 0% 5%WachusettRegionalHighSchool 3% 6% 5%ShrewsburyHighSchool 6% 0% 3%MassachusettsAcademyofMathandScience 3% 3% 3%LincolnSudburyHighSchool 6% 0% 3%PlainfieldHighSchool 0% 6% 3%HopkintonHighSchool 3% 3% 3%AdvancedMathandScienceAcademyCharter 0% 6% 3%WorcesterAcademy 0% 6% 3%SaintJohn’sHighSchool 0% 3% 1%HopedaleJunior-SeniorHighSchool 3% 0% 1%QuinebaugMiddleCollege 3% 0% 1%NorthamptonHighSchool 0% 3% 1%HolyNameJuniorSeniorCentralCatholic 3% 0% 1%AuburnHighSchool 3% 0% 1%

EarlyCareerSTEMProfessionalsDr.NanRenner,DirectoroftheArtofScienceLearningSanDiegoIncubatorandtheAoSLteamconductedrecruitingfortheSanDiegoInnovationChallenge.Theteamsreachedouttocontactsinthecommunitytoraiseawarenessabouttheworkshop,suchasnonprofitorganizations,educatorsandothereducationalinstitutions.InformationabouttheSanDiegoInnovationChallengewasdistributedviatheBalboaParkLearningInstitutee-newsletter,whichhas1,800subscribers.TheteaminSanDiegoemailedapproximately150educatorsfromSanDiegoStateUniversity,UniversityofCaliforniaSanDiego,andthecommunitycollegesinthearea.Additionally,theyreachedouttoabout50educatorsfromBalboaParkandcontactedmorethan100additionalindividualsfromtheirpersonalnetworks.SpecificcriteriaforselectionincludedcurrentemploymentinaSTEM-relatedprofessionalfieldorrole(includingSTEMeducation),lessthansevenyearsofprofessionalexperienceworkinginaSTEMfield,completionofatleasttwoyearsofcollegeeducationandprofessionalcompetenceinEnglish.TwoAVCresearchersindependentlyreviewedtheinformationsubmittedinthequestionnairetodetermineeligibilityforeachpotentialparticipant.Eachpotentialparticipantwaslabeledas“yes,”“no,”or“maybe”basedonthecriteria.Inordertoprovideasdiverseandbalancedagroupaspossible,secondarycriteriawerealsotakenintoaccount.Theseincludedage,whetherornotpotentialparticipantswereteachersorcurrentstudents,andartsbackground.Alsotakenintoconsiderationwasavailability;participantsunabletocommittoattendingallofthesessionswerenotinvitedto


participate.Aftereachpotentialparticipantwascategorizedindependently,thetworesearchersthencomparedandfinalizedthelistofthoseinvitedtoparticipate.ParticipantsfortheInnovationChallengeworkshopwereaskedtofilloutashortquestionnaireabouttheirexperiencewithSTEMsubjects(Science,Technology,Engineering,Math)andtheircurrentinvolvementwithcreativeorartisticpursuits.RecruitersfocusedtheireffortstofindparticipantswhowereintheearlystagesoftheircareerintheSTEMfield,howeversomeparticipantswhowereacceptedtotheprogramdidnotworkdirectlyintheSTEMfield,andsomeparticipantsdidinfactworkwithintheSTEMfieldbutwerenotinthebeginningoftheircareer.Itwasnotnecessarythataparticipantparticipateinanycreativeorartisticendeavor.Thequestionnairealsoaskedwhetherparticipantswerestudents,employedpart-timeorfull-time,orunemployed/retired.ParticipantswererequiredtosubmitanelectronicconsentforminordertoparticipateintheSanDiegoInnovationChallenge.ParticipantswhodidnotsubmitthisformwerenotallowedtocompletethePreWorkshopSurveyperIRBregulationsandthereforecouldnotcontinueintherecruitmentprocess.TherewerefewerobstaclesforincludingandsortingteamsintheSanDiegoInnovationChallengebecausetheparticipantswerenotminors,andthereforeonlyhadtocompleteoneconsentform.BytheendoftherecruitmentperiodthereweremoreindividualswhohadregisteredfortheSanDiegoInnovationChallengeworkshopthancouldparticipate;atotalof175applicationswerereceived,someincomplete,andonly88participantscouldbeacceptedtotheprogram.Participantsweretakenonafirstcomefirstservedbasis,thendividedintoeighttemporaryteamsofelevenindividualswiththeunderstandingthatafewindividualsmightnotshowthefirstday.Considerationwasalsogiventoparticipantswhohadrequestedparticipatingineitherthemorningorafternoonworkshopsduetoschedulingortransportationconflicts.Otherthanthesestipulations,participantswererandomlyassignedtothecontrolgroup(afternoonsession)andtreatmentgroup(morningsession).ParticipantswerethensortedbasedontheirresponsestothePreWorkshopsurveyabouttheirexperiencewithartandscienceeducationandtheir“creativityscore”(acompositescoreofspecificquestionsfromthePreWorkshopSurvey).Teamswereconfiguredtohaveanoverallaverageofartscore,sciencescoreandcreativityscore.Itwascriticaltomaintaintheseaveragescores,whichdidnothaveastatisticallysignificantdifference,inorderfortheteamstobecomparablebetweencontrolandtreatmentgroups.Onthefirstdayoftheworkshopparticipantswereresortedintogroupsifindividualsonthesameteamhadpersonalrelationshipswitheachother,whichcouldnothavebeenforeseenbytheplanningteam.Teamswerealsoresortedintheeventthatmultipleparticipantstemporarilyplacedonacertainteamdecidednottoparticipate,toevenouttheaveragesizeofeachteam.However,onceindividualswereofficiallyplacedonateamduringweekoneoftheWorcesterInnovationChallenge,individualswerenotallowedtoswitchtoadifferentteamatanypointduringthelengthoftheChallenge.TheInnovationChallengeworkshopsfortheearlycareerSTEMprofessionalcohortwereheldonconsecutiveSaturdaysfromJanuary10th,2015throughFebruary7th,2015.TheworkshopswereheldatBalboaParkinSanDiego,California.Eachhalfdaysessionranforaperiodoffourhours;themorningsessionwasheldfrom9:00amto1:00pmandtheafternoonsessionwasheldfrom1:30pmto5:30pm.ParticipantsintheWorcesterworkshopwereofferedastipendof$500,whichtheywouldreceiveafterattendingallfiveworkshopdatesandcompletingthepostworkshopsurvey.ThestipendfortheSan


DiegoparticipantswassignificantlylargerthanthestipendfortheWorcesterparticipants-thiswasapurposefuldecisionmadetogiveadultsagreaterincentivetocontinuetotheendoftheprogram.ByweekfiveoftheSanDiegoInnovationChallengeatotalof69individualsparticipatedinthecohort.ThismeansthattherewassomeattritionthroughoutthefiveweeksoftheChallenge;however,themajorityofindividualsdroppedoutoftheprogramweekone.Asnotedabove,theattritionratewashigherforthecontrolgroupthanforthetreatmentgroup.Researchersnotedthat37ofthe40initialparticipantsinthetreatmentgroupcompletedallfivesessions,withanattritionrateof8%;whilethecontrolgroupsaw32oftheinitial36participantscompletethefivesessions,withanattritionrateof11%.DuetothenatureoftherecruitingrequirementsfortheSanDiegoInnovationChallenge,therewereawidervarietyofagesinthisprogram,comparedtoWorcester(seeTable4).Themajorityofparticipantswereyoungadults19to25yearsold(42%)or26-30yearsold(17%)inthebeginningstagesoftheircareer.Thereweresmallerpercentagesofadults,31to40yearsold(26%),41to50yearsold(7%),51to60yearsold(6%)and61to70yearsold(1%).TherewerestillmorewomenparticipantsintheSanDiegoChallenge,likeWorcester;however,theratioofwomentomenwasmoreeven(59%women,41%men).ThemajorityofparticipantsreportedlyidentifiedwiththeCaucasianorWhiteethniccategory(52%),withthenextmostcommonlyidentifiedcategoriesbeingHispanicorLatino(19%)andAsian(16%).AsmallerpercentageofparticipantsidentifiedasAfricanAmericanorBlack(4%)andNativeHawaiianorotherPacificIslander(1%).Someparticipantspreferrednottoanswer(7%)andsomechose“Other”andwroteinhowtheyidentifiedthemselves.Forexample,someparticipantswroteinamoredetaileddescriptionoftheirethniccategorysuchas:

Chicano Brazilian AlsoHispanic,wouldnotletmecheckmorethanone EuropeanThemajorityofparticipantswerenotcurrentlyattendingaschool(62%)butthereweresomeparticipantswhowerefull-timestudents(29%)orpart-timestudents(9%).OfthoseindividualswhoweretakingclassesatthetimeoftheSanDiegoInnovationChallenge16%wereenrolledincollegeorcommunitycollegecourses,and22%wereenrolledingraduateschool.ThoseparticipantswhowerestudentsatthetimeoftheworkshopattendedavarietyofschoolsintheSanDiegoarea,includingUniversityofCalifornia,SanDiego(17%)andSanDiegoStateUniversity(7%).Manyparticipantshadalreadycompletedacommunitycollegeortechnicalcertificate(9%),acollegedegree(43%)oragraduateorpostgraduatedegree(36%).Asmallpercentageofparticipants(10%)reportedtheirhighestlevelofeducationasahighschooldiplomaorGED.AlmostallparticipantswereinvolvedincareersthatwereSTEMrelated(77%)orinsomewayrelatedtotheSTEMfield(20%).Themajorityofparticipantswereemployedeitherfull-time(56%)orpart-time(26%)withjustafewparticipantsreportingtobeunemployedorretired(4%)


Table4:DemographicsforEarlyCareerSTEMProfessionalParticipantsCharacteristic Control(n=32) Treatment(n=37) Total(n=69)Gender

Male 37% 43% 41%Female 62% 57% 59%

AgeCategory 19to25 44% 40% 42%26to30 16% 19% 17%31to40 22% 30% 26%41to50 6% 8% 7%51to60 12% 0% 6%61to70 0% 3% 1%

EthnicCategory CaucasianorWhite 47% 57% 52%AfricanAmericanorBlack 3% 5% 4%Asian 22% 11% 16%HispanicorLatino 12% 24% 19%NativeHawaiianorotherPacificIslander 3% 0% 1%Prefernottoanswer 12% 3% 7%

StudentStatus Notcurrentlyastudent 62% 62% 62%Part-timestudent 12% 5% 9%Full-timestudent 25% 32% 29%

CurrentEducationLevel Collegeorcommunitycollege 22% 11% 16%Graduateschool 16% 27% 22%Notapplicable 62% 62% 62%

HighestLevelofEducation Highschool/GED 12% 8% 10%CommunityCollege/technicaltraining 6% 11% 9%CollegeDegree(BA/BS) 47% 40% 43%GraduateorPostgraduateDegree 31% 40% 36%Didnotanswer 3% 0% 1%

CurrentSTEMProfessional? Yes 75% 78% 77%No 0% 3% 1%Tosomeextent 25% 16% 20%I’mnotsure 0% 3% 1%

EmploymentStatus Notemployedand/orretired 9% 16% 13%Employedpart-time 28% 24% 26%Employedfull-time 59% 54% 56%Didnotanswer 3% 5% 4%

(Table4continuedbelow)


Table4:DemographicsforEarlyCareerSTEMProfessionalParticipantscontinuedCharacteristic Control(n=32) Treatment(n=37) Total(n=69)SchoolAttendance

UniversityofCalifornia,SanDiego 12% 22% 17%SanDiegoStateUniversity 9% 5% 7%PointLomaNazareneUniversity 3% 3% 3%SouthwesternCollege 0% 3% 1%MiamiUniversity,Ohio 0% 3% 1%NationalUniversity 3% 0% 1%CaliforniaStateUniversity,LongBeach 0% 3% 1%SanDiegoCityCollege 3% 0% 1%CUNYGraduateCenter,NewYork,NY 3% 0% 1%SanDiegoStateUniversity&UniversityofCalifornia,Davisjointprogram 3% 0% 1%


Hypothesis1Findings:CreativeThinkingSkillsFundamentally,theresearchcomponentoftheArtofScienceLearningprojectwasdesignedtoinvestigatewhetherparticipatinginarts-basedinnovationtraininggivesoneanadvantageoverparticipatinginmoretraditionalinnovationtrainingofthetypethatonemightencounterinanacademicorworkplacesetting.Withinthisframework,ourresearchwasbrokendownintothreeprimaryhypotheses.Thefirstoneconsideredtherelativeadvantageofarts-basedinnovationtrainingonarangeofcreativethinkingskills.Theteambrokethishypothesisdownintoaseriesofsub-hypothesesinordertobetterinvestigateeachelement.Hypothesis1:Arts-basedinnovationtraining,comparedtotraditionalinnovationtraining,improvesanindividual'screativethinkingskillsincludingcriticalthinking,divergentthinking,problemidentification,convergentthinkingandproblemsolving.

• Hypothesis1a:Artsbasedinnovationtrainingincreasesanindividual'sabilitytoemploydivergentthinkingovertraditionalinnovationtraining.

• Hypothesis1b:Artsbasedinnovationtrainingincreasesanindividual'sabilitytoemployconvergentthinkingovertraditionalinnovationtraining

• Hypothesis1c:Artsbasedlearninginfluencesindividuals’criticalthinkingskills.Asmentionedwithinthemethodssection,eachofthesesub-hypotheseswasmeasuredthroughavarietyofmethods.Detailedinformationontheunderlyingconstructsofeachofthesescalescanbefoundinthesummarytableofmethods(seeTable2).OneoftheareaswherewefoundthemostsignificantdifferencesbetweengroupsandbetweencohortswaswithintheCreativeSkillsmeasure.Participantswereaskedtocompleteabrief“InnovationWarm-up”exerciseatthestartofthefirstdayoftheworkshop,andasimilar“InnovationWrap-up”atthecloseofthefinaldayoftheworkshop.ParticipantswereaskedtoidentifyproblemsrelatedtoagivenInnovationChallenge,selectonetoworkon,generatepossiblesolutionstotheselectedproblem,selectonesolution,andexplaintheirchoices(seeAppendicesFthroughHforthefullexercises).AdifferentInnovationChallengewasusedfortheseconduseoftheexercisetopreventanypracticeeffects.Participantshad15minutestocompletetheskillstest.ThesetaskswerederivedfromconceptssimilartothechallengesnotedinRuncoandBasadur’s(1993)articleonAssessingIdeationalandEvaluativeSkillsandCreativeStylesandAttitudes.Arubricthatincludedtheskillcategorieslistedbelowwascreatedforthescoringofthistest.Therubricwasdesignedtoinvestigatethefollowingcreativethinkingskills,andincludethetypeofskillitexamined:

Skill1:Howmanydistinctproblemswereidentified?(Divergent)Skill2:Howmanyideaclustersdothoseproblemsrepresent?(Divergent)Skill3:Howclearistheproblemstatementasrelatedtothechallenge?(Convergent)Skill4:Howmanyreasonsgivenforwhytheindividualchoosesthatproblem?(Convergent)Skill5:Howmanydistinctsolutionsweregenerated?(Divergent)Skill6:Howstrongisthesolutionstatement?(Convergent)Skill7:Howmanyreasons/evidencestatementsgivenforselectionoftheproblem?

(Convergent)Skill8:Howmanyreasons/rationalesgivenforchoosingsolution?(Convergent)


Skill9:Howspecificiswhattheparticipantproposestodo?(Convergent)Skill10:Howspecificastohowtheparticipantproposestoenacttheirsolution?(Convergent)Skill11:Howmanyspecificideaclustersdothesesolutionsrepresent?(Divergent)

DifferencesWithingroupsOurcoreobjectivewastodeterminewhetherornotthetypeofinnovationtrainingindividualsreceivedimpactedtheircreativethinkingskills.Beforewecouldcomparetheimpactoftrainingbetweenthecontrolandtreatmentgroups,weneededtocomparetheimpactofthetrainingoneachgroupindividually.AsshowninTable5below,thehighschoolstudenttreatmentgroupshowedstatisticallysignificantincreasesfrompretesttoposttestwithin6ofthe11creativethinkingskillvariables.Forthecontrolgroup,therewerenogainsonanyvariableafterthetraining;infact,thehighschoolcontrolgrouphadhigherpretestscoreswithinthecriticalthinkingvariablesand3ofthecreativethinkingskillswhencomparedwiththeirpost-testscores.Astrikingpatternemergedforthehighschoolstudents:outofthe13totalvariablesthatshowedsignificantdifferences,onlyoneofthetwogroups(controlortreatment)showedasignificantdifference.Furthermore,12ofthese13variablesshowedapositiveresultinthedirectionofthetreatmentgroup.Forexample,thehighschoolstudenttreatmenthad7variablesthatshowedasignificantincreasefrompretopost,andnoneoftheseweresignificantlydifferentforthecontrolgroup;treatmentincreased,controlstayedthesame.Likewise,thehighschoolcontrolgrouphad5variablesthatshowedasignificantdecreasefrompretopostforthehighschoolcontrolgroup,yettherewasnosignificantdifferenceforthetreatmentgroup;treatmentstayedthesame,whilecontroldecreased.Theseresultsstronglysuggestthatinadditiontoanybeneficialimpactofarts-basedlearningonspecificcreativethinkingskills,arts-basedlearningmayhavethecapacitytoovercomeandneutralizeanynegativeimpactoftraditionalinnovationtrainingonhighschoolstudentcreativethinking.Itwillbeinterestingtofurtherinvestigatethispossibleeffect,aswellastoconsiderwhetherthereareanypositiveimpactsoftraditionalinnovationtrainingonotherareaswedidnotmeasure,suchasapotentialincreaseincontentknowledgeaboutinnovation.Thispossibleimpactmeritsfurtherinvestigation.Itwouldalsobeinterestingtoconsiderwhetherthereareanypositiveimpactsofeitherarts-basedortraditionalinnovationtrainingonotherareaswedidnotmeasure,suchasapotentialincreaseincontentknowledgeaboutinnovation.Thehighschooltreatmentgroupshowednoperceivablegainonfourofthecreativethinkingskillsoronthecriticalthinkingandcreativityself-reportscales;theincreasewasonlymeasuredonthecreativethinkingskillstest,whichsuggeststhepossibilitythatarts-basedtrainingspecificallyhasanimpactonthetypeofskillsthistestwasdesignedtogauge:theabilitytoidentifyaproblem,articulatepotentialsolutions,andidentifyasolutionandarticulatearationaleforthatsolution.Thebreadthandspecificityofproblemsandsolutionsdescribedarealsoevaluatedwithinthisassessment.Itisalsopossiblethatdespiteeffortstobalancethegroupswithregardtoindependentvariablessuchaspriorartsexperience,scienceexperience,andcreativityscores,thecontrolgroupenteredwithahigherlevelofcreativeskills,andthusshowednochange;orthatthecreativeskillsassessment,withascoringrubricdevelopedaftertheimplementationoftheassessment,isinsomeunknownwaybiased.Nonetheless,thesimplestexplanationisthatthearts-basedtrainingincreasesthecreativethinkingskillsofhighschoolstudents.


TheresultsfromtheearlycareerSTEMprofessionalsarelessdefined.Asinthehighschooltreatmentgroup,buttoalesserdegree,theearlycareerSTEMprofessionalstreatmentgroupshowssomeincreasesincreativethinkingskills.Theyshownochangeinthecriticalthinkingskillsmeasures.Ononeself-reportscale,Creativity1,theearlycareerSTEMprofessionalstreatmentgrouphadstatisticallysignificantlyhigherscoresonthepre-test.Aswedidnotexpectthemtolosecreativityoverthecourseofthetraining,itisunclearwhatthisfindingmeans.TheearlycareerSTEMprofessionalscontrolgroupshowedlittleimpactfromthetraining,withtheexceptionofincreasesintwocreativeskills:numberofsolutionstatementsgenerating,andnumberofideaclustersrepresentedbythesesolutions.Table5:SummaryTableofIndividualCreativeThinkingSkillsScores(includingCriticalThinking)

Note:Seepages37-38aboveforexplanationofSkill1throughSkill11.

DifferencesBetweengroupsThissectionwillfocusonthedifferencesbetweenthecontrolandtreatmentgroups,organizedwithinthesub-hypothesesarticulatedabove.ThefourtablesbelowaddressthemainHypothesis1(seeTable6)aswellassub-hypotheses1a(seeTable7),1b(seeTable8),and1c(seeTable9).Forthehighschoolstudentcohorts,thesetablesshowedalargenumberofsignificantdifferencesthatwerefoundincreativeandcriticalthinkingscoresbetweenthecontrolandthetreatmentgroups,withthetreatmentgroupsoutperformingthecontrolgroups,mostfrequentlywithinthecreativeskillstests.Innocasedidcontroloutperformtreatment.WithrespecttoSub-Hypothesis1a(divergentthinking,Table7),thetreatmentgroupsignificantlyoutperformedthecontrolgroupin4of5variablesmeasured,withtheremainingvariableshowingnosignificantdifference.WithrespecttoSub-Hypothesis1b(convergentthinking,Table8),thetreatmentgroupsignificantlyoutperformedthecontrolgroupin3of6variablesmeasured,withtheremaining3variablesshowingnosignificantdifference.WithrespecttoSub-Hypothesis1c(criticalthinking,Table9),thetreatmentgroupsignificantlyoutperformedthecontrolgroup.

VariableHighSchool

StudentsTreatmentHighSchool

StudentsControlEarlyCareerSTEM

ProfessionalsTreatmentEarlyCareerSTEM

ProfessionalsControlCriticalThinking1 -- Pretesthigher -- --CriticalThinking2 -- Pretesthigher -- --

Mini-ECCI Posttesthigher

CPSP CreativeThinkingSkills:Skill1

Posttesthigher--

Posttesthigher--

Skill2 Posttesthigher -- -- --Skill3 Posttesthigher -- Posttesthigher --Skill4 -- Pretesthigher Posttesthigher --Skill5 Posttesthigher -- -- PosttesthigherSkill6 Posttesthigher -- -- --Skill7 -- Pretesthigher -- --Skill8 -- Pretesthigher -- --Skill9 Posttesthigher -- -- --Skill10 -- -- -- --Skill11 Pretesthigher -- -- Posttesthigher


ThesehighschoolfindingsmustbeinterpretedinthecontextofTable5above;insomecases,thetreatmentgroup’sstrongercomparativeperformanceresultedinpartfromadeclineintheperformanceofthecontrolgroupoverthecourseofthefivesessions,aswellasfromanincreaseintheperformanceofthetreatmentgroup.Inparticular,thecontrolgrouphadstatisticallyhigherscoresoncriticalthinkingandsomecreativeskillsonentrytothetrainingthantheyshowedfiveweekslater.SoinTable6throughTable9below,whilethehighschooltreatmentgroupsoutperformedthehighschoolcontrolgroupsbyastatisticallysignificantdifference,animportantfactorcontributingtothisdifferenceisthatinsomecases,thehighschooltreatmentgroupshadrelativelymodestpositivechangesincriticalthinkingskills,whilethehighschoolcontrolgroupshadadropinscoresfrompretesttoposttest.Whilethehighschooltreatmentgroupsdoshowincreasesincreativeskillsduringthecourseofthetraining,thedifferencesbetweentheirgroupandthehighschoolcontrolgroupisenhancedbythefactthatthehighschoolcontrolgrouphaddecreasesintheirscores,showinghigherscoresontheentrypretestthantheposttest.Onepossibleexplanationthatwarrantsfurtherstudyisthatamoretraditionalapproachtolearninginnovationmayinsomewaydepresscreativethinking,whilethearts-basedapproachmayoffsetthatnegativeimpact.SomeofthesedifferenceswerealsofoundwithintheearlycareerSTEMprofessionalsgroups,buttoaveryslightdegree(seeTable10throughTable13below).TherewerenostatisticallysignificantdifferencesbetweentheearlycareerSTEMprofessionalstreatmentandcontrolgroups,onlyaslightnon-significanttrendinCreativeSkill3(clarityoftheproblemstatement)towardsbetterperformanceinthetreatmentgroup. Table6:SummaryTableofDifferencesbetweenTreatmentandControlGroups,Hypothesis1 HighSchoolStudents EarlyCareerSTEMProfessionalsPretest-PosttestDifferenceScores

SignificantGroupDifferences

GroupwithBetterPerformance



CreativeSkill2 Yes Treatment No -CreativeSkill6 Yes Treatment No -CreativeSkill7 Yes Treatment No -CreativeSkill8 Yes Treatment No -CreativeSkill9 No - No -CreativeSkills10 Trend Treatment No -CreativeSkills11 No - No -Mini-ECCI Yes Treatment No -Slover-LinnettCreativeProcess No - No -

Basadur’sProblemSolving(CPSP) No - No -

CommonPurpose No - No -ObservedCommonPurpose

No - No -

CreativeSkills11 No No -Note:Seepages37-38aboveforexplanationofSkill1throughSkill11.


Table7:SummaryTableofDifferencesbetweenTreatmentandControlGroups,Hypothesis1a HighSchoolStudents EarlyCareerSTEMProfessionalsPretest-PosttestDifferenceScores





CreativeSkills1 No - No -CreativeSkills2 Yes Treatment No -CreativeSkills3 Yes Treatment Trend TreatmentCreativeSkills4 Yes Treatment No -CreativeSkills5 Yes Treatment No -Note:Seepages37-38aboveforexplanationofSkill1throughSkill11. Table8:SummaryTableofDifferencesbetweenTreatmentandControlGroups,Hypothesis1b HighSchoolStudents EarlyCareerSTEMProfessionalsPretest-PosttestDifferenceScores





CreativeSkills6 Yes Treatment No -CreativeSkills7 Yes Treatment No -CreativeSkills8 Yes Treatment No -CreativeSkills9 No - No -CreativeSkills10 Trend Treatment No -CreativeSkills11 No - No -Note:Seepages37-38aboveforexplanationofSkill1throughSkill11. Table9:SummaryTableofDifferencesbetweenTreatmentandControlGroups,Hypothesis1c HighSchoolStudents EarlyCareerSTEMProfessionalsPretest-PosttestDifferenceScores





CriticalThinking Yes Treatment No -


Inthefollowingsection,wewillgointomorespecificdetailontheresultsforthedescribedindicatorsandsub-scales. Table10:TableofDivergentCreativeSkillsDifferences,HighSchoolStudents,Hypothesis1a

MeasurementResultsforHighSchoolStudents

Averages,PretoPost Notes

Changeinnumberofdistinctproblemsidentified

Thetreatmentgroupshowedahighermeannumberofviableproblems(approximatelytwiceasmanyonaverage)thanthecontrolgroup,butthatdifferencedisappearedoncepre-sciencewascontrolledfor.

ControlPre:1.7ControlPost:2.3TreatmentPre:1.5TreatmentPost:2.9

Variabilityinbothgroupswashigh.Thiswasduetoanumberofparticipantswholostgroundfromthepretesttotheposttest.Twiceasmanycontrolgrouphighschoolstudentslostground,asdidtreatmentgroupstudents.

Changeinnumberofideaclusterstheproblemsrepresent.

Treatmentgrouphadagreaterchangeinthenumberofideaclusters.


Changeinnumberofdistinctsolutions

Highschoolstudentsintreatmentgroupchangedmoreinnumberofdistinctsolutionslisted.


Changeinnumberofideaclustersthesolutionsrepresent

Nosignificantdifferenceinchangebetweengroups.


Thetreatmentgroupmademoregains.Nostatisticallysignificantdifferenceislikelybecausethevariabilityisconsiderable,andlargerinthecontrolgroup.

Changeinnumberofclustersofreasonsfortheselectionofthesolution

Treatmentgroupgainedsignificantlyinthenumberofclustersofreasonsofferedcomparedtothecontrolgroup.


Itmaybethatthesignificantdifferenceisduetocontrolslosinggroundonthismeasureratherthantreatmentgroupmembersgainingsignificantground.Thisisthefirstmeasureinwhichsomanytreatmenthighschoolstudentsshowedstronggains;wouldbegoodtoinvestigatethismeasuretounderstandwhytheymighthavedoneso.


Table11:TableofConvergentCreativeSkillsDifferences,HighSchoolStudents,Hypothesis1b

MeasurementResultsforHighSchoolStudents


Changeinclarityofproblemstatement

Treatmentgroupsgainedmoreincreatingbetterproblemstatements.


Theconfidenceintervalofthedifferencebetweenmeansshowedthatthedifferencebetweenthegroupswasasmuchasanentirepoint,whichisquitelargeona3-pointscale.

Changeinstrengthofsolutionstatement

Treatmentgrouphadmoregainsinprovidingstrongersolutionstatementscomparedtothecontrolgroup.


Theconfidenceintervalofthedifferencebetweenmeansshowedthatthedifferencebetweenthegroupswasaboutoneentirepoint.

Changeinnumberofreasonsfortheselectionoftheproblem

Highschoolstudentsinthetreatmentgroupgainedsignificantlymoreinnumberofrationalesoffered.


Generallyspeaking,highschoolstudentsinthecontrolgroupdidnotincreasetheirnumberofrationalesgivenfrompretopost.

Changeinnumberofreasonsfortheselectionofthesolution.

Treatmentgroupgainedmorethanthecontrolgroupinthenumberofrationalesfortheirsolutions.


Whiletherewasaslightincreaseintreatmentpretesttoposttest,thesignificanceresultedfromthedecreaseinthecontrolgroup.

Changeinspecificityofwhatthesolutionis



Neitheragainnoralossonthismeasure.

Changeinspecificityofhowthesolutionwillbeenacted

Treatmentgroupchangedmoreinspecificityinhowtoenactasolution.

ControlPre:0.2ControlPost:0.1TreatmentPre:Lessthan0.1TreatmentPost:0.1

Notethat81%showedneitheragainnorlossonthismeasure.Onaverage,thecontrolgrouplostground,thetreatmentgroupgainedground.


Table12:TableofDivergentCreativeSkillsDifferences,EarlyCareerSTEMProfessionals,Hypothesis1a

Measurement

ResultsforEarlyCareerSTEMProfessionals


Changeinnumberofdistinctproblemsidentified



Whiletherewasnostatisticallysignificantdifferenceinthemeans,theadulttreatmentgroupsawalargergainonthismeasurepretopost,whilemoreofthecontrolgrouplostgroundfrompretopost.

Changeinnumberofideaclusterstheproblemsrepresent.



Changeinnumberofdistinctsolutions



Changeinnumberofideaclustersthesolutionsrepresent



Thecontrolgroupmadefractionallymoregainsfromthepretesttotheposttest,onaverage,thandidthetreatmentgroup.Theamountofvariabilitywasconsiderable.

Changeinnumberofclustersofreasonsfortheselectionofthesolution




Table13:TableofConvergentCreativeSkillsDifferences,EarlyCareerSTEMProfessionals,Hypothesis1b

Measurement

ResultsforEarlyCareerSTEMProfessionals


Changeinclarityofproblemstatement

Trendwherethetreatmentgroupgainedmoreincreatingbetterproblemstatements.


Thisoutcomemayhavehitstatisticalsignificance:1)iftheeffecthadbeenalittlestronger,2)thereweremanymoreparticipants,or3)therewaslessvariabilityinthedata.

Changeinstrengthofsolutionstatement



Thedifferencewentinthedirectionofslightlyhighercreativityamongthetreatmentadults,comparedtocontroladults,whoaveragednogainonthisdifferencemeasure.However,thedifferencewasnotsignificant.

Changeinnumberofreasonsfortheselectionoftheproblem



Thismeasurewasdivisiveforcontroladults:somegained,otherslost.

Changeinnumberofreasonsfortheselectionofthesolution.



Changeinspecificityofwhatthesolutionis



Highdiversityinscoresamongtheadultsbutthecontrolgroupshowedmorelosses,overtime.

Changeinspecificityofhowthesolutionwillbeenacted

Asmallnon-significantdifferencebetweenthecontrolandtreatmentadultsonthismeasure.


Onaverage,bothcontrolandtreatmentgroupslostasmallbitofground.


CriticalThinkingWeusedBasadurandFinkbeiner’s(1985)scaleonpreferenceforideationandtendencyforprematurecriticalevaluationofideas.ThisscalewasusedinRuncoandBasadur’s(1993)studyonideationandevaluativethinkingchangeduringworkplace-basedinnovationtraining,anddemonstratedadifferencebetweencontrolandtreatmentgroups.Eachoftheseitemswithinthescalewasscoredinbothpreandpostsurveys,andthentheresultswereusedtogenerateameanscoreforbothpreandpost.Theamountofchangeinmeanscoreforthetreatmentgroupswasthencomparedagainsttheamountofchangeforthecontrolgroups.Forthehighschoolgroups,thetreatmentgroupmadesignificantlygreaterpre/postgainsthanthecontrolgroup(seeTable14).Theimpactwaslarge;withthetreatmentgroupscoringasmuchas.75pointshigherthanthecontrolgrouponsomeitems.Thecorrelationmatrixshowedthatforthesecohorts,pre-artandpre-sciencescorescorrelatedstrongly,directlyandsignificantlywitheachother,r(63)=.52,p<.01.However,thecriticalthinkingdifferencedatadidnotcorrelatesignificantlywithpre-art,r(59)=-.14,p=.29,orwithpre-science,r(59)=.10,p=.43.Therefore,highschoolstudentdifferenceswereexaminedwithanindependentsamplest-test.Therewasasignificantdifferenceinthedirectionofgreatergainsmadebythetreatmentgroup,with32participants,comparedtothecontrolgroup,with29participants,t(59)=-2.84,p<.01.Theeffectofthearts-basedlearningwaslarge,Glass’Delta=.70.The95%confidenceinterval(CI)forthemeandifferences,-0.69,-0.11,showedthatthecontrolgroupscoredasmuchasthree-quartersofapointloweroncriticalthinkingthanthetreatmentgroup,respectively.FortheearlycareerSTEMprofessionals,therewasnosignificantdifferenceinchangebetweenthetwogroups(seeTable14).Thecorrelationmatrixshowedthatnoneofthecorrelationsweresignificant(pre-artandpre-science:r(67)=.11,p=.39;pre-scienceandcriticalthinkingdifference:r(67)=-.02,p=.88;pre-artandcriticalthinkingdifference:r(67)=.13,p=.29.Therefore,earlycareerSTEMprofessionalsgroupdifferenceswereexaminedwithanindependentsamplest-test.Therewasnosignificantdifferencebetweenthetreatmentandcontrolgroup,t(67)=1.35,p=.18.Thedifferencebetweenthetreatment(with37participants)andcontrolgroup(with32participants)wasasmuchashalfapointbutdidnotreachsignificance,95%CI;-0.10,0.51.Amongthetreatmentgroup,someadultsgainedasmuchasawholepoint(maximum=1.07)whereassomeotheradultslostnearlyawholepoint(minimum=1.00).Insum,thismeansthatboththearts-basedandthetraditionalinnovationtraininghaddivisiveeffectsoncriticalthinkingforadults,inthatsomebenefitedgreatlybutotherslostground.Table14:CriticalThinkingScore(BasedonBasadurandFinkbeiner)HighSchoolStudent

ResultsHighSchoolStudent

NotesEarlyCareerSTEMProfessionalResults

EarlyCareerSTEMProfessionalNotes

Significantdifferenceinthedirectionofgreatergainsmadebythetreatmentgroupcomparedtothecontrol.

Theeffectwaslarge,showingthatthecontrolgroupscoredasmuchasthree-quartersofapointlowerthanthetreatmentgroup.


Amongthetreatmentgroup,someadultsgainedasmuchasawholepointwhereassomeotheradultslostnearlyawholepoint.Bothtreatmentandcontrolhaddivisiveeffectsoncriticalthinkingforadults,inthatsomebenefitedgreatlybutotherslostground.


CreativeThinkingSkillsSelf-Report

CreativeCompetenciesInventory(mini-ECCI) Oneofthemeasureswithinourpre-postassessmentsforCreativeThinkingwasthemini-ECCI.TheECCIstandsforEpsteinCreativityCompetenciesInventoryforIndividuals,andwasdevelopedspecificallyasaninstrumentformeasurable,trainablecompetencies.Previousstudies(Epstein,Schmidt,&Warfel2008)havesuggestedthatcreativeoutputcanbeincreasedthroughworkonstrengtheningthefollowingfourcompetencies:1)capturing(preservingnewideasastheyoccur),2)challenging(takingondifficulttasks),3)broadening(seekingknowledgeandskillsoutsideone'scurrentareasofexpertise),and4)surrounding(seekingoutnewstimuliorcombinationsofstimuli.TheversionoftheECCIusedherewasamini-version,designedforusewithindividuals,andwithfeweritemsthantheoriginal.Forthehighschoolstudents,thetreatmentgroupshowedsignificantlygreatergains,whenstatisticallycontrollingfortheeffectofpre-scienceexposure(seeTable15).Interestingly,themoreCreativeCompetencygainsparticipantsshowedfrompretopost,thelesstheyreportedpre-artorpre-scienceexposure.Pre-sciencebutnotpre-artwasasignificantcovariate.Thismightimplythathighschoolstudentswithlessexperienceinartorsciencepriortotheworkshoparemorelikelytoshowshort-termmeasurableimpactsfromarts-basedinnovationtraining.FortheearlycareerSTEMprofessionals,therewerenosignificantdifferencesinchangebetweenthecontrolandthetreatmentgroups.Neitherpre-artnorpre-scienceexposureshowedsignificantcorrelationsfortheadults.Overall,creativity,asmeasuredbytheECCIscale,significantlyincreasedinthehighschooltreatmentgroup,anddecreased(thoughnotsignificantly)withinthecontrolgroup(seeTable16).Table15:ECCIHighSchoolStudentsPre/PostChangeScoreBetweenGroups

MeanControl SD MeanTreatment SD Sig?

ECCIChangeScore -0.6 4.8 4.1 9.2Yes

(p<.05) Table16:ECCIHighSchoolStudentsPre/PostChangeScore

MeanPre SD MeanPost SD Sig?

Control 59.3 7.1 58.7 6.2 No

Treatment 60.6 7.5 64.7 9.1Yes

(p<.05)Thedifferenceinchangescoreswasnotduetothegroupsbeginningatadifferentstartingpoint,astherewasnostatisticaldifferenceintheprescoreforthetwogroups(seeTable17).


Table17:ECCIHighSchoolStudentsPreScoreComparisonBetweenGroups MeanControl SD MeanTreatment SD Sig?

ECCIPreScore 59.3 6.9 60.6 7.5

No PreandpostscoresfortheearlycareerSTEMprofessionalshadfairlysimilarmeans(seeTable18).Inboththecontrolandthetreatmentgroups,ECCIscoreswentdowninthepostassessment,significantlyso.TheECCIscorefellmoreinthecontrolgroupthaninthetreatmentgroup,andthedifferencebetweenthepreandthepostscoreswassignificant.Incomparingtheamountsofthedeclinebetweencontrolandtreatment(seeTable19),therewerenosignificantdifferencesinthechangescores.Whilethecontrolgrouphadbegunhigherthanthetreatmentgroup,thiswasnotstatisticallysignificant(seeTable20).Table18:ECCIEarlySTEMProfessionalsPre/PostChangeScoreWithinGroups

MeanPre SD MeanPost SD Sig?

Control 63.2 6.2 60.9 6.2Yes

(p<.05)

Treatment 61.7 6.1 59.9 5.4Yes

(p<.05)Table19:ECCIEarlySTEMProfessionalsPre/PostChangeScoreComparisonBetweenGroups


ECCIChangeScore -2.3 6.1 -1.8 4.2 No Table20:ECCIEarlySTEMProfessionalsPreScoreComparisonBetweenGroups


ECCIPreScore 63.2 6.2 61.7 6.1 No Overall,therewasnotanincreaseinthecreativityaspectsasmeasuredbytheECCIscaleintheearlycareerSTEMprofessionals.Infact,therewasastatisticallysignificantdecreaseincreativityinbothtreatmentandcontrolgroups.

CreativeProblemSolvingProfile(CPSP)Anotherscalewithinthepre-postassessmentwastheCreativeProblemSolvingProfile(CPSP)developedbyBasadur,Graen,andWakabayashi(1990).Thescalemeasuresindividualstrengthwithinfourdifferentcomponentsofthecreativityprocess:generation,conceptualization,optimization,andimplementation.Eachphasehasuniqueattributes.Ageneratorcreatesoptionsintheformofnewpossibilitiesornewproblemsthatmightbesolvedandnewopportunitiesthatmightbecapitalizedon.Aconceptualizercreatesoptionsintheformofalternatewaystounderstandanddefineaproblemor


opportunity,andgoodideasthathelpsolveit.Anoptimizercreatesoptionsintheformofwaystogetanideatoworkinpracticeanduncoveringallofthefactorsthatgointoasuccessfulimplementationplan.AnImplementercreatesoptionsintheformofactionsthatgetresultsandgainacceptanceforimplementingachangeoranewidea.Oneindividualmayhaveamixofthesestrengths.Basaduretal.(1990)assertthatdifferentindividualshavestrengthswithindifferentphasesofthecreativeprocess,andthatthisscalemeasurestheirrelativestrengths.Thisknowledgecanhelpthemimprovewithinareasofthecreativeprocessorchoosetocontributetheireffortstocertainphasesofcreativitythataremoreproductive.Theyholdthatcreativitytrainingcanimproveindividuals’strengthswithinthesedomains.Whenwecomparechangescoreswithinthehighschoolstudents(seeTable21),therewerenosignificantdifferencesbetweenthetreatmentandcontrolgroups.Table21:CPSPHighSchoolStudentsPre/PostChangeScoreComparisonBetweenGroups

Sub-score MeanControl SD MeanTreatment SD Sig?GenerationChangeScore

-0.8 5.7 -1.2 6.5 No

ConceptualizationChangeScore

1.3 3.1 1.2 4.5 No

OptimizationChangeScore

-0.2 3.5 -1.3 3.8 No

ImplementationChangeScore 0.7 4.7 1.4 4.2 No

Withingroups,thecontrolgroupdidexperienceastatisticallysignificantincreasewithintheconceptualizationsub-score(seeTable22).Table22:CPSPHighSchoolStudentsPre/PostChangeScore,ControlGroupOnly

Sub-score MeanPre SD MeanPost SD Sig?GenerationChangeScore

24.8 4.9 24.7 4.8 No

ConceptualizationChangeScore 19.3 2.9 20.6 3.6 Yes

(p<.05)OptimizationChangeScore 23.6 2.5 23.4 3.9 No



Thetreatmentgrouphadnosignificantlydifferentscoresonthesub-scales(seeTable23).Table23:CPSPHighSchoolStudentsPre/PostChangeScore,TreatmentGroupOnly

Sub-score MeanPre SD MeanPost SD Sig?GenerationChangeScore 27.2 3.6 26.0 5.0 No

ConceptualizationChangeScore 21.3 3.1 22.5 3.7 No

OptimizationChangeScore 25.0 3.4 23.7 3.7 No

ImplementationChangeScore

26.6 3.7 28.0 4.3 No

Onthissamesub-scorewheretherewasanincreaseinthecontrol,therewasalsoadifferenceinwherethegroupsbegan(seeTable24).Thetreatmentgroupbegantheprogramsignificantlyhigherthanthecontrolgroupinconceptualization.Table24:CPSPHighSchoolStudentsPreScoresComparisonBetweenGroups

Sub-score MeanControl SD MeanTreatment SD Sig?GenerationChangeScore 25.0 5.0 27.2 3.6 No

ConceptualizationChangeScore 19.3 2.9 21.2 3.2

Yes(p<.05)


23.9 2.5 25.0 3.3 No


26.1 3.5 26.4 3.7 No

Thelackofstatisticallysignificantdifferencesamongthehighschoolstudentcohortswhenmakingadirectcomparisonbetweencontrolandtreatmentscoresforthefoursub-scoresontheCPSPmeansthatnoneofthefoursub-scalesshowedadistinctadvantagepre/postforonecohort(controlortreatment)overtheother.Whenlookingjustatthedifferencescoreswithineachgroup,thetreatmentgroupdidnotshowanysignificantdifferencesfrompretopostforthefoursubscales.Thecontrolgroupshowedonesignificantdifferenceinthesubscales,asignificantincreasefortheconceptualizationchangescore.Onefactorinthisdifferencewasthatthecontrolgroup’smeanpretestscorewassignificantlylowerafullpointthanthetreatmentgroup,whichleftmoreroomforthecontrolgrouptoimprove.TheCreativeProblemSolvingProfilecouldbeseenasmeasuringasetofskillsthatcouldbeverydifficulttochangeinashortperiodoftimegiventhattheyrepresentaperson’sgeneralapproachtothecreativityprocess.


TherewerenosignificantdifferencesbetweenchangescoresofthecontrolandtreatmentgroupsforearlySTEMprofessionals(seeTable25).Table25:CPSPEarlySTEMProfessionalsChangeScoreComparisonBetweenGroups

Sub-score MeanControl SD MeanTreatment SD Sig?GenerationChangeScore -1.8 2.8 -1.7 3.1 No

ConceptualizationChangeScore -0.2 2.8 -0.6 3.1 No

OptimizationChangeScore -0.6 3.5 -1.2 2.6 No


-0.7 2.9 -1.1 3.7 No

ThecontrolgroupdidhaveaslightdeclineintheGenerationsub-score(seeTable26),asdidthetreatmentgroupofearlycareerSTEMprofessionals(seeTable27).EarlycareerSTEMprofessionalsalsoexperiencedaslightdecreaseinontheoptimizationsub-scale.Table26:CPSPEarlySTEMProfessionalsPre/PostChangeScore,ControlGroupOnlySub-score MeanPre SD MeanPost SD Sig?GenerationChangeScore 27.9 3.7 26.1 3.7

Yes(p<.01)



24.0 3.3 23.4 2.2 No


26.4 3.9 25.8 3.7 No

Table27:CPSPEarlySTEMProfessionalsPre/PostChangeScore,TreatmentGroupOnly

Sub-score MeanPre SD MeanPost SD Sig?GenerationChangeScore 28.1 3.6 26.4 3.8

Yes(p<.01)



24.3 3.2 23.2 2.4 Yes(p<.05)


27.0 3.1 25.9 3.9 No


TherewerenostatisticallysignificantdifferencesinthestartingpointsofeachofthesegroupsontheCPSPscales(seeTable28).Table28:CPSPEarlySTEMProfessionalsPreScoresComparisonBetweenGroupsSub-score MeanControl SD MeanTreatment SD Sig?GenerationChangeScore

27.9 3.7 28.3 3.7 No


OptimizationChangeScore 24.2 3.3 24.3 3.2 No


TransferabilityofSkillsIncreasingindividualcreativethinkingskillsandhelpinglearnersdevelopattributestopreparethemforthe21stCenturySTEMworkplacearecommongoalsinmanyinformalSTEMlearningprojects.Nonetheless,theArtofScienceLearningprojectstandsoutintheproject’sfocusoninnovationandtotheextenttowhichitaimstochangepracticeinthehome,workplace,andelsewhere.Asshowninthecommentselsewherewithinthisreport,someindividualstookhomethelessonsandexperiencesfromtheirinnovationtrainingandreportedapplyingthemtoawiderangeofissues.TheAVCteamwishedtomeasurehowwellthoseskillsweresynthesizedandappliedelsewhereinparticipants’lives.Fourmonthsaftertheparticipantshadcompletedtheirinnovationtraining,theywereaskedtofilloutapost-workshopsurveyandreflection,askingtheextenttowhichthelessonstheylearnedduringthetrainingcouldbeappliedtoothercontexts,bothcurrentandfuture.Thequestionswerevariationsof“TowhatextenthaveyoubeenabletoapplytheInnovationChallengeexperiencetoyourcurrentworkorvolunteeractivities?”Thesewerepost-testitemsonly.Responseswereratedonascaleof1to7,with7indicatingthehighestleveloftransference.AcompositetransferabilityofskillsscorewascalculatedasanaverageofthefiveitemslistedinTable29.Forthehighschoolstudents,thetreatmentgroupratedthetransferabilityoflessonsfromthechallengetocurrentandfuturecontextssignificantlyhigherthandidthecontrolgroup.Theeffectwasverylarge;theactualdifferencebetweenthetwogroups’assessmentsrangedasmuchastwopoints.Thissuggeststhatthosewithinthetreatmentgroupexperiencedamuchgreatertransferenceoftheskillsintotheireverydaylives.Oneitem,usinga7-pointscale,askedtheextenttowhichtheparticipantswouldapplytheirexperiencestofutureworkandvolunteering:1outof5(20%)inthecontrolgroupratedita1“notatall,”whilenotoneperson(0%)inthetreatmentgroupratedita1,2or3.Conversely,while1outof5(21%)inthecontrolgroupsaidtheywereverylikely(a6or7)toapplytheirexperiencestofutureworkandvolunteering,peopleinthetreatmentgroupwerethreetimesmorelikely(65%)tosaytheywereverylikelytodoso.Anotheritemusingthesamescaleaskedtheparticipantsaboutapplyingtheirexperiencestoschoolorextracurricularexperiences:again,1outof5(20%)inthecontrolgroupratedita1“notatall,”andagainnooneinthetreatmentgroupratedita1,2or3.Ontheotherendofthescale,onlyoneoutofthree(30%)inthecontrolgroupratedita6or7,whiletwiceasmany(60%)inthecontrolgroupthoughttheywouldbeverylikelytoapplytheseexperiencestoschoolorextracurricularactivities.


Itisimportanttonote,however,thereweremanymissingvaluesfromtheindividualswhochosenottofilloutthefinalsurvey.Asapproximatelyhalfofeachofthecontrolandtreatmentgroupsdidnotfilloutthesurvey,itispossiblethisfindingisanartifactofthesampleresponding.Furtherworkwoulddowelltoresearchthisaspectoftransferabilitymorethoroughly.FortheearlycareerSTEMprofessionalsgroupstherewerenostatisticallysignificantdifferencesinthecompositescorebetweenthecontrolandtreatmentgroups(seeTable29).Withintheparticipantsthereweremanymissingvalues,uptohalftheoriginalsamplewhocompletedthefive-weekworkshop.Thisquestionshouldbereplicatedwithinanyfurtherworkinthisvein.Table29:TransferabilityofSkillsLearningAverageScores

HighSchoolStudent

AveragesEarlyCareerSTEM

ProfessionalAveragesTowhatextenthaveyoubeenabletoapplytheexperiencetoyourcurrentworkorvolunteeractivities?

Control:3.7Treatment:4.8


Towhatextentdoyouthinkyouwillbeabletoapplytheexperiencetoyourfutureworkorvolunteeractivities?



Towhatextenthaveyoubeenabletoapplytheexperiencetoyourcurrentschoolorextracurricularactivities?



Towhatextentdoyouthinkyouwillbeabletoapplytheexperiencetoyourfutureschoolorextracurricularactivities?



Towhatextenthaveyoubeenabletoapplytheexperiencetoyourcurrenthome/personallife?




DefinitionofInnovation

HighSchoolStudentsHighschoolstudentparticipantswereaskedtowritetheirdefinitionofinnovationaspartofthePreWorkshopSurvey.Individualshadawidevarietyofresponsesthatfellintoafewdifferentcategories(seeTable30).Mostoften,theseparticipantsdescribedinnovationas“newideas”(55%);thenextmostcommondefinitionofinnovationinvolvedreferencestoproblemsandsolutions(25%).Table30:DefinitionofInnovationPreSurvey,HighSchoolStudents

Control(n=33)

Treatment(n=32)

Total(n=65)

Newideas 58% 53% 55%Referencetosolutionsand/orproblems 24% 25% 25%Uniquenessofidea 15% 19% 17%Processofinnovation–brainstorm,reframing,iterating,experiment,collaboration,strategicthinking,etc.

9% 19% 14%

Beingcreative 12% 12% 12%Workethic/workapproach,personalcharacteristicsoftheinnovator 3% 12% 8%

Productimplementation,bringingtomarket/world,adopter,incremental 0% 3% 1%

Miscellaneous 12% 6% 9%Themostpopulardefinitionofinnovationgivenbyhighschoolstudentswasasnewideas.Morethanhalfofthehighschoolstudents(55%)includedthisthemeaspartoftheirdefinition. MydefinitionofInnovationistobringinnewideas.

Innovation,inmyopinion,istheuseofnewideasandsolutionsforagreater,goodpurpose.Mydefinitionofinnovationiscreatingsomethingnewtohelpothers.

Onequarterofstudents(25%)definedinnovationintermsofsolutionsandproblems.

Mydefinitionofinnovationwouldbetheprocessofbrainstormingandputtingtoworkideasandtheories.

Mydefinitionofinnovationishavinganew,better,andstrongersolutiontoaproblemthatisputinfrontofyou.Formeinnovationisexperimentingwithaspecificprobleminordertosolveit.Iseeitasawayofmakingthingsbetterforourworld,howeveritdoesn'tneedtoonlybemechanical,itcanbeemotionalaswell.

Somehighschoolstudents(17%)wroteabouttheuniquenessofanideaasadefinitionofinnovation.

Mydefinitionofinnovationisanewideathatnobodyhaseverthoughtof.Anideathatstands


outfromtherest.However,standingoutisnotthemostimportantelement.Themostimportantelementisputtinganeffortintoone'swork. Innovationistakinganideaandmakingitwilder.Takingtheoldandthenewandformingsomethingoutofthisworld.Innovationisbeingunique,beingdifferent,notfollowingthenormbutratherstrengtheningtheweirdandwacky,itstakingtheidea'softheinsaneandfindingasolutiontoaproblemwiththem,inventinganewmachinewiththemandbetteringhumanitywiththem.

Somestudents(14%)usedwordsabouttheprocessofinnovationtodefinetheirideas.

Innovationisdevelopmentofaconceptorideainordertoimproveasituationorwayoflife.

Someonewhointroducesandcommunicatesanovelwayofthinking,interpreting,producing,orsolving.

Ithinkthatinnovationisusingnewideasthatareusuallyunrelatedtoacertainsituationtosolveaproblemthathasarisen.Ialsothinkitissomewhatofatrialanderrorprocess,workingthingsoutuntiltheyareperfect.

Afewstudents(12%)thoughtthatbeingcreativewasadefinitionofinnovation.

Innovationisyourabilitytobeoriginal,createnewthings,likeanideaorproject.Itstronglycorrelateswithcreativity.Mydefinitioniswhenoneusestheirresourcestocontractandsuggestnewandcreativeideasormethods.Innovationiswhenyoucancreativelyorartisticallyallowyourselftochallengeyourselftocreatenewideas.

Afewstudents(8%)wroteabouttheworkethicandpersonalcharacterofaninnovator. Mydefinitionofinnovationissomeonewhoworkshardandisproductiveandorganized. Tomethatworkinghardandbringingcleverideaswillleadtoaninnovator.Averysmallnumberofstudents(1%)definedinnovationinveryhigh-levelterms,includingproductimplementationandbringingproducttomarket.

Creatingnewwaysandcomingupwithnewideasandtakingthemtoawholenewlevel,meaningtoputthemintogooduseandgoodservicetothecommunity.

Andafewhighschoolstudentsgaveresponsesthatdidnotfitintoanyofthesecategories.

Isthatinnovationisthatyoustartanideafromzeroandleadittobecomehero.Thenextupandcoming.


EarlyCareerSTEMProfessionalsEarlycareerSTEMprofessionalparticipantswerealsoaskedtowritetheirdefinitionofinnovationaspartofthePreWorkshopSurvey.Individualshadawidevarietyofresponsesthatfellintoafewdifferentcategories(seeTable31).Theseresponsesshowedabroadsimilaritytothoseofthehighschoolstudentcohorts.Mostoften,fortheearlycareerSTEMprofessionals,participantsdescribedinnovationas“newideas”(58%);thenextmostcommondefinitionofinnovationinvolvedreferencestoproblemsandsolutions(39%).Table31:DefinitionofInnovationPreSurvey,EarlyCareerSTEMProfessionals

Control(n=32)

Treatment(n=37)

Total(n=69)

Newideas 59% 57% 58%Referencetosolutionsand/orproblems 53% 27% 39%Processofinnovation–brainstorm,reframing,iterating,experiment,collaboration,strategicthinking,etc.

28% 22% 25%

Uniquenessofidea 19% 27% 23%Beingcreative 16% 11% 13%Productimplementation,bringingtomarket/world,adopter,incremental

12% 5% 9%

Workethic/workapproach,personalcharacteristicsoftheinnovator 3% 0% 1%

Miscellaneous 3% 8% 6%Themostpopulardefinitionofinnovationgivenbyadultparticipantswasasnewideas.Morethanhalfoftheadultsincludedthisthemeaspartoftheirdefinition. Idefineinnovationasanythingnew. Toinventorbegintoapplynewmethodsorideas

Anewidea,anewwayofdoingsomething.Overonethirdofadults(39%)definedinnovationintermsofsolutionsandproblems.

Innovationislookingataproblem,generatingdifferentsolutions,andfindinganoutcomethatimprovesthecurrentstate.Developingnewapproachesormethodstoreachsolutions.Innovationistakingsomethingyoualreadyhaveorknowandapplyinganewmethodtosolveaproblem.

Onequarterofadults(25%)usedwordsabouttheprocessofinnovationtodefinetheirideas.

Innovationistheprocessofbrainstorminganddevelopingneworuniquesolutionstoexistingissues.Theinnovationdoesnotnecessarilyhavetobesubstantialinnature.Evenincrementalinnovationcanleadtomassiveimprovements.


Innovationisideationthatcomestofruition--itisthewholeprocessfromdefiningtheproblemtobrainstorming,toevaluating,toselecting,toexecutingasuccessfulsolution.Itisusedtosolveimmediateproblemsthatinhibitwork/school/progress.Innovationiscombiningcreativityandknowledgetodevelopideas/products/solutionsthathavethepotentialtoaffectmanypeoplein(hopefully)apositiveway.Innovationhasnolimits.

Almostonequarterofadultparticipants(23%)wroteabouttheuniquenessofanideaasadefinitionofinnovation.

Thinkingoutsidetheboxtocomeupwithnovelsolutionsortotakefromoutsidedisciplinesorideasinordertobettersolveanissueorproblem.Youdon'tneedtorecreatethewheel,younearlyneartomakeitbetter.Usingcreativemethodsforthinkingandcollaborationtogeneratenewandusefulideasandsolutionstodailyproblems.Oftenaprocessthatleadstobetterefficiencyintheworkplaceandwithinpersonalspacesaswell.Developingorcreatingsomethingnovel.

Someadults(13%)thoughtthatbeingcreativewasadefinitionofinnovation.

Usingthecreativeprocesstodevelopnewideasorprocesses.

Innovationisoriginalthoughtasitrelatestocreativewaystosolvedifficultproblems.Theseproblemsareonesthatimpactcommunities,socreativesolutions/innovationsarenecessarywhentherearesomanystakeholdersinvolved.Thinkingofnewwaysforthingstowork.Creativeproblemsolving.

Afewadultparticipants(9%)definedinnovationinveryhigh-levelterms,includingproductimplementationandbringingproducttomarket.

Whenyouturnanideaintoaproductormethodthatisbetterthanothersthatalreadyexists.

Anewmethod,idea,orproduct,putontrialwiththehopeofsuccessandacceptanceofscienceorartintheeyeofthepublic.

Averysmallnumberofadults(1%)wroteabouttheworkethicandpersonalcharacterofaninnovator.

Innovationcantakemanyforms.Peoplewhoareinnovativearenotafraidtochallengethenorm.Theytakeapartanissue,wraptheirheadsaroundit,andthengooutintospaceandlookattheprobleminitsentirety.Inanutshell,Ibelievethegenerationofanynewwaytotackleaproblemcanbeclassifiedasinnovation.Innovationcanalsobethemodificationofexistingknowledge.Comingfromanengineeringbackground,Icanseeanyincreaseinefficiencyorproductionasinnovative.

AndafewearlySTEMprofessionalsgaveresponsesthatdidnotfitintoanyofthesecategories.

Orderoutofchaos...


SelfPerceptionasInnovatorThisstudyhadmanyscalesoncreativityandcriticalthinking,whichtheprojectteamhypothesizedwouldincreaseinindividualsoncetheyhadhadtrainingininnovation.Asthetrainingsubjectmatterwithinboththetreatmentandthecontrolgroupsfocusedoninnovation,weaskedparticipants,preandpost,whethertheyperceivedthemselvestobeinnovatorsatwork,school,andhome.Thequestionwasaskedona1to7scale,with1representingStronglyDisagreeand7representingStronglyAgree.Withinthehighschoolstudentcohorts,therewerenosignificantdifferencesbetweencontrolandtreatmentgroups,buttherewasaslighttrend(seeTable32).Thistrendwasnotduetoanincreaseinthetreatmentgroup.Rather,thecontrolgroupmeanshowedamildloss,treatmentgroupdidnotshowagainorloss.FortheearlycareerSTEMprofessionals,therewasnotmuchdifferencebetweenthepretestandposttestscoresforeitherthetreatmentorcontrolgroupininnovatorself-assessmentsamongadults,butwhatdifferencestherewere,tendedtobelessintheposttest. Table32:ChangeinSelfPerceptionofInnovation(InnovatorSelf-AssessmentDifferenceScore)

HighSchoolStudentResults

HighSchoolStudentAverages

EarlyCareerSTEMProfessionalResults

EarlyCareerSTEMProfessionalAverages

Trend.Therewasanon-significantdifferenceinscoresbetweengroups.


Trend.Therewasanon-significantdifferenceinscoresbetweengroups.


HighSchoolStudentsDiggingfurtherintotheperceptionofinnovation,weaskedparticipantsinboththepreandthepostsurveywhytheyhadgiventhemselvestheratingstheydid(seeTable33).Inadditiontotalkingaboutnewanduniqueideas(41%),highschoolstudentsalsocitedtheirabilitytobeinnovatorsintheirpersonallife(28%)andintheirschoolworkandprofessionalwork(21%).Table33:SelfPerceptionofInnovationPreSurvey,HighSchoolStudents

Control(n=33)

Treatment(n=32)

Total(n=65)

Newideas/uniqueideas 39% 44% 41%Iaminnovativeinmypersonallife/hobbies 33% 22% 28%Mywork/schoolinvolves/requiresinnovation 27% 16% 21%Ichooseadifferentpath,creativesolution,creativeoutput

9% 31% 20%

Referencetosolutionsand/orproblem 3% 16% 9%Mywork/schooldoesnotallowmetobeaninnovator 9% 6% 8%

Idon’thavetimeathometobeinnovative/Idon’twanttochangethingsinmypersonallife 0% 0% 0%

Miscellaneous 21% 12% 17%


Highschoolstudentsmostoften(41%)referredtonewideaswhentalkingaboutthemselvesasaninnovator.

Iseemyselfasaninnovatorinmanyregards.I’malwayslookingforcoolnewideasandIalwaysfindthemintheunlikeliestofplaces.I’malwayslookingtoimprovesomething,I’malwaysbusywithonethingoranother.ItendtostartprojectsbeforeIfinishothers.Iliketodothingsthathaveneverbeendonebefore.Also,Iliketo“gobigorgohome”.

Yes,IseemyselfasagoodsampleofaninnovatorasItrytobringnewthingseveryday.

IliketointroducenewideasandnewstrategiestoeverythingIworkwith.Whetherit’sschoolorworkorpersonal,Ilovebringinginnewconcepts.

Morethanaquarterofstudents(28%)felttheywereinnovativeintheirpersonallifeandhobbies.

WhenitcomestothingslikeschoolworkIliketodoitinunconventionalways.UnlikemostIdon’tjustsitdownanddomyhomework.Ineedtobedoingsomethingtogetmyselftobetterformideas.WhenitcomestomyhobbieshoweverIamalwaystryingnewthingsandexperimentingwithdifferentways.Forexamplewhenitcomestomybeat-boxingI’malwaystryingtocomeupwithnewtechniquesandsoundstoaddtomyarsenaltomakemeabetterbeat-boxer.

Isometimesthinkofeasier,simplerwaysordoingeverydaythings.ButIrarelycomeupwiththingstodowithschool.

IstronglyagreethatIamaninnovatorinregardstomypersonallifebecauseIhavehadtocomeupwithwaystokeepwhatIholddearinmybusyschedule.Icookalmosteveryday,andIhavebegunrecreatingtraditionallyunhealthyrecipestohealthieralternatives.Myfavoritesofarisacookiecakemadewithgarbanzobeansandmaplesyrupinsteadoftraditionalwhiteflourandsugar.Oneofmycriteriaisthatithastotastegreat,anditalwaysdoes!

Overonefifthofthehighschoolstudents(21%)reportedbeinginnovativeintheirschoolworkandprofessionalwork.

IseemyselfasagoodexampleofaninnovatorbecauseI’malwaysopenandenthusiasticaboutnewideas.Duringmysummerjob,IbecameefficientinthemethodstheytaughtmetoputcasefilesintothecomputerandthenIcreatedmyownmethodthatwasmoreefficientthantheirs.Iseemyselfdoingthisinmylifeaswell.IlovetakinganideaandimprovingonitandIseemyselfdoingthisinmyeverydaylife.IfI’mnotrememberingthematerialtaughtinschool,IfindamoreeffectivewayformetoprocesstheinformationwhetherthisbechangingthewayIstudyorthewayIlearn(avisualvs.verballearner).

Ifindmyselfaveryinnovativeandresourcefulperson.Atwork,(lifeguardingandrestaurantwaitstaff)Itendtolookforsolutionstoproblemsandthinkingofnewwaystodocertaintasksuntilthebestsolutionhasbeenfound.Ialsotendtoincorporatesimilarideasintomypersonallifeandespeciallymyhobbies(tennis,piano).Inschool,Itrytobeasinnovativeaspossible,butIdefinitelyfeelrestrictedduetocurriculumrequirements.


InmyschoolworkIoftenfindcompletelydifferentwaysofdoingassignments,waysthattheteacherneverthoughtofwhencreatingtheproject.

Somestudents(20%)wroteaboutbeinginnovativebychoosingadifferentpathoracreativesolutionintheirlife.

OftentimesIgoaboutadifferentapproachtoaproblemversusadifferentapproachmypeershavechosen.

IbelieveIamaninnovatorbecauseIusuallytakeadifferentapproachthanothersattemptingtocreatemoreefficientwaysofproducingasolution.IseemyselfasaninnovatorbecauseIfocusoncreativityinmylife.Ifollowthisbecausetheworldwouldnotbeuniquewithoutdifferences.

Afewstudentparticipants(9%)talkedaboutinnovationintheirlifebydefiningproblemsandfindingsolutions.

IfeellikeIamaninnovatorbecauseI’malwaystryingtofindnewsolutionsfortheproblemsthatIface,especiallyinmypersonallifeandatschool.SinceIdon’thaveaprofessionallife,myschoolandmyhomearetheonlyplaceswhereIcantrytochange,butI’malwayslookingfornewplaceswhereIcantrytochange.

AslongasIhaveareallygoodfeelforwhattheproblemI’maddressingis,andhavesomesenseofthehistoryoftryingtosolvetheproblem(previousattemptstosolve),IliketothinkI’minnovativeenoughtohelpprogresstowardsasolution,ifnotsolveaproblem.Inmyshopatschool,Biotech,wefacealotofproblemsandareconstantlytryingtosolvethem,likemakingbacteriafluoresceunderUVlightsandsomanyotherexperiments.

Itendto,nomatterthesituation,trytodevelopaneasiersolutionthatcanbenefitmeandothersaroundme.

Others(8%)felttheycouldnotbeinnovativeintheirschoolorprofessionalwork.

Idon’tfeelthatschoolworkgivesmetheopportunitytoinnovateasmuchasidlike.Ihaveabusinesswithmymomandthatgivesmealotofartisticinnovation.AndIamanathleteandmusicianandIcomeupwithnewplaysandcomposemusic,whichletsmeinnovatealot.Myschoolworkisnotvery“innovatable”tobeginwith,myworkisdeskjob,andmyinterestsareopenenoughtocreateapersonalstyletogoaboutthem.

17%oftheparticipantsgaveresponsesthatdidnotfitintoanyofthesecategories. IseemyselfasaninnovatorinmostchallengesIamfacedwith.

Myratingsonwhetheriamainnovatoristhatiamorganizedandtrytounderstandeverything


IratedmyselfthatsinceIbelieveIamnotthebestnortheworstinnovatorandthatitdependsonthesituationatthetime.Ikindofwantedtodolaserworkinthefuture,likebuildingthatkindofstuffinthefuture.Idon’tseemyselfasaninnovator.Iprefertohearotherpeople’sideasandagreeordisagreewiththem.SometimeswiththoseideasIwouldaddmoreontothem.

HighSchoolparticipantswereaskedtorateandexplaintheirself-perceptionasaninnovatoragainaftercompletingtheInnovationChallenge(seeTable34).Table34:SelfPerceptionofInnovationPostSurvey,HighSchoolStudents

Control(n=33)

Treatment(n=32)

Total(n=65)

Iaminnovativeinmypersonallife/hobbies 39% 47% 43%Ichooseadifferentpath,creativesolution,creativeoutput

24% 31% 28%

Newideas/uniqueideas 33% 16% 25%Mywork/schoolinvolves/requiresinnovation 18% 31% 25%Referencetosolutionsand/orproblem 12% 28% 20%Mywork/schooldoesnotallowmetobeaninnovator 18% 16% 17%


Miscellaneous 18% 12% 15%Thehighschoolstudentsmostoften(43%)saidtheyfelttheywereinnovatorsintheirpersonallifeandhobbies.

IgavetheratingsIdidbecauseIusedifferenttechniquestofindasolutioninschool.And,inmypersonallife,Ienjoydrawing,whichiscreativeandinnovative.

IdonotfeelIammuchofaninnovatorwhenitcomestoschoolworkbecausetheassignmentsarewithinrestrictions.TherearesomeprojectswherecreativitydoesallotandIcanbeinnovative.Ontheotherhand,IconsidermyselfasaninnovatorinmypersonallifebecausethosesettingsareopenendedandIinnovatepassionatelyforwhatIamdoing.AtschoolIalwaysworkinawaythatdefiesthenormandreallysurprisesmyteachers.InmypersonallifeIamconstantlytryingoutnewthingsinallmyhobbies,aswellasnewhobbies,usingtrialanderrortodevelopmyownwayofdoingthings.


Morethanaquarterofstudents(28%)identifiedasinnovatorsbecausetheychoseamorecreativesolutionorpath.

I'maverycreativeandimaginativeperson;Itendtowonderinmymindalot(whenthetimeisrightofcourse)yetIfindthatthepeoplearoundmejustknowmeasacreativeperson.IconsidermyselfaninnovatorbecauseI'malwayslookingfornewthingsandtryingtochangewhatisaroundme.Iliketoseethingsinadifferentperspectiveandwonderwhatwouldhappenifthingsweredifferent.

Onequarterofhighschoolstudents(25%)describedthemselvesasinnovatorsbecauseoftheirneworuniqueideas.

IconsidermyselftobeaninnovatorbecauseI’mconstantlythinkingaboutalternateideasandwaysthatcanhelpineverydaylife.IconsidermyselfasaninnovatorbecauseIalwaysliketotakeonnewchallengesandplanorgetanoriginalideaoutofit.Iliketothinkofnewideasandmethodstodothings,andnotjustfollowdirections

Onequarterofhighschoolstudents(25%)believedthemselvestobeinnovatorsintheirschoolwork,orfeltthattheirschoolworkrequiredinnovation.

Inschoolwork,Iamaninnovatorbecausemyprocrastinationisabigproblemforme,soIestimatehowlongaprojectthatIhaveforaweekwouldactuallytakeme.Ifinreality,theprojecttakesmetwodays,Iletmyselftoprocrastinateuntiltherearetwomoredaysleft.InmyschoolworkIalmostalwaysgoaboveandbeyondtohavesomethingnew,better,greater,tohavethebestpossible"thing"possible,youseeeverythingyoudoisareflectionofyousodoitthebesteverydaybecauseputtingsomethingoffonlyhurtsyou.IwilltakeasimpleDNAmodelneededforbiologyandmakea4foottallDNAmodelchemicalandregularthatmovetogether;howeverthepartsarecommonlyusedthings.InmyhobbiesaswellasinanyareaofmylifeIdon'tfollowtherulespersayIdoeverythingoutofthebox,takesoccerforexampleIdon'tjustplayconventionallyIalwaysseenewwaysofdoingsomething.Icreatenewwaysforthingsthatareseeminglyaone-solutionproblem.InmyschoolworkandpersonallifeIhavedefinitelyseenashiftinthewayIgoaboutdoingsomething.I'malwayslookingfordifferentwaystogettoasolutionandevencomingupwithnewerwaystodothings.Someofmycoachesandteachershaveactuallyincorporatedmyideasintothewaytheyteachsomethingtomyteamandinclass.

Onefifthofstudents(20%)madereferencetofindingsolutionstoproblemsintheirlives.

IconsidermyselftobeaninnovatoratmyschoolworkandinmypersonallifebecauseItendtobeaproblemsolvertonoextent,IalwaystrynewsolutionsandnewideasbutalsoIenjoyusingoldsolutionsthatwork,stickingtothebasics.


Itakedifferentapproachestosolutionsthanpeoplenormallydo.AndIamconfidentinmywaytocommunicate.

Ispendalotoftimelookingforthequickest,mosteffectivewaytocompletebothinandoutofschooltasks.Icreatethingsorputideastogethertosolveproblems.

Somehighschoolstudents(17%)felttheirschoolorworkdidnotallowthemtobeinnovative.

InschoolIdon'thavetoomanyopportunitiestobeinnovative.InmymindIcanbecreativeandexpressthings.

Itendtobelessinnovativeinschoolbecauseteacherswillsometimessetlimitsthatwemustconformtoinordertosucceed,whichcanbeverystifling.Inmypersonallife,idon'tlimitmyselftoanythingandIammoreconfidentbeingmyownpersonwhenihavecontroloverhowicanbesuccessful.

Averysmallnumberofstudents(1%)chosenottobeinnovativeintheirpersonallife,orfelttheydidn’thavetheopportunitytobeinnovativeintheirpersonallife.

Inschoolwork,Icomeupwithnewsolutionsandnewwaystodotheworkonmyown,IfindmyownmethodsandIlearnbestthisway.Inmyhobbies,suchasbasketball,I'mworkingonestablishedplays,notdevelopingmyownplays.Ivaluehowthecoachtellsmetodoit.Butinschool,Ifindthereismoreflexibilitytobeinnovativeanddomyownthing.

Andafewhighschoolstudentsgaveresponsesthatdidnotfitintoanyofthesecategories. FeelmorefreetosayanythingIthought. IgavethisratingbecauseIfeellikeIamainnovator Incomparingthechangesinthehighschoolstudents’perceptionsofthemselvesasinnovatorsbetweenthepreandposttests,therewerebroadsimilaritiesbetweencontrolandtreatmentgroups.Onenotabledifferencewasinthenumberofparticipantswhomentionedwaysinwhichtheywereinnovativeintheirownpersonalandprofessionallives.Thepercentageoftreatmentparticipantswiththisresponsemorethandoubled(22%pre/47%post)whilethecontrolgroupshowedonlyaveryslightincreaseinthiscategory(33%pre/39%post).Interestingly,thecategorysayingthatworkorschooldoesnotallowthemtobeaninnovatordoubledfrompretopostforthecontrolparticipants(9%pre/18%post)andmorethandoubledforthetreatmentparticipants(6%pre/16%post).


EarlyCareerSTEMProfessionalsBeforethestartoftheInnovationChallenges,eachparticipantwasaskedtoratehimorherselfasaninnovator,andthenaskedtoexplaintheratinginhisorherownwords(seeTable35).Table35:SelfPerceptionofInnovationPreSurvey,EarlyCareerSTEMProfessionals

Control(n=32)

Treatment(n=37)

Total(n=69)

Mywork/schoolinvolves/requiresinnovation 62% 49% 55%Iaminnovativeinmypersonallife/hobbies 28% 38% 33%Ichooseadifferentpath,creativesolution,creativeoutput 41% 19% 29%

Newideas/uniqueideas 22% 35% 29%Mywork/schooldoesnotallowmetobeaninnovator

0% 30% 16%

Referencetosolutionsand/orproblem 9% 19% 14%Idon’thavetimeathometobeinnovative/Idon’twanttochangethingsinmypersonallife

16% 6% 10%

Miscellaneous 6% 5% 6%MorethanhalfoftheearlycareerSTEMprofessionals(55%)reportedbeinginnovativeintheirschoolworkandprofessionalwork.

Inmyprofession,Iamresponsibleforcreatingandimplementingmanyenjoyablelessonplansandengagingcurriculum.Irarelyreuselesson,thereforehavetocreatenewactivitiesregularly.

Iseemyselfastakingontasksrelatedtoourresearchanddeterminingthebestwaystosolvetheissue.Forexample,weworkedtodevelopacalibrationschemetoproduceaccurateandreliabledataforatmosphericwatermeasurements.

Itrytodonewthingsandchallengemyselfwithdifferentanduniqueprojectsatschool,home,andwork.

Onethirdofadults(33%)felttheywereinnovativeintheirpersonallifeandhobbies.

Idon'tfeelthatI'mcomingupwithnewideasfortheclassroom,justputtingmyownspinonthings.IdothinkthatIcanbemorecreativeinmypersonallifethough.Iliketodoandtryallsortsofnewthings!

Ifeelmuchmorepassionateaboutmyhobbiesandpersonallifeversusmyprofessionallifeand,assuch,ammorewillingtotrynovelthingstokeepthingsinterestingandprogressive.I'moftentoldthatwhenI'minterestedinsomething,Ireallygetintoit.ThisofteninvolvescreatingsomethingoutoftheordinarywithregardtovarioushobbiesIhave.

Someadultparticipants(29%)wroteaboutbeinginnovativebychoosingadifferentpathoracreativesolutionintheirlife.

Student:Ihaveacreativeedgewhenitcomestoschoolengineeringprojects.


InmyprofessionallifeIhavehadtopulltogethermaterialsandgetcreativewithlessonplanning.InmypersonallifeIhavehadtobecreativegatheringequipmentfordifferenthobbies.Iliketoresearchideas-whetherexperiments,Pinterest,art,recipes-thenspinofffromthemtocreatesomethingnew.IalsocreatenewthingswhenIresearchtheinternetandcan'tfindwhatIamlookingfor.

Someadults(29%)referredtonewideaswhentalkingaboutthemselvesasaninnovator. Alwayslookingfornewapproachesonoldthings.

IliketothinkIamaninnovatorbecauseIliketocomeupwithnewideasonhowtodothisandchallengemyselftoit.Ithinktestingoutthingsthatmightnotalwaysworkoutisgoodandcanleadtoprogress.

AfewearlycareerSTEMprofessionals(16%)felttheycouldnotbeinnovativeintheirschoolorprofessionalwork.

Atwork,Imaythrowoutideasoften,butIamnotinapositiontoputmyideasintomotion-itisuptosomeoneelse.IamalsonotoftenputinasituationwhereIneedtocomeupwithsolutionstoproblemsatwork.Inpersonallife,IlovetocomeupwithnewideasforthehobbiesIaminterestedin.Idon'thavemuchroomforflexibilityinmyjobbutIdocomeupwithwaystomakemyjobrolemoreefficientandeffective.Idon'tseemyselfasagreatinnovatorinmyschoolworkbecausetheclassesItakealldon'trequiretoomanydifferentwaystoapproachthesubject.ItsverystraightforwardandIdon'ttrytofindawaytothinkofitdifferentlyoreventrytorelatetoanothersomeothersubjects.

Afewadultparticipants(14%)talkedaboutinnovationintheirlifebydefiningproblemsandfindingsolutions.

Ienjoybecomingcompetentinmyfieldsofinterest.IknowI'matahighlevelwhenIcanprovideinnovativeacceptablesolutionstodifficultsituations.Iseemyselfasaninnovatorwhenbrainstormingideastosolveuserexperienceissuesrelatedtotechnology.Ienjoyengagingindesignthinkingandbouncingideasoffotherpeople.However,Iamalsoafanofthetriedandtruesolution.Ifsomethingexiststhathaspreviouslybeendonebeforeandworkseffectivelythenitshouldbeutilizedratherthanforcinganewinnovation.


Afewadults(10%)felttheydidnothavetimetobeinnovatorsintheirpersonallife,orchosenottobeinnovativeintheirpersonallife.

Iseemyselfascreativewithmyjobandfindingnewwaystoteach.Inmypersonallife,IfeellikeIhavemyroutinesandsometimesgetstuckinthem.

Inbothschoolworkandmyprofessionallife(academic)thereisapremiumplaceduponinnovation,definedasnewideastosolveproblems.Inmypersonalinterests,andmypersonallife,Itendtoavoidtheadoptionofaproblem-solvingattitude.

Andafewadults(6%)gaveresponsesthatdidnotfitintoanyofthesecategories. Youcaneithertalkaboutit...orbeaboutit! Iliketobeproactiveandaforewordthinker.AftercompletingtheInnovationChallengeparticipantswereagainaskedtorateandexplaintheirself-perceptionasaninnovator,toseeifindividualsreportedanychangeintheirresponse(seeTable36).Table36:SelfPerceptionofInnovationPostSurvey,EarlyCareerSTEMProfessionals

Control(n=33)

Treatment(n=32)

Total(n=65)

Mywork/schoolinvolves/requiresinnovation 50% 38% 43%Newideas/uniqueideas 37% 27% 32%Iaminnovativeinmypersonallife/hobbies 34% 27% 30%Mywork/schooldoesnotallowmetobeaninnovator 22% 22% 22%


Ichooseadifferentpath,creativesolution,creativeoutput

25% 8% 16%

Referencetosolutionsand/orproblem 12% 13% 13%Miscellaneous 9% 13% 12%EarlycareerSTEMprofessionalsmostoften(43%)believedthemselvestobeinnovatorsintheirschoolwork,orfeltthattheirschoolworkrequiredinnovation.

Idoresearchinauniversitylab.NearlyeverydayIgettocomeupandtestoutnewideas-thingsthathaveneverbeendonebefore.Atwork,I'madesigner.AlldayIhavetocreativelyandefficientlysolveanythingfromsmallproblems,tolargeinvolvedproblems.Constantlykeepinginmindwhatourcompetitionisdoingandresearchingnewtechnologiesandapplications.Athome,Iamalwaystinkeringwithmyjewelrybusinessorotheraround-the-housetasks

Myworkrequiresmetobeinnovativeeverydaytosolveproblemsthathavenosolutions.


Almostonethirdofadultparticipants(32%)describedthemselvesasinnovatorsbecauseoftheirneworuniqueideas.

Ialwaystrytothinkoutsideofthebox.Iamconstantlythinkingofnewinventionsofnewtechnologyandjottingthemdowninmyphone.

IconsidermyselfsomewhatofaninnovatorbecausewhenIhaveideasIliketestingthemoutanddon'tshootthemdownrightaway.Itryoutnewactivitiesandideasofferedtomeinmylife,butIalsofeelthat,asastudent,Idonothavemuchopportunitytobeamajorinnovatorinworkorpersonallife.

AlmostonethirdofearlycareerSTEMprofessionals(30%)saidtheywereinnovativeintheirpersonallifeandhobbies.

Idon'thavemuchcreativefreedomatwork.Withmyhobbies,Iamabletodevotemoretimetocomingupwithnewideas.Ifeelthatmyhobbiesallowformorecreativeexpression(danceandaerialacrobaticperformance)andinnovationindevelopingnewmovements.Atwork,therearemorerulestofollow.

Almostonequarterofadultparticipants(22%)felttheirschoolorworkdidnotallowthemtobeinnovative.

Atwork,itisdifficulttobeaninnovatorwhensomanyrulestofollowarealreadyinplace:forexample,underaprofessorduringresearchorasanEMT,youdon'treallygettostrikeoutonyourown.Inmypersonallife,however,Iwouldconsidermyselfmuchmoreofaninnovatorthanaverage,asIlovetoexplorenewareasofscience,literature,andoutdooradventures,astheseexperienceshelppreventmefromgettingbored.Idon’tdoanythingmeaningfulatwork.Myworklacksinterestandrecognitionforinnovation.Inmypersonallife,Iknowmycapabilitiesforexperience.

Someadults(19%)chosenottobeinnovativeintheirpersonallife,orfelttheydidn’thavetheopportunitytobeinnovativeintheirpersonallife.

Ialwaystrytoimprovemylessonsplansandimplementnewstrategieswithmystudentseachyear.Inmypersonallife,IknowwhatIlikeandtendtostickwiththosethings.IratedmyselffairlyhighontheinnovatorscaleinmyworklifebecauseIalwaystrytothinkofwaystoimproveprocessesatwork.Inmypersonallifethough,Idon'tthinkItrytocomeupwithnewideasthatoften.Itrynewexperiences,butIdon'tthinkofwaystoimprovethoseexperiences.


ProfessionallyIhavesupervisorsthatImustanswertosoalotofmy"innovationatwork"needstofallwithinparametersthatmeettheirgoals/expectationsofaprogram.Withthatbeingsaid,Iworkinasafespacewherenewideasareacceptedandsupported.IthinkIhavemoremainstream/basichobbies/interestsoutsideofworksoIdon'tthinkI'mallthatinnovativeinmypersonallife.

Someadultparticipants(16%)identifiedasinnovatorsbecausetheychoseamorecreativesolutionorpath.

Irarelyseelimits.IamalwaysbeingtoldbyfriendsandfamilythatIimagineideasthatothersthinkarebrilliant--butneverwouldhavethoughtof.

IconsidermyselfaninnovatorinmyresearchbecauseIusedcreativethinkingtodevelopanewmethodinmyfieldofscience.Inmypersonallife/hobbies,Iamsomewhatofaninnovator.Itakecreative/newishapproachestosomeofmyartsyandathletichobbies.Forexample,Icreatedmyownexercisehobbythatisamixofbellydancing,tai-chiandbalanceboarding.

Afewadults(13%)madereferencetofindingsolutionstoproblemsintheirlives.

IseemyselftobeaninnovatorbecauseI'vebeenabletostepbackonmyparticipationinanorganizationandfigureoutwaysforpublicitytobedramaticallychangedandhelped.MyideasbuildoffsomeotherexistingonesbutithasatwistonitanddefinitelycaterstomydemographicandI'mabletoseetheproblemgettingtheminvested.

IfeelIamaninnovatoratworkbecauseIdonotliketoworkharder,butsmartersoItrytofindnewwaystoworkataproblembutdifferently.

Iworkasanengineerandoftenhavetocomeupwithanewideaforanygivenproblem.IworkwithmanyproblemssoIhavemanyopportunitiestocomeupwithnewsolutions.AthomeIloveworkingonartprojectsandrestorationprojectsthatrequireinnovativesolutionstoproblems.

AndafewearlycareerSTEMprofessionalsgaveresponsesthatdidnotfitintoanyofthesecategories.

MybiggeststrugglewithsayingIamaninnovatorismystrugglewiththeconceptofnewness.Iliveinmyheadandidesleakoutofmybody!IconsideranalmostgraduatinginnovatorbecauseIhaveyettoproduceaproduct,butIamstilllearningto.

IncomparingthechangesintheadultcareerSTEMprofessionals’perceptionsofthemselvesasinnovatorsbetweenthepreandposttests,therewereevenmoresimilaritiesthanwiththehighschoolstudents,withnolargepercentageshiftsfrompretotheposttests.


Asacomplementtothequestionofwhetherindividualsperceivedthemselvestobeinnovators,wealsoaskedparticipantsthefollowingquestion:“Hasyourperceptionofyourselfasaninnovatorchanged,evenalittlebit,duringyourparticipationinthisproject?”Forthoserespondingaffirmatively,weaskedtheopen-endquestion“inwhatways?”todeterminethenatureofthatperceivedchange.Therewerenosignificantdifferencesbetweenthehighschoolgroupstothefirstquestion.Bothgroupsrankedtheirperceptionofthemselvesasinnovatorsaschanged,withanaverageof85%inthecontrolgroupand81%inthetreatmentgroup.TheearlycareerSTEMprofessionalsalsohadnostatisticallysignificantdifferencesbetweengroups,though73%oftheadults(87%inthecontrolgroupand68%inthetreatmentgroup)respondedthatthecoursehadchangedtheirperceptionsofthemselvesasinnovators.SeeTable37belowtoseehowthosestudentswhoreportedachangeinself-perceptionasinnovatorscharacterizedthatchange,withsomenotablepatternsofdifferentiation.Table37:ChangeinPerceptionofSelfasanInnovator,HighSchoolStudents

Control(n=33)

Treatment(n=32)

Total(n=65)

Ihaveimprovedcertainbehaviors 61% 78% 69%Reframingtheprocessofinnovation 33% 34% 34%Collaboration 12% 28% 20%Divergentthought 12% 19% 15%Confidence 12% 9% 11%Findingproblem/solution 6% 9% 8%Leading/following 6% 3% 5%

Ihavethecapacityorinteresttobeaninnovator 9% 6% 8%InnovationisharderthanIthought,it’schallenging,Idon’tfeelasmuchlikeaninnovator

15% 0% 7%

Myperceptionhasnotchanged 6% 6% 6%Miscellaneous 3% 3% 3%Themajorityofhighschoolstudents(69%overall,61%control/78%treatment)reportedimprovementofcertainbehaviors,attitudesorskillsduetoparticipationintheInnovationChallenge.Breakingdownthoseareasofimprovementintomoredetail:Aboutonethirdofhighschoolstudents(34%)felttheirideasabouttheprocessofinnovationhadbeenreframedandimproved.

Ilearnedmoreaboutinnovationinthisproject,andIlearnedtheprocessusedtoproperlyfilterideasintoasolution.

Ithastaughtmetofocusonthecreativesidelonger.

IfeellikeitimprovedmywayofthinkingnowIapplyittoeverything.Onefifthofstudents(20%overall,12%control/28%treatment)saidtheircollaborationskillshadimprovedasaresultofparticipatingintheprogram.

BeforethisprojectIdidn'tthinkitwaspossiblethatIcouldworkinagroupandaccomplishaprojectbutnowIrealizeitwasjustthatI’vejustneverworkedonaprojectwithlike-minded


individualswhoactuallywanttobeworkingontheproject.

Ineverthoughtinnovationcouldbeacollaborativegroupeffort.Normally,Iampersonwhopreferstoworkalone.Thisgroupinnovationprocesshasreallyopenedmyeyestohowpeoplecancollaboratetogetsomethingdone.Thisskillisextremelybeneficialtomylifeespeciallymyacademiccareer.I'lldefinitelybringtheskillsIacquiredfromthisexperiencetocollegenextyear.Irealizedthatinnovationcanhappeninagroup,ratherthanjustinaperson.

Someparticipants(15%)believedtheirabilitytothinkdivergentlyhadimprovedaftertakingpartintheInnovationChallenge.

Icanthinkofmoreideasatonceandmoreangles.Ifeellikemyideasaremuchmorecoherentthantheywerebefore,Ithinkofallthepossibilities.Forexample,beforetheprojectIusedtowalkaroundhavingmyeyesfixedinjustonedirection,nowIlookatalldirectionsandItrytoseenewthingseverywhere.

Someparticipants(11%)felttheirconfidencehadimprovedasaresultofparticipatingintheexperience.

Ifeelmoreconfidentinmyabilitytoinnovate.Ifeelmuchmoreconfidentinmyabilitiesofmymindbecauseoftheideasme,alongwithmyteamcameupwith,alongwiththecapacityofmybraintoworkonmultipleproblemsatonce.IfeelIamaleaderandnotafollowerandwilltakeaction,whichIknewthese,buttheyarerenewedandgivenmenewfoundconfidence.IbelieveIhaveanewprocess.

Afewparticipants(8%)reportedanimprovementinfindingproblemsandsolutions. Icanthinkmoreclearlyaboutproblemsandopportunitiestofixthem. I'mmoreinclinedtothinkaboutproblemsinadifferentwaythanbefore.Afewparticipants(5%)reportedanimprovementinleadingand/orfollowing.

IhavelearntthatIcanletotherscontrolthesituationandthatIcansitbackandlistentoothersequallybrilliantideas.IhavelearnthowtostoppanickingaboutthelittlethingsandseethebiggerpictureandmostimportantlyIhavelearnthowtocommunicatewithothersinordertogetmypointacrossandintegratetheirideastothegrandschemeofthings.MyperceptionofmyselfhaschangedthroughoutthedurationofthisprojectbecausenowIunderstandhowtothinkofusingoldthingsorsystemsinnewexcitingways.AlsoIthinkaboutproblemsolvingandgroupworkdifferently.IhaveseenhowIhavechangedandbecomemoreofaleaderinmygroup.


Outsideofthosehighschoolstudentswhosaidtheyhadimprovedcertainbehaviors:Afewparticipants(8%)felttheirself-perceptionasaninnovatorchangedbecausetheynowfeltliketheycouldbeaninnovator,comparedtotheirperceptionbeforeparticipatingintheInnovationChallenge.

Afterparticipatinginthisprojectandlearningthespecificsofwhatmakessomeoneaninnovator,Icanconfidentlylabelmyselfasaninnovator.

IactuallyfindthatIamevenmoreofaninnovatorthanIthought.Iwasn'tawarehowquickIamatthinkingupaplethoraofsolutions.

Afewparticipants(7%overall,15%control/0%treatment)learnedthatinnovationorbeinganinnovatorismoredifficultthantheyoriginallyperceived.

IhaverealizedthatIamlessofaninnovatorthatIthoughtIwas,oratleastthatmostofmyinnovationsareshort-termandrelativelyeasytoimplement.MyperceptionhaschangedbecausethisprojectwaschallengingattimesandIrealizedhowhardIactuallycanworkandhowmuchIactuallycaninnovatewhenIputmymindintoit.BeforeIdidthingsveryimpetuously,notreallythinkingthingsthrough.NowIknowtheprocessforinnovationandcansystematicallyworkthroughproblems.

Afewhighschoolstudents(6%)reportedthattheirperceptionofthemselvesasaninnovatordidnotchangeasaresultofparticipatingintheInnovationChallenge.

No,myperceptionofmyselfasaninnovatordidnotchangeduringmyparticipationinthisproject.Myperceptiondidn’treallychange

Andafewparticipantsgaveresponsesthatdidnotfitintoanyofthesecategories.

ItshowedmetoseeandrealizehowdisorganizedWorcesteris

IfeltasifIwaslettingthepanelreallyinterestedintomygroup'sinnovationsotheywouldwanttoseeitinthefuture.

EarlycareerSTEMcareerparticipantswerealsoaskedinwhatwaystheirperceptionofselfasaninnovatorhadchangedasaresultofparticipatingintheproject(seeTable38).ThemajorityofearlycareerSTEMprofessionalparticipantsmentionedanimprovementofcertainskillsafterparticipatingintheInnovationChallenge(54%).


Table38:ChangeinPerceptionofSelfasanInnovator,EarlyCareerSTEMProfessionals

Control(n=32)

Treatment(n=37)

Total(n=69)

Ihaveimprovedcertainbehaviors 59% 49% 54%Collaboration 19% 24% 22%Reframingtheprocessofinnovation 22% 19% 20%Confidence 16% 5% 10%Divergentthought 9% 3% 6%Leading/following 6% 3% 4%Findingproblem/solution 0% 0% 0%

InnovationisharderthanIthought/Idon’tfeelasmuchlikeaninnovator

6% 22% 14%

Myperceptionhasnotchanged 12% 16% 14%Ihavethecapacity/interesttobeaninnovator 9% 11% 10%Miscellaneous 6% 3% 4%ThemajorityofearlycareerSTEMprofessionals(54%overall,59%control/49%treatment)reportedimprovementofcertainbehaviors,attitudesorskillsduetoparticipationintheInnovationChallenge.Webreakdownthoseareasofimprovementintomoredetailbelow.OveronefifthofearlycareerSTEMprofessionals(22%)saidtheircollaborationskillshadimprovedasaresultofparticipatingintheprogram.

Thisprojectgavemeadeeperunderstandingofwhatitlookedliketopurposefullyinnovateasagroup.IthinkIhaveabetterunderstandingofthecollaborationprocessandquantitativestepsthatareinvolved.Inthepast,Iexperiencedgeneratingnewideasinagroupsetting.TheChallengemadeitvery/moreclearthatinnovationbyiteration(offotherpeople'sideas)generatesnewideasmuchfasterthanworkinginisolation.ThisissomethingI'veexperiencedbefore,butdidn'treallyrecognizethefullstrengthofworkingingroupsinthismanneruntiltheChallenge.

Onefifthofadultparticipants(20%)felttheirideasabouttheprocessofinnovationhadbeenreframedandimproved.

Ireallyappreciatedhowthoroughlywewentthroughtheinnovationprocess.Irecognizedtheflowofthehugeamountofideasgeneratedduringbrainstorming,tohoninginonasingleaspectofaproblem,andthendevelopingamultifacetedapproach,butIhadneverreallypinned/brokendownthoseseparatesteps.Ibuckedattheideaofastructuredformulaforinnovationatfirst,butthencametoacceptitasIcouldseeitworkinginthegroup.Ifeelwaymoreopentousingdifferentapproachestodevelopingideasandsolutions,especiallyusingmoreofanartisticapproachforwhatmayseemmorelikeascientificproblem-theuseofmusic,doodling,movement,etc.Ihavelearnedalotmoreabouttheprocessofinnovation,aswellasthewaythatIrespondtoit.


Ihavelearnedhowtobeinnovativewithagroup.Someparticipants(10%overall,16%control,5%treatment)felttheirconfidencehadimprovedasaresultofparticipatingintheexperience.

I'mmoreconfidentinmythinking,honestly.IhadapositiveexperienceduringthewaterchallengeandwasproudthatsomeoftheideasIhadwereincorporatedintoourfinalproject.Enteringtheproject,Iwasn'tsureifI'dbeabletocontributemuchtotheinnovationprocesssinceIfiguredthatI'dbeoneoftheyoungestmembersintheproject.However,IwasexcitedtoseethatIcouldcontributeoriginal,innovativeideasbasedonmyownexperiencesinlife,soI'vegainedmoreconfidenceinmyselfasaninnovator.

Afewadultparticipants(6%)believedtheirabilitytothinkdivergentlyhadimprovedaftertakingpartintheInnovationChallenge.

Irealizethatyoudon’tneedtobethesmartestpersonintheroomortheloudesttobeinnovative.Lookingatdifferentideasevenifyouthinktheymaybecrazyatfirst(thehumanfecescompositingforexample)couldhavepotentialifthereissomeideastomakeitmainstreamenoughtohavelargergroupgainacceptance.It’sthebalanceofideasthathelpthewildideasgetclosertousablemarketableproducts.IsupposeIhaverealizedthatIcanbemorecreativeandcomeupwithmoreideasthanwhatIthoughtIcould.

Afewparticipants(4%)reportedanimprovementinleadingand/orfollowing.

Mostlylearningtoshareresponsibilitiesandarticulation.BeforeIdidn'tbelieveIwasinnovativebutintheprocessIfeltvalidatedbymyteambecausemyleadershipskillswereabletoshinebutalsomyactualengineeringskills.BecausemyteamchosetheideasthatIhelpeddevelopedIbelievethatIamsomewhatofaninnovator.Ialsofeellikewewereabletobuildoffeachother'sideastoo.

OutsideofthoseearlycareerSTEMprofessionalswhosaidtheyhadimprovedcertainskillsorattitudes:Someadultparticipants(14%)learnedthatinnovationorbeinganinnovatorismoredifficultthantheyoriginallyperceived. IsupposeIfeellessinnovative;Ihadmomentsofdesiringnoinnovationwhenfrustrated.

Beforetheproject,innovationwasabroadertermthatincludedanyeventthatoneusedtheircreativitytoimproveorsolveaproblem.Aftertheproject,innovationequalspressure.Thepressuretodevelopacompletelynewideathathasthepotentialtodowellinthemarketingworld.Thepressuretodevelopanideathatislifechangingtomorethanjustoneperson.Tome,innovationnowseemslikesomethingI'malmostincapableofachieving.


ItmademefeellikeIwaslessofaninnovatorbecauseitseemedmyexistingperceptionsoftheproblemathandkeptallmyideasinaboxrelatedtohowpracticaltheywouldbeinreallife.

Someadults(14%)reportedthattheirperceptionofthemselvesasaninnovatordidnotchangeasaresultofparticipatingintheInnovationChallenge.

Ithasn't.Idon'tfeelIchangemywaytoseethingsduringthisproject.Iappeartohavelesspatienceforpoorlythoughtoutideas.

Afewadultparticipants(10%)felttheirself-perceptionasaninnovatorchangedbecausetheynowfeltliketheycouldbeaninnovator,comparedtotheirperceptionbeforeparticipatingintheInnovationChallenge.Andafewparticipantsgaveresponsesthatdidnotfitintoanyofthesecategories.

UsingChallengeSkillsintheFutureThelastquestionofthetransferabilityinstrumentfocusedonthefutureandparticipant’sabilitytoimplementnewskillsandknowledge(seeTable39andTable40).Participantswereasked,“WhatwastheonethingyouexperiencedintheChallengethatyouthinkwillbemosthelpfultoyouinthefuture,andwhy?”Theskillmostoftencitedbythehighschoolstudentcohortwascollaborationandteamwork(21%).TheskillmostoftencitedbytheearlycareerSTEMprofessionalscohortwasalsocollaborationandteamwork(27%). Table39:WhatWillbeMostHelpfulintheFuture,HighSchoolStudents

Control(n=33)

Treatment(n=32)

Total(n=65)

Collaborationskills 18% 28% 21%Process-basedinnovationskills 15% 25% 20%Divergentthinkingskills 0% 9% 5%Contentknowledge 6% 0% 3%Miscellaneous 3% 0% 1%Morethanonefifthofhighschoolstudents(21%)saidcollaborationskillswouldbethemostusefulthingtheylearnedduringtheInnovationChallenge.

Howtoproblemsolvewithgroupsbecauseitoccursineveryone'sdailylivesanditisahardtaskwithsomanydifferentopinionsandpointofviews.Teamworkskillsandnevershuttingdownsomeone'sidea,nomatterhowstrangeitis.Knowinghowtoworkbetterinagroupandeffectivelycommunicate.IthinkthiswillbemosthelpfulbecauseItendtoshyawayfromsharingmyopinion,butIlearnedbetter.


Onefifthofhighschoolparticipants(20%)felttheprocess-basedinnovationskillswouldbethemostusefulthingtheywouldtakeawayfromtheChallenge.

Learningabouthowinnovationisaprocess,willhelpmefollowalongthatprocesswhenthetimecomesforprojectsandsuch.

Thefunnelingofpossiblesolutionstoanexactsolution. Brainstormingideasandideadevelopmentbecauseitimportanttotrynewthings.Afewstudents(5%)mentioneddivergentthinkingasaskilltheylearnedduringtheInnovationChallengethatwouldbehelpfultotheminthefuture.

Ithinkthinkingfromadifferentperspectivebecauseitopensupawholenewworldofideas.Afewstudents(1%)gainedcontentknowledgeduringtheInnovationChallengethatwouldbehelpfultotheminthefuture. Howtravelcanbeimproved.Andafewhighschoolstudentsgaveresponsesthatdidnotitintoanyofthesecategories. Thatwhenforcedtoworkwithidiots,nothinggetsdone.Table40:WhatWillbeMostHelpfulintheFuture,EarlyCareerSTEMProfessionals

Control(n=32)

Treatment(n=37)

Total(n=69)

Collaborationskills 28% 27% 27%Process-basedinnovationskills 28% 16% 22%Contentknowledge 9% 11% 10%Divergentthinkingskills 0% 0% 0%

Note:Visitorscouldprovidemorethanoneresponsetothisitemsothecolumnpercentagestotalmorethan100%MorethanonequarterofearlycareerSTEMprofessionals(27%)saidcollaborationskillswouldbethemostusefulthingtheylearnedduringtheInnovationChallenge(seeTable40).

UnifyingthoughtsandteammembersfromvaryingbackgroundsaroundatechnicalsolutionforaproblemthatdeeplyaffectsallindividualslivinginSouthernCalifornia.

Collaborationinaninterdisciplinaryteam.

Beingabletoworkwithagroupandgetthetaskdoneunderpressure.


Morethanonefifthofadultparticipants(22%)felttheprocess-basedinnovationskillswouldbethemostusefulthingtheywouldtakeawayfromthechallenge.

Writingthebusinessplan-IhaveasmallsidebusinessthatI'mworkingongrowing,soI'llneedto. Theideatoiterate,design,iterate,anddesignagain.

Brainstormingactivities.Someadults(10%)gainedcontentknowledgeduringtheInnovationChallengethatwouldbehelpfultotheminthefuture.

Ilearnedthatwaterisapublicgood,notacommodity.IthinkthatwasveryessentialtoapproachthewaterproblemthatIdidn'tknowuntilWeek3.Thatledmetothinkaboutenergy,whichisacommodity,notapublicgood,andhowthatchangestheeconomicenvironmentbetweenthetwodifferententities.Outsideofthetechnicalbackground,thisenlightenedmetolookintootheraspectsthatcanbeverycriticaltomakingnewservices/productssuccessful.

Ihaveabetterunderstandingofwatershedsandhowtohelppreservethem. Knowledgeofhowthewatersystemworks.Unlikethehighschoolstudents,noearlycareerSTEMprofessionalsmentioneddivergentthinkingasaskilltheylearnedduringtheInnovationChallengethatwouldbehelpfultotheminthefuture.


Hypothesis2Findings:CollaborationBehaviors Hypothesis2:Arts-basedinnovationtraining,comparedtotraditionalinnovationtraining,increasesindividualcollaborativebehaviorswithinateamcontext.Collaborationplaysacentralroleinvirtuallyallinnovationprocesses,whichisreflectedintheteam-basednatureoftheArtofScienceLearninginnovationcurriculum.Hypothesis2tookacloserlookattheimpactofarts-basedlearningonthecollaborativebehaviorsofinnovationteamsfacedwithincreasinglycomplexanddemandingcollaborativetasksoverafive-weekperiod.Collaborationwasresearchedbyobservingelevencollaborativebehaviorswithintheteams,aswellasbyaskingteamstoreflectonhowwelltheyworkedtogether,withoverlapbetweenitemsthatformedthebasisforobservationalandparticipantreports.Fortheobservedbehaviors,datacollectorsobservedteamswhentheywereworkingtogetherinanycapacity,suchasactivities,brainstorming,orontheirproblemsandsolutions.Asparticipantswerewearingnametags,datacollectorsspecificallyrecordedwhichoftheindividualstheysawengagingineachoftheelevenbehaviorsduringagivenobservationperiod,whichusuallywasbetween10and30minutes.Ifanyactivitylastedlongerthan30minutes,halfwaythroughobserverswouldswitchtoanotherteamsotobetterbalancetheamountofcollaborativeactivitythatwasobservedamongthevariousteams.Thetotalnumberofindividualswhoengagedinthebehaviorswasthentotaledforeachteampereachobservedblockofactivity;theseblockswerethenaddedtogethertoproducedataontotalnumberofobservedinstancesofeachactivityforeachoftheteamsthatday.Thislastnumberiswhatisrepresentedintheanalysesbelow.Fortheself-reportedteamratingmeasuresincludedinthissection,eachindividualfilledoutashortsurveyattheendofeachweekthatincludedratingsforthesamecollaborativebehaviorsbeingobserved.Fortheself-reportteamratingmeasures,individualsratedtheirteam(asopposedtoparticularindividualsontheirteam)oneachoftheelevenitems.Inthismanner,theobservationsandsurveyswereintendedtoprovidemultiplewaystomeasurethesamevariablesandacomparisoncouldbemadebetweenwhattheresearchersobservedandwhattheteammembersperceived.AppendicesCandDincludetheobservationalandself-reportedteamcollaborationratingmeasures,respectively.

ObservedCollaborationInordertodeterminewhetherornottherewasachangeinspecificcollaborativebehaviors,ananalysiswasruntodeterminewhethereachindividualshowedthatcollaborativebehaviormore,lessorthesameattheendoftheresearchperiodastheydidatthebeginning.Then,thepercentagebreakdownofthethreecategorieswascomparedforthetreatmentandcontrolgroupsindependently.Inotherwords,theanalysistoldwhethertherewasadifferenceforeithercohortfromthebeginningtotheendoftheirparticipation.Forthehighschoolstudentsthereweresignificantincreasesinpositivecollaborativebehaviorsinboththecontrol(6of11measures)andtreatment(5of11measures)conditions(seeTable41).WhenlookingattheearlycareerSTEMprofessionals,itwasmoreone-sided:thecontrolconditionshowedasignificantincreaseforonly1ofthe11positivecollaborativebehaviors,whilethetreatmentconditionshowedasignificantincreasefor7ofthe11.


Table41:SummaryTableofObservedCollaborationandOtherInnovationProcessesDifferences(Pre/PostObservationalDifferenceScores) HighSchoolStudents EarlyCareerSTEMProfessionals

ObservationalCategory

StatisticallySignificantGroupDifferences?

GroupwithBetterPerformance(Pre/Post)



Sharesleadership Yes Control Yes TreatmentActivefollower No - Yes TreatmentEmotionallyintelligentbehavior

Yes Treatment Yes Treatment

Empathiclistening Yes Treatment Yes TreatmentMutualrespect No - Yes Treatment,ControlTrustinmovingtowardsolution Yes Treatment,Control Yes Treatment

Transparentincommunication

Yes Control Yes Treatment

Abilitytodisagreeproductively Yes Treatment No -

Definingacommonpurpose

YesControl No -

Creatingcultureofmutualaccountability

Yes Control No -

Productivelymanagesdisruption

Yes Treatment,Control No -

Self-ReportedCollaborationFortheself-reportedteamcollaborationratingmeasures,individualsratedthemselvesonfifteencollaborativebehaviors;thiswasdoneduringapre-testinthefirstsession,andduringapost-testinthelastsession.Adifferencescorewascalculatedbysubtractingthepre-testratingorscoreonanitemfromthepost-testscoreonthatsameitem.Ifoneorbothoftheitemswasmissing,thenthatindividualwasnotincludedinthecomparison.Twostatisticallysignificantdifferencesemergedforthehighschoolstudents;thecontrolgroupwasmorelikelytoreportapositivechangefrompre-testtopost-testinmutualrespectandalsotrust(seeTable42).InlookingattheearlycareerSTEMprofessionalstherewerenostatisticallysignificantdifferencesinthepre-testpost-testchangescores.Thismeansthattheadultsdidnotperceiveanynotabledifferencesfromthebeginningofparticipationtotheendinthevariouscollaborationscores,eitherthoseinthecontrolortreatmentconditions.


Table42:SummaryTableofSelf-ReportedTeamcollaborationRatingsandOtherInnovationProcessesDifferences(Pre/PostSelf-ReportedDifferenceScores) HighSchoolStudents EarlyCareerSTEMProfessionals

Self-ReportCategory





Sharesleadership No - No -Activefollower No - No -Emotionallyintelligentbehavior

No - No -

Empathiclistening No - No -Mutualrespect Yes Control No -Trustinmovingtowardsolution Yes Control No -

Transparentincommunication

No - No -

Abilitytodisagreeproductively

No - No -

Definingacommonpurpose

No - No -

Creatingacultureofmutualaccountability

No - No -


No - No -

Successfullycompletedtask

No - No -

RelationshipBetweenObservedandSelf-ReportedCollaborationItisinterestingthattheobservedcollaborationmeasuresshowedamuchlargerimpactthantheself-reportedteamcollaborationratingmeasures.Althoughneithercontrolnortreatmentgroupsratedthemselvesashavingimprovedonanyoftheself-reportedteamratingmeasures,observersrecordedthetreatmentgroupsashavingengagedinmanyofthesebehaviorssignificantlymorebythelastsession.ThedifferencewasparticularlyevidentamongtheearlycareerSTEMprofessionals,butthehighschoolstudentsalsoshowedamuchhigherrateofsignificanceforobservedcollaborativebehaviorscomparedtotheirself-reportedteamratingbehaviors.Thesefindingssuggestthatsimplyaskingpeopleabouttheircollaborativebehaviorsmaynotbeenoughtoaccuratelyportraythenatureandextentofcollaborationoccurringduringgrouptasks;andthatthirdpartyobservationisimportanttounderstandingtheoccurrenceofthesebehaviors.

HighSchoolStudentObservedBehaviorFindingsThetablesinthefirstpartofthissection(Table43throughTable53)showtheobservedbehaviorratingsbytheresearchersfromR2(week2)throughR5(week5)andincludetotalbehaviorsperteampersession;seeAppendixCfortheindividualbehaviorsstudied.Atableisincludedforeachoftheelevenbehaviorsobservedbytheresearcher,breakingdownthenumberoftimesthisparticular


behaviorwasobservedforeachteamduringeachweek.Theoverallsumofbehaviorsobservedineachweekisincludedatthebottomofthetable.Note:R1(week1)observationaldatawerecollectedforthetreatmentgroup,butnotforthecontrolgroup.SincetheobservationaldataforhighschoolstudentsinR1areincomplete,R1hasbeenexcludedfromthetablesandfiguresbelow,aswellasallrelatedstatisticalanalyses.Inordertocomparethegroupsoneachbehavior,statisticaltestswererunonthedifferencesbetweenthefrequencyandpatternsofoccurrencethesebehaviorswereobservedincontrolandtreatmentovertheR2toR5period.Iftherewasastatisticallysignificantdifferencebetweenthecontrolandtreatmentgroups,alinegraphwasincludedbelowthetable.Tableswithoutaccompanyinglinegraphsshowednostatisticallysignificantdifferencesbetweencontrolandtreatment.BasedontheR2toR5totals,behaviorsmostcommonlyobservedwerebeinganactivefollower(seeTable44)andempathiclistening(seeTable46).Thesewerefollowedbyemotionallyintelligentbehaviors(seeTable45)andbeingtransparentincommunications(seeTable49).Thenexthighesttotalswerefortheabilitytodisagreeproductively(seeTable50),beingmutuallyrespectful(seeTable47),creatingacultureofmutualaccountability(seeTable52),andmovingtowardsasolution(seeTable48).Asmentionedabove,therewasacomparisonoftreatmentandcontrolgroupsinthefrequencyandpatternoftheelevenbehaviorsfromR2throughR5.Cross-tabulationswererunlookingatthepercentageofindividualsinthecontrolandtreatmentgroupsengagingineachspecificbehavior,acrossweeks2through5.Thistypeofanalysiswaschosensinceitallowedadirectcomparisonbetweenthetwogroups,toseewhetherornotbeinginonegroupincreasedordecreasedthelikelihoodofengaginginthatparticularbehavior.Theeightthatshowedastatisticallysignificantdifferenceincludebehaviorsrelatedtosharingleadership,trustinmovingtowardsasolution,beingtransparentincommunication,emotionallyintelligentbehavior,theabilitytodisagreeproductively,definingacommonpurpose,creatingacultureofmutualaccountability,andproductivelymanagingdisruption.

ComparisonBetweenControlandTreatmentbyBehaviorSharesLeadershipTheoccurrenceofthebehaviorwashighestinR3,followedbyR2andR5,thenR4.Therewasastatisticallysignificantdifferenceinthefrequencyandpatternofthesebehaviorsovertimeincomparingcontrolandtreatmentgroups.WhilethecontrolgrouphadmoreoccurrencesthisbehaviorinR2comparedtothetreatmentgroup,andtheywereroughlyeveninR4,thetreatmentgrouphadroughlytwiceasmanyoccurrencesinR3andR5,comparedtothecontrolgroup.


Table43:SumofSharesLeadershipBehaviorsObserved,HighSchoolStudents Team R2 R3 R4 R51(treatment) 6 10 3 112(treatment) 10 15 6 63(treatment) 3 8 5 94(treatment) 3 10 4 105(control) 10 7 5 86(control) 10 7 6 27(control) 8 1 3 68(control) 2 9 2 2Overallsum 52 67 34 54Figure2:SharesLeadership,SumofObservedBehaviorsinGroupsOverTime,HighSchoolStudents

ActivefollowerThebehaviorofactivefollowingwasoneofthemorecommonbehaviorsrecorded,withtotalsofover200instancesobservedforeachweekfromR2throughR5;thehighestfrequencieswereinR2andR4.Therewasnotastatisticallysignificantdifferenceinthefrequencyandpatternofthesebehaviorsovertimeincomparingcontrolandtreatmentgroups.Table44:SumofActiveFollowerBehaviorsObserved,HighSchoolStudents Team R2 R3 R4 R51(treatment) 19 26 37 312(treatment) 32 31 24 403(treatment) 32 33 31 294(treatment) 32 28 26 355(control) 48 34 35 366(control) 39 27 27 337(control) 36 28 25 348(control) 45 24 20 35Overallsum 283 231 225 273

22

43

18

36

30

2416

18

R2 R3 R4 R5

Treatment Control


EmotionallyintelligentbehaviorTherewereahighernumberofoccurrencesinR3andR5,comparedtoR2andR4.Therewasastatisticallysignificantdifferenceforthefrequencyandpatternofemotionallyintelligentbehaviorobservedincomparingthecontrolandtreatmentgroups.WhilethecontrolgrouphadaslightlyhighernumberofoccurrencesinR2,thetreatmentgrouphadahighernumberthanthecontrolgroupforR3,R4andR5.Table45:SumofEmotionallyIntelligentBehaviorsObserved,HighSchoolStudents Team R2 R3 R4 R51(treatment) 10 20 16 222(treatment) 13 26 14 193(treatment) 10 12 16 234(treatment) 8 17 12 235(control) 16 22 20 206(control) 16 13 11 177(control) 8 14 7 98(control) 13 12 5 7Overallsum 94 136 101 140Figure3:EmotionallyIntelligentBehavior,SumofObservedBehaviorsinGroupsOverTime,HighSchoolStudents

41

7558

87

53

61

43

53

R2 R3 R4 R5

Treatment Control


EmpathiclisteningFromR2throughR5therewasasteadyincreaseinempathiclisteningacrossallfoursessions.Therewasnotastatisticallysignificantdifferenceinthefrequencyandpatternofthesebehaviorsovertimeincomparingcontrolandtreatmentgroups.Table46:SumofEmpathicListeningBehaviorsObserved,HighSchoolStudentsTeam R2 R3 R4 R51(treatment) 12 23 35 302(treatment) 13 26 24 323(treatment) 23 20 24 334(treatment) 14 25 25 325(control) 35 24 31 376(control) 26 21 27 317(control) 23 21 23 298(control) 31 22 18 26Overallsum 177 182 207 250MutualrespectThetotalinstancesofmutualrespectobservedweresimilaracrossalleightteamsinR2andR3,dippedinR4andthenshotupsubstantiallyinR5.Therewasnotastatisticallysignificantdifferenceinthefrequencyandpatternofthesebehaviorsovertimeincomparingcontrolandtreatmentgroups.Table47:SumofMutualRespectBehaviorsObserved,HighSchoolStudentsTeam R2 R3 R4 R51(treatment) 5 11 0 192(treatment) 6 14 7 173(treatment) 18 8 9 174(treatment) 3 11 0 175(control) 9 6 3 126(control) 6 4 1 177(control) 7 4 0 128(control) 7 7 3 6Overallsum 61 65 23 117


TrustinmovingtowardsolutionThebehaviorsobservedfortrustinmovingtowardasolutionincreasedsubstantiallyfromR2toR5.Therewasastatisticallysignificantdifferenceinthefrequencyandpatternofthesebehaviorsbetweencontrolandtreatmentgroups.Thetreatmentgroupshowedahighernumberofthesebehaviorseachofthefourweeks,withthelargestdifferencesbeinginR3andR4.Table48:SumofTrustinMovingTowardSolutionBehaviorsObserved,HighSchoolStudentsTeam R2 R3 R4 R51(treatment) 0 14 5 212(treatment) 4 10 10 173(treatment) 4 11 12 164(treatment) 0 0 6 145(control) 6 7 8 226(control) 0 6 4 187(control) 6 3 4 98(control) 2 3 4 13Overallsum 22 54 53 130Figure4:TrustinMovingTowardSolution,SumofObservedBehaviorsinGroupsOverTime,HighSchoolStudents

14

35 33

68

819 20

62

R2 R3 R4 R5

Treatment Control


TransparentincommunicationForthisbehavior,therewasasteadyincreasefromR2throughR5.Therewasastatisticallysignificantdifferencebetweencontrolandtreatmentgroupsinthefrequencyandpatternofthesebehaviors.ThenumbersforeachgroupwerealmostidenticalforR2andR5,whilethetreatmentgrouphadmoreoccurrencesinR3andR4.Table49:SumofTransparentinCommunicationBehaviorsObserved,HighSchoolStudents Team R2 R3 R4 R51(treatment) 4 13 11 142(treatment) 9 16 16 173(treatment) 6 7 15 204(treatment) 4 10 13 165(control) 7 8 12 186(control) 1 8 10 157(control) 9 5 6 168(control) 5 7 7 18Overallsum 45 74 90 134Figure5:TransparentinCommunication,SumofObservedBehaviorsinGroupsOverTime,HighSchoolStudents

23

4655

67

2228

35

67

R2 R3 R4 R5

Treatment Control


AbilitytodisagreeproductivelyParticipants’abilitytodisagreeproductivelyevolvedinsomewhatofabellcurve,withthelargernumberofoccurrencesinR3andR4,andfewerinR2andR5.Therewasastatisticallysignificantdifferencebetweencontrolandtreatmentgroupsinthefrequencyandpatternofthesebehaviors,withthetreatmentgrouphavingmoreoccurrencescomparedtothecontrolgroupinthreeofthefoursessions.Sinceweonlyobservedproductivedisagreement,itisnotpossibletoinferwhethertherewasadifferenceinnon-productivedisagreement.Table50:SumofAbilitytoDisagreeProductivelyBehaviorsObserved,HighSchoolStudents Team R2 R3 R4 R51(treatment) 3 15 9 82(treatment) 11 20 10 93(treatment) 6 3 16 104(treatment) 8 14 17 65(control) 14 15 16 36(control) 7 8 9 47(control) 6 0 4 58(control) 6 9 5 5Overallsum 61 84 86 50Figure6:AbilitytoDisagreeProductively,SumofObservedBehaviorsinGroupsOverTime,HighSchoolStudents

28

5252

165

32

32 34

159

R2 R3 R4 R5

Treatment Control


DefiningacommonpurposeThehighestlevelofoccurrencewasinR5,followedbyR3,R4andR2.Therewasstatisticallysignificantdifferenceinthefrequencyandpatternofthisbehaviorbetweencontrolandtreatmentgroups.WhileoccurrenceswererelativelysimilarinR2andR4,therewasamuchhigheroccurrenceofthisbehaviorforthetreatmentgroupinR3andR5.Table51:SumofDefiningaCommonPurposeBehaviorsObserved,HighSchoolStudentsTeam R2 R3 R4 R51(treatment) 2 10 7 152(treatment) 10 8 6 93(treatment) 8 10 9 104(treatment) 1 5 3 105(control) 7 7 3 106(control) 7 8 5 57(control) 6 1 8 98(control) 0 6 5 4Overallsum 41 55 46 72Figure7:DefiningaCommonPurpose,SumofObservedBehaviorsinGroupsOverTime,HighSchoolStudents

21

3325

44

20 22 21

28

R2 R3 R4 R5

Treatment Control


CreatingcultureofmutualaccountabilityThebehaviorofcreatingacultureofmutualaccountabilityhadoneofthelargestincreasesfromR2toR5.Therewasastatisticallysignificantdifferenceinthefrequencyandpatternofthebehaviorbetweencontrolandtreatmentgroups.Thisdifference,however,wasdrivenbyasingleweek(R4)wherethecontrolgrouphadahigherdegreeofthisbehaviorcomparedtothetreatmentgroup.ThetreatmentgrouphadahigherdegreeofthisbehaviorcomparedtothecontrolgroupinR2,R3andR5.Table52:SumofCreatingCultureofMutualAccountabilityBehaviorsObserved,HighSchoolStudentsTeam R2 R3 R4 R51(treatment) 0 6 12 252(treatment) 0 9 0 163(treatment) 7 9 11 164(treatment) 0 4 4 115(control) 4 10 22 196(control) 1 6 7 227(control) 1 3 5 88(control) 0 8 3 14Overallsum 13 55 64 131Figure8:CreatingaCultureofMutualAccountability,SumofObservedBehaviorsinGroupsOverTime,HighSchoolStudents

7

28

27

68

627

3763

R2 R3 R4 R5

Treatment Control


ProductivelymanagesdisruptionThehighestnumberofoccurrenceswasobservedinR2andR4,withlowernumbersforR3andR5.Therewasastatisticallysignificantdifferenceinthefrequencyandpatternofthebehaviorbetweencontrolandtreatmentgroups,thewithcontrolgroupshowingthestrongerperformance.ThetreatmentgrouphadahighernumberofoccurrencesinR2,duetoalargenumberofoccurrencesinoneparticulartreatmentteam,whilethecontrolgrouphadahighernumberofoccurrencesinR3toR5.Table53:SumofProductivelyManagesDisruptionBehaviorsObserved,HighSchoolStudents Team R2 R3 R4 R51(treatment) 0 1 7 32(treatment) 13 6 3 63(treatment) 5 4 9 14(treatment) 5 1 7 25(control) 5 5 10 86(control) 3 5 9 47(control) 8 2 6 08(control) 2 2 5 1Overallsum 41 26 56 25Figure9:ProductivelyManagesDisruption,SumofObservedBehaviorsinGroupsOverTime,HighSchoolStudents

ComparisonBetweenControlandTreatmentintheFinalSessionLookingatteams’observedbehaviorsoverthecourseoftheprojectallowedforanunderstandingofhowtheteamsprogressedduringtheweeklysessions.Thefinalsession,R5representstheculminationoftheproject’sinnovationtraining,andlookingspecificallyattheR5outcomesgivesustheabilitytostudythewaysinwhichthecollaborativebehaviorsofthecontrolandtreatmentgroupswereimpactedbythe20hoursoftraining.Inordertocomparethecomparativeimpactsoffullthefull20-hourintervention,astatisticaltestwasrun,lookingatthedifferenceinthefrequencyofeachoftheelevenbehaviorsinR5,betweenthecontrolandtreatmentgroups.

23

12

26

12

1814

30

13

R2 R3 R4 R5

Treatment Control


Oftheelevencomparisons,thefollowingtableshowsfivebehaviorswithstatisticallysignificantdifferences:sharesleadership,emotionallyintelligentbehaviors,mutualrespect,abilitytodisagreeproductively,anddefiningacommonpurpose(seeTable54).Alloftheseshowedahigherlevelofoccurrenceforthetreatmentgroup.BothcontrolandtreatmentteamsspentR5finalizingtheirfinalpresentationsandbusinesscasesrelatedtothechallenge,sotherewerealotofopportunitiesforcollaborativebehaviors.Thetreatmentgroupengagedinthesetypesofbehaviorssignificantlymorethanthetreatmentgroup,suggestingthatthetreatmentgroupwasactingmorecollaborativelyastheyfiguredoutthefinalpiecesanddetailsoftheirpresentations.Table54:SummaryTableofObservedCollaborationandOtherInnovationProcessesDifferences,OnlyforWeek5(R5)(Pre/PostObservationalDifferenceScores) HighSchoolStudentsObservationalCategory



Sharesleadership Yes TreatmentActivefollower No -Emotionallyintelligentbehavior

Yes Treatment

Empathiclistening No -Mutualrespect Yes TreatmentTrustinmovingtowardsolution

No -

Transparentincommunication No -


Yes Treatment

Definingacommonpurpose Yes Treatment


No -


No -


Self-ReportedTeamCollaborationRatingsofHighSchoolStudentsThetablesinthissectionaretheindividualself-reportedteamratingsofthehighschoolparticipantsoftheirownbehaviorsduringeachsession(seeTable55throughTable65).WhiletheobservedbehaviorsdidnotincludeR1,eachparticipantfilledoutthesurveyattheendofeachsession,includingR1.Therefore,comparisonsforself-reportedsessionsaremadebetweenR1andR5.AllofthemeasuressawanincreaseintheaverageratingfromR1toR5.Thethreebiggestincreaseswereanincreaseof1.3foractivefollowing(seeTable56),1.1forempathiclistening(seeTable58),and1.0forabilitytodisputeproductively(seeTable62).Thehighestratingsinthelastweek(R5)includedsharesleadership(seeTable55)andbeinganactivefollower(seeTable56),followedbymutualrespect(seeTable59),andtheabilitytodisagreeproductively(seeTable62).Nextwereempathiclistening(seeTable58),trustinmovingtowardsasolution(seeTable60),definingacommonpurpose(seeTable63),andcreatingacultureofmutualaccountability(seeTable64).R5averagescoresrangedfrom6.0to6.8.NosignificantdifferencesinR1/R5changewereseenbetweencontrolandtreatmentgroups.Table55:SharesLeadershipAverageofSelf-ReportedTeamRatings,HighSchoolStudents Team R1 R2 R3 R4 R51(treatment) 5.5 6.0 5.8 6.8 7.02(treatment) 6.1 6.1 6.3 6.6 6.83(treatment) 6.3 6.1 6.4 5.2 6.04(treatment) 5.1 6.3 6.1 6.5 6.65(control) 3.9 6.0 5.8 4.4 6.26(control) 5.2 6.3 6.6 6.3 6.77(control) 4.7 5.3 5.9 6.1 6.18(control) 5.6 6.8 6.7 6.6 6.6Overallaverage 5.3 6.1 6.2 6.1 6.5Note:Scalewasfrom1(Notatall)to7(Completely)Table56:ActiveFollowerAverageofSelf-ReportedTeamRatings,HighSchoolStudentsTeam R1 R2 R3 R4 R51(treatment) 5.3 6.1 6.0 6.9 6.62(treatment) 6.5 5.5 5.4 6.5 6.63(treatment) 6.3 6.0 6.6 5.2 6.14(treatment) 3.8 6.4 6.0 6.5 6.55(control) 3.4 5.9 5.8 4.4 6.16(control) 5.4 6.3 6.3 6.5 6.87(control) 5.3 5.1 6.4 5.9 6.28(control) 5.8 6.6 6.6 6.7 6.8Overallaverage 5.2 6.0 6.1 6.0 6.5Note:Scalewasfrom1(Notatall)to7(Completely)


Table57:EmotionallyIntelligentBehaviorAverageofSelf-ReportedTeamRatings,HighSchoolStudentsTeam R1 R2 R3 R4 R51(treatment) 5.0 5.9 4.8 6.6 6.52(treatment) 6.1 5.4 6.8 6.3 6.53(treatment) 6.1 6.3 6.1 5.2 4.64(treatment) 5.1 6.0 5.0 6.3 6.35(control) 5.2 5.8 5.8 3.8 6.26(control) 5.3 6.2 6.1 6.0 6.77(control) 5.0 4.9 6.4 6.1 6.38(control) 4.9 6.6 6.7 6.6 6.6Overallaverage 5.3 5.9 6.0 5.8 6.2Note:Scalewasfrom1(Notatall)to7(Completely)Table58:EmpathicListeningAverageofSelf-ReportedTeamRatings,HighSchoolStudentsTeam R1 R2 R3 R4 R51(treatment) 4.9 5.1 4.1 6.7 6.92(treatment) 6.3 5.6 6.1 6.6 6.83(treatment) 6.2 6.3 6.5 6.2 5.34(treatment) 5.0 6.1 5.4 6.3 5.85(control) 3.6 5.7 6.0 3.8 6.16(control) 5.6 6.2 6.1 6.2 6.37(control) 5.2 5.4 6.6 6.3 6.38(control) 5.3 6.7 6.6 6.6 6.5Overallaverage 5.2 5.9 5.9 6.1 6.3Note:Scalewasfrom1(Notatall)to7(Completely)Table59:MutualRespectAverageofSelf-ReportedTeamRatings,HighSchoolStudentsTeam R1 R2 R3 R4 R51(treatment) 5.9 5.1 4.4 6.7 6.92(treatment) 6.9 6.5 4.8 6.4 6.13(treatment) 6.4 6.9 6.5 5.2 6.24(treatment) 5.6 6.2 6.0 6.4 6.05(control) 4.8 5.3 5.4 4.4 6.86(control) 5.5 6.6 6.2 6.6 6.47(control) 5.8 5.6 6.4 6.0 6.38(control) 4.7 6.9 6.7 5.9 6.8Overallaverage 5.7 6.2 5.8 5.9 6.4Note:Scalewasfrom1(Notatall)to7(Completely)


Table60:TrustinMovingTowardSolutionAverageofSelf-ReportedTeamRatings,HighSchoolStudentsTeam R1 R2 R3 R4 R51(treatment) 5.7 5.4 4.4 6.8 6.52(treatment) 6.1 5.8 6.8 6.5 6.63(treatment) 6.9 6.0 6.4 5.6 5.44(treatment) 5.4 5.4 6.0 6.5 6.15(control) 3.7 5.9 5.1 4.3 6.76(control) 5.3 6.5 6.6 6.6 6.47(control) 5.4 5.1 5.8 6.4 6.28(control) 5.6 6.8 6.7 6.8 6.4Overallaverage 5.5 5.9 6.0 6.1 6.3Note:Scalewasfrom1(Notatall)to7(Completely)Table61:TransparentinCommunicationAverageofSelf-ReportedTeamRatings,HighSchoolStudentsTeam R1 R2 R3 R4 R51(treatment) 5.1 5.3 4.8 6.4 6.82(treatment) 6.4 6.8 5.7 6.3 6.73(treatment) 6.1 6.1 6.4 5.1 5.64(treatment) 4.8 6.4 6.0 6.1 6.15(control) 5.3 5.6 6.0 4.9 5.46(control) 5.4 5.9 6.3 6.0 6.07(control) 5.3 5.3 5.7 6.1 6.18(control) 6.0 6.8 6.7 6.6 6.6Overallaverage 5.5 6.0 5.9 5.9 6.2Note:Scalewasfrom1(Notatall)to7(Completely)Table62:AbilitytoDisagreeProductivelyAverageofSelf-ReportedTeamRatings,HighSchoolStudentsTeam R1 R2 R3 R4 R51(treatment) 5.5 4.1 5.8 6.4 6.02(treatment) 6.4 6.0 5.8 6.8 6.13(treatment) 6.1 6.0 6.5 4.7 6.84(treatment) 5.5 5.9 5.4 6.4 6.55(control) 4.4 5.7 5.7 4.1 6.46(control) 5.5 6.1 6.1 6.7 6.47(control) 5.2 5.3 5.8 6.1 6.48(control) 5.2 6.0 6.4 6.6 6.5Overallaverage 5.4 5.6 5.9 5.9 6.4Note:Scalewasfrom1(Notatall)to7(Completely)


Table63:DefiningaCommonPurposeAverageofSelf-ReportedTeamRatings,HighSchoolStudentsTeam R1 R2 R3 R4 R51(treatment) 5.4 5.6 5.6 6.7 6.82(treatment) 6.4 5.9 5.6 5.5 6.53(treatment) 6.2 5.7 6.6 5.0 4.74(treatment) 5.4 5.9 6.0 6.9 6.45(control) 4.6 5.2 5.2 3.7 6.46(control) 5.7 6.5 6.1 6.4 6.77(control) 5.1 4.9 5.8 6.3 6.38(control) 5.7 6.4 6.7 6.6 6.8Overallaverage 5.5 5.8 5.9 5.8 6.3Note:Scalewasfrom1(Notatall)to7(Completely)Table64:DefiningaCultureofMutualAccountabilitySelf-ReportedTeamRatings,HighSchoolStudentsTeam R1 R2 R3 R4 R51(treatment) 5.5 5.9 3.8 6.9 7.02(treatment) 6.1 5.6 5.6 6.6 6.63(treatment) 6.1 5.4 6.3 5.1 5.34(treatment) 5.6 6.3 6.4 6.5 6.65(control) 4.4 5.4 5.4 3.2 5.46(control) 4.7 6.5 6.2 6.1 6.77(control) 5.4 5.1 5.9 5.6 6.38(control) 5.1 6.7 6.7 6.6 6.8Overallaverage 5.3 5.9 5.8 5.8 6.3Note:Scalewasfrom1(Notatall)to7(Completely)Table65:ProductivelyManagedDisruptionAverageofSelf-ReportedTeamRatings,HighSchoolStudentsTeam R1 R2 R3 R4 R51(treatment) 3.9 3.5 3.6 5.0 6.12(treatment) 6.0 5.5 5.3 5.8 6.43(treatment) 5.7 5.3 6.6 5.9 6.04(treatment) 5.5 6.1 5.4 5.9 6.45(control) 4.5 6.1 4.7 4.2 6.46(control) 4.3 5.5 6.7 6.2 5.97(control) 4.8 4.9 5.7 5.9 6.38(control) 4.5 5.2 5.7 5.7 4.3Overallaverage 4.8 5.3 5.5 5.6 6.0Note:Scalewasfrom1(Notatall)to7(Completely)


HighSchoolStudentBehaviorFindingsSummaryTherewereparticipationeffectsforbothcontrolandtreatmentgroupsofhighschoolstudents.Thiswasespeciallytruefortheobservedbehaviors,whichshowedstatisticallysignificantincreasesovertheweeklysessionsforboththecontrolgroupsandtreatmentgroups.Intermsofthebehaviorsimpacted,itisinterestingtonotethatthecontrolgroupshowedincreasesoverthefourweeksobservedinevenmoreofthebehaviors(6ofthe11)thanthetreatmentgroup(5ofthe11).Whenlookingattheself-reportedteamratingsofthesamemeasures,oneseesadifferentpicture.Onlytwoofthebehaviors,mutualrespectandtrustinmovingtowardsasolution,showedsignificantdifferencesbetweenthetwogroups;inbothcases,itwasthecontrolgroupthatgaveitselfhigherratingsfromR1toR5.Whencomparingthetwogroupsdirectly,therewere8ofthe11behaviorswherethefrequencyandpatternsofthebehaviorsdifferedsignificantlybetweencontrolandtreatment.In6ofthese8behaviors,thetreatmentgroupshowedthestrongerperformance.Thesebehaviorsweresharesleadership,trustinmovingtowardasolution,transparentincommunication,emotionallyintelligentbehavior,disagreeproductivelyanddefiningacommonpurpose.Intheremaining2ofthese8behaviors(creatingacultureofmutualaccountabilityandproductivelymanagesdisruption)thecontrolgroupshowedmarginally,butstatisticallysignificant,strongerperformance.TheR5comparisonallowedforacomparisonbetweenthecontrolandtreatmentgroupsduringtheirlastsessions,whenparticipantswerecompletingtheircourseofstudyandteamswerefinishingtheirworkonthechallengeandmakingalloftheirfinaldecisionswithrespecttobusinesscasesandpresentations.Thus,R5datagiveusasenseofthecumulativeimpactofthefulltwenty-hourinterventiononcollaborativebehaviorofcontrolandtreatmentgroups;ifbehaviordifferenceswouldbeexpectedtoexistanywherebetweenthetwogroups,itwouldbeduringthislastsessionattheculminationoftheproject.StatisticallysignificantbehavioraldifferencesforfiveoftheelevenbehaviorswereobservedinR5:sharesleadership,beingemotionallyintelligent,mutualrespect,theabilitytodisagreeproductively,anddefiningacommonpurpose.Forallfive,thetreatmentgrouphadasignificantlyhigheroccurrenceofthesebehaviors.


EarlyCareerSTEMProfessionalBehaviorFindingsThetablesinthefirstpartofthissection(Table66throughTable76)showtheobservedbehaviorratingsbytheresearchersfromR1throughR5;seeAppendixCfortheindividualbehaviorsstudied.Atableisincludedforeachoftheelevenbehaviorsobservedbytheresearcher,breakingdownthenumberoftimesthisparticularbehaviorwasobservedforeachteamduringeachweek.Theoverallsumofbehaviorsobservedineachweekisincludedatthebottomofthetable.Inordertocomparethegroupsoneachbehavior,statisticaltestswererunonthedifferencesbetweenthefrequencyandpatternsofoccurrencethesebehaviorswereobservedincontrolandtreatmentovertheR1toR5period.Iftherewasastatisticallysignificantdifferencebetweenthecontrolandtreatmentgroups,alinegraphwasincludedbelowthetable.Tableswithoutaccompanyinglinegraphsshowednostatisticallysignificantdifferencesbetweencontrolandtreatment.BasedontheR1toR5totals,behaviorsmostcommonlyobservedwerebeinganactivefollower(seeTable67)andmutualrespect(seeTable70).Thesewerefollowedbyempathiclistening(seeTable69)andsharingleadership(seeTable66).Thenexthighesttotalswereforemotionallyintelligentbehavior(seeTable68),definingacommonpurpose(seeTable74),andtrustinmovingtowardsasolution(seeTable71).Asmentionedabove,therewasacomparisonoftreatmentandcontrolgroupsinthefrequencyandpatternoftheelevenbehaviorsfromR1toR5.Cross-tabulationswererunlookingatthepercentageofindividualsinthecontrolandtreatmentgroupsengagingineachspecificbehavior,acrossweeks1through5.Thistypeofanalysiswaschosensinceitallowedadirectcomparisonbetweenthetwogroups,toseewhetherornotbeinginonegroupincreasedordecreasedthelikelihoodofengaginginthatparticularbehavior.Theseventhatshowedastatisticallysignificantdifferenceincludebehaviorsrelatedtosharingleadership,beinganactivefollower,emotionallyintelligentbehavior,empathiclistening,mutualrespect,trustinmovingtowardsasolution,andbeingtransparentincommunication.

ComparisonBetweenControlandTreatmentbyBehaviorSharesLeadershipTherewerefeweroccurrencesofthebehaviorinR1andR2,thenhigheroccurrencesinR3throughR5.Therewasastatisticallysignificantdifferenceinthefrequencyandpatternofthebehaviorbetweencontrolandtreatmentgroups.ThecontrolandtreatmentgroupswereverysimilarinR1,R2,andR4,thecontrolgroupwashigherinR3andthetreatmentgroupwashigherinR5.ItisinterestingtonotethatwhilethetreatmentgroupincreasessteadilyeachsessionfromR1toR5,thecontrolgrouppeaksatR4andthendeclinesinR5.


Table66:SumofSharesLeadershipBehaviorsObserved,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 7 8 10 27 242(treatment) 2 4 18 22 283(treatment) 1 1 14 20 294(treatment) 1 5 14 31 275(control) 0 5 21 26 246(control) 6 4 26 32 267(control) 4 3 28 24 228(control) 2 8 16 18 15Overallsum 23 38 147 200 195Figure10:SharesLeadership,SumofObservedBehaviorsinGroupsOverTime,EarlyCareerSTEMProfessionals

11

2056

100 108

12

18

91

10087

R1 R2 R3 R4 R5

Treatment Control


ActivefollowerOccurrencesofthisbehaviorincreasedinfrequencyeachweekbetweenR1andR3(particularlyinR3)andthenessentiallyplateauedduringtheremainingsessions.Therewasastatisticallysignificantdifferenceinthefrequencyandpatternofthebehaviorbetweencontrolandtreatmentgroups.ThetreatmentgroupwashigherthanthecontrolgroupinR1andR2,thecontrolandtreatmentgroupswereverysimilarinR3andR4,whilethetreatmentgroupwashigherthanthecontrolgroupinR5.Table67:SumofActiveFollowerBehaviorsObserved,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 22 22 38 44 502(treatment) 12 17 53 34 493(treatment) 20 15 40 35 524(treatment) 15 20 33 50 525(control) 7 13 37 40 356(control) 8 15 47 40 357(control) 14 13 50 44 418(control) 5 11 34 31 29Overallsum 103 126 332 318 343Figure11:ActiveFollower,SumofObservedBehaviorsinGroupsOverTime,EarlyCareerSTEMProfessionals

69 74164

163203

34 52

168

155 140

R1 R2 R3 R4 R5

Treatment Control


EmotionallyintelligentbehaviorTherewereloweroccurrencesforthebehaviorinR1andR2,alargeincreaseforR3,aslightdecreaseinR4,thenanotherincreaseinR5.Therewasastatisticallysignificantdifferenceinthefrequencyandpatternofthebehaviorbetweencontrolandtreatmentgroups.WhiletherewasahigheroccurrenceforthecontrolgroupinR2,theotherweekshadverysimilaroccurrencesforthecontrolandtreatmentgroups.However,acrossthefivesessions,thetreatmentgrouphadagreaterfrequencyofthisbehavior.Table68:SumofEmotionallyIntelligentBehaviorObserved,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 2 3 10 20 122(treatment) 1 3 12 5 193(treatment) 3 0 11 6 114(treatment) 1 4 14 8 125(control) 2 6 6 13 186(control) 2 5 15 8 147(control) 1 1 12 12 98(control) 1 5 14 4 10Overallsum 13 27 94 76 105Figure12:EmotionallyIntelligentBehavior,SumofObservedBehaviorsinGroupsOverTime,EarlyCareerSTEMProfessionals

7

10

4739

54

6

17

47

37

51

R1 R2 R3 R4 R5

Treatment Control

AudienceViewpointsConsultingArtofScienceLearningResearchReport100

EmpathiclisteningTherewasasteadyincreaseofoccurrencesofthebehaviorfromR1toR5,withtentimesthenumberofoccurrencescomparingR1toR5.Therewasastatisticallysignificantdifferenceinthefrequencyandpatternofthebehaviorbetweencontrolandtreatmentgroups.TherewasaverysimilarnumberofoccurrencesforthecontrolandtreatmentgroupsforR1andR2,higheroccurrencesforthecontrolgroupinR3andR4,thenahigheroccurrenceforthetreatmentgroupinR5.ItisinterestingtonotethatwhilethetreatmentgroupincreasessteadilyeachsessionfromR1toR5,thecontrolgroupplateausatR3andR4,andthendeclinesinR5.Table69:SumofEmpathicListeningBehaviorObserved,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 3 11 11 21 282(treatment) 3 9 17 13 403(treatment) 2 8 14 17 284(treatment) 2 10 8 24 305(control) 2 11 14 22 216(control) 3 7 29 26 197(control) 2 7 24 31 218(control) 3 10 21 16 20Overallsum 20 73 138 170 207Figure13:EmpathicListening,SumofObservedBehaviorsinGroupsOverTime,EarlyCareerSTEMProfessionals

10

38

50

75

126

1035

8895

81

R1 R2 R3 R4 R5

Treatment Control


MutualrespectTherewereloweroccurrencesforthebehaviorinR1andR2,thenmuchhighernumbersofoccurrencesinR3throughR5.Therewasastatisticallysignificantdifferenceinthefrequencyandpatternofthebehaviorbetweencontrolandtreatmentgroups.WhilethenumberofoccurrenceswassimilarfromR1throughR4,therewasahighernumberofoccurrencesforthetreatmentgroupinR5.Table70:SumofMutualRespectBehaviorsObserved,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 5 13 36 45 542(treatment) 2 4 46 35 543(treatment) 1 3 43 36 544(treatment) 1 9 38 50 505(control) 3 8 39 40 396(control) 3 7 48 40 407(control) 5 9 54 45 458(control) 0 11 35 27 35Overallsum 20 64 339 318 371Figure14:MutualRespect,SumofObservedBehaviorsinGroupsOverTime,EarlyCareerSTEMProfessionals

9 29

163

166

212

1135

176

152159

R1 R2 R3 R4 R5

Treatment Control


TrustinmovingtowardsolutionTherewerelowernumberofoccurrencesofthebehaviorinR1andR2,anincreaseforR3andR4,thenanotherincreaseforR5.Therewasastatisticallysignificantdifferenceinthefrequencyandpatternofthebehaviorbetweencontrolandtreatmentgroups.ThecontrolandtreatmentgroupsweresimilarforR1andR2,thecontrolgroupwasmodestlyhigherinR3,whilethetreatmentgroupwashigherinR4andmuchhigherinR5.Thetreatmentgrouproseeachweek,whilethecontrolgroupdeclinedconsiderablybetweenR3andR4.Table71:SumofTrustinMovingTowardSolutionBehaviorsObserved,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 3 7 8 9 222(treatment) 3 2 9 7 243(treatment) 1 1 9 9 104(treatment) 0 5 7 18 255(control) 0 5 12 6 36(control) 1 4 8 1 137(control) 2 4 14 12 108(control) 0 5 10 7 11Overallsum 10 33 77 69 118Figure15:TrustinMovingTowardSolution,SumofObservedBehaviorsinGroupsOverTime,EarlyCareerSTEMProfessionals

715

33

43

81

3

18

44

2637

R1 R2 R3 R4 R5

Treatment Control


TransparentincommunicationTherewererelativelyfewoccurrencesofthebehaviorinR1toR3,withsubstantialincreasesinbothR4andR5.Therewasastatisticallysignificantdifferenceinthefrequencyandpatternofthebehaviorbetweencontrolandtreatmentgroups.TherewererelativesimilaroccurrencesforthecontrolandtreatmentgroupsforR1toR3,thenhigheroccurrencesforthetreatmentgroupforR4andR5.Table72:SumofTransparentinCommunicationBehaviorsObserved,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 0 0 0 11 132(treatment) 0 2 0 8 113(treatment) 0 1 1 3 84(treatment) 0 3 2 13 215(control) 0 1 1 5 96(control) 2 0 2 5 117(control) 1 1 2 5 148(control) 0 0 1 9 7Overallsum 3 8 9 59 94Figure16:TransparentinCommunication,SumofObservedBehaviorsinGroupsOverTime,EarlyCareerSTEMProfessionals

0

6

3

35

53

3 26

24

41

R1 R2 R3 R4 R5

Treatment Control


AbilitytodisagreeproductivelyTherewasasteadyincreaseintheoccurrenceofbehaviorsfromR1toR5.Therewasnotastatisticallysignificantdifferenceinthefrequencyandpatternofthebehaviorbetweencontrolandtreatmentgroups.Table73:SumofAbilitytoDisagreeProductivelyBehaviorsObserved,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 0 3 3 7 42(treatment) 0 0 2 3 93(treatment) 0 2 0 3 04(treatment) 0 2 0 7 105(control) 0 1 3 3 76(control) 0 1 8 7 07(control) 1 2 3 5 58(control) 1 0 5 3 5Overallsum 2 11 24 38 40DefiningacommonpurposeTherewasasteadyincreaseintheoccurrenceofbehaviorsfromR1toR5.Therewasnotastatisticallysignificantdifferenceinthefrequencyandpatternofthebehaviorbetweencontrolandtreatmentgroups.Table74:SumofDefiningaCommonPurposeBehaviorsObserved,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 4 3 3 14 162(treatment) 1 3 13 6 113(treatment) 4 3 5 8 124(treatment) 2 7 5 10 165(control) 2 6 9 7 86(control) 3 3 16 16 117(control) 2 3 11 14 148(control) 1 10 10 9 10Overallsum 19 38 72 84 98


CreatingcultureofmutualaccountabilityTherewererelativelylowoccurrencesofthebehaviorfromR1toR5.Therewasnotastatisticallysignificantdifferenceinthefrequencyandpatternofthebehaviorbetweencontrolandtreatmentgroups.Table75:SumofCreatingCultureofMutualAccountabilityBehaviorsObserved,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 0 0 0 2 12(treatment) 0 0 1 1 03(treatment) 0 0 0 2 04(treatment) 0 0 0 4 05(control) 0 1 1 1 06(control) 0 0 2 2 37(control) 0 0 0 2 18(control) 0 1 3 0 0Overallsum 0 2 7 14 5ProductivelymanagesdisruptionTherewerelessfrequentoccurrencesofthebehaviorinR1andR2,anincreaseinR3andR4,thenadecreaseinR5.Therewasnotastatisticallysignificantdifferenceinthefrequencyandpatternofthebehaviorbetweencontrolandtreatmentgroups.Table76:SumofProductivelyManagesDisruptionBehaviorsObserved,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 2 1 0 9 42(treatment) 0 0 0 8 43(treatment) 0 0 0 5 34(treatment) 1 0 0 5 35(control) 0 0 4 8 46(control) 2 0 8 9 37(control) 2 0 5 8 48(control) 0 2 6 2 3Overallsum 7 3 23 54 28

ComparisonbetweenControlandTreatmentintheFinalSessionLookingatteams’behaviorsoverthecourseoftheprojectallowedforanunderstandingofhowtheteamsprogressedduringtheweeklysessions.Thefinalsession,R5representstheculminationoftheproject’sinnovationtraining,andlookingspecificallyattheR5outcomesgivesustheabilitytostudythewaysinwhichthecollaborativebehaviorsofthecontrolandtreatmentgroupswereimpactedbythe20hoursoftraining.Inordertocomparethecomparativeimpactsoffullthefull20-hourintervention,astatisticaltestwasrun,lookingatthedifferenceinthefrequencyofeachoftheelevenbehaviorsinR5,betweenthe


controlandtreatmentgroups. Oftheelevencomparisons,thefollowingtableshowstwobehaviorswithstatisticallysignificantdifferences:mutualrespectandtrustmovingtowardasolution(seeTable77).Bothoftheseshowedahigherlevelofoccurrenceforthetreatmentgroup.BothcontrolandtreatmentteamsspentR5finalizingtheirfinalpresentationsandbusinesscasesrelatedtothechallenge,sotherewerealotofopportunitiesforcollaborativebehaviors.Thetreatmentgroupengagedinthesetypesofbehaviorssignificantlymorethanthetreatmentgroup,suggestingthatthetreatmentgroupwasactingmorecollaborativelyastheyfiguredoutthefinalpiecesanddetailsoftheirpresentations.Table77:SummaryTableofObservedCollaboration,OnlyforWeek5(R5)(Pre/PostObservationalDifferenceScores) EarlyCareerSTEMProfessionalsObservationalCategory



Sharesleadership No -Activefollower No -Emotionallyintelligentbehavior

No -

Empathiclistening No -Mutualrespect Yes TreatmentTrustinmovingtowardsolution

Yes Treatment

Transparentincommunication No -


No -

Definingacommonpurpose No -


No -


No -


Self-ReportedTeamCollaborationRatingsofEarlyCareerSTEMProfessionalsThefollowingtablesaretheindividualself-reportedteamratingsoftheadultparticipantsoftheirownbehaviorsduringeachsession.AllbutoneofthemeasuressawanincreaseintheaverageratingfromR1toR5;sharesleadership(seeTable78)showedneitheranincreasenoradecrease.Thethreebiggestincreaseswereanincreaseof0.9forbeingabletodisagreeanddisputeproductively(seeTable85),anincreaseof0.6fordefiningacommonpurpose(seeTable86),andcreatingacultureofmutualaccountability.Thehighestratingsinthelastweek(R5)includedmutualrespect(seeTable82)andcreatingacultureofmutualaccountability(seeTable87).Thesewerefollowedbybeinganactivefollower(seeTable79),empathiclistening(seeTable81),beingtransparentincommunication(seeTable84),anddefiningacommonpurpose(seeTable86).Table78:SharesLeadershipAverageofSelf-ReportedTeamRatings,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 6.4 6.6 6.7 6.2 6.02(treatment) 6.0 5.6 6.2 6.0 6.63(treatment) 5.8 5.7 6.1 5.8 6.64(treatment) 6.6 6.4 6.5 6.6 65(control) 6.3 6.6 6.2 6.6 36(control) 6.2 6.9 6.9 5.4 6.97(control) 5.6 6.1 6.0 6.8 6.28(control) 5.4 6.7 6.7 5.4 7.0Overallaverage 6.3 6.3 6.4 6.1 6.3Note:Scalewasfrom1(Notatall)to7(Completely)Table79:ActiveFollowerAverageofSelf-ReportedTeamRatings,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 6.4 6.4 6.0 6.3 5.92(treatment) 6.2 5.3 6.6 5.9 6.43(treatment) 5.8 5.8 6.0 6.1 6.34(treatment) 6.6 6.3 6.5 6.4 6.65(control) 6.5 6.4 6.2 6.7 6.96(control) 5.5 6.1 6.6 5.9 6.27(control) 5.4 5.8 5.7 5.8 6.98(control) 5.9 6.0 6.4 5.9 5.7Overallaverage 6.0 6.0 6.2 6.1 6.4Note:Scalewasfrom1(Notatall)to7(Completely)


Table80:EmotionallyIntelligentBehaviorAverageofSelf-ReportedTeamRatings,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 6.6 6.2 6.7 6.7 6.02(treatment) 6.6 4.8 6.6 6.1 6.73(treatment) 5.4 5.4 6.0 5.2 6.44(treatment) 6.3 6.0 6.7 6.2 6.05(control) 5.7 6.5 5.9 6.7 6.76(control) 5.9 6.2 6.6 5.4 6.17(control) 6.2 6.1 5.0 6.0 6.68(control) 5.4 5.7 6.3 5.4 5.4Overallaverage 6.0 5.9 6.2 6.0 6.3Note:Scalewasfrom1(Notatall)to7(Completely)Table81:EmpathicListeningAverageofSelf-ReportedTeamRatings,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 6.7 6.3 6.8 6.2 5.82(treatment) 6.7 6.2 6.4 6.4 6.73(treatment) 6.2 5.7 6.3 6.1 6.74(treatment) 6.5 6.4 6.8 6.1 6.25(control) 6.5 6.7 6.0 6.7 6.96(control) 5.9 6.2 6.6 6.0 6.47(control) 5.7 5.4 5.7 6.7 6.98(control) 6.0 6.4 6.7 5.9 5.9Overallaverage 6.3 6.2 6.4 6.3 6.4Note:Scalewasfrom1(Notatall)to7(Completely)Table82:MutualRespectAverageofSelf-ReportedTeamRatings,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 6.9 6.8 6.8 6.7 6.02(treatment) 6.9 6.3 6.8 6.6 6.93(treatment) 5.6 6.2 6.3 6.1 6.74(treatment) 6.8 6.6 6.9 6.6 6.35(control) 6.7 6.7 6.5 6.7 7.06(control) 6.1 6.7 6.6 5.7 6.57(control) 5.6 5.9 6.6 7.0 7.08(control) 6.0 6.9 6.7 6.4 6.0Overallaverage 6.3 6.5 6.6 6.5 6.5Note:Scalewasfrom1(Notatall)to7(Completely)


Table83:TrustinMovingTowardSolutionAverageofSelf-ReportedTeamRatings,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 5.8 6.3 6.3 6.0 5.42(treatment) 6.3 5.3 6.2 6.3 6.83(treatment) 5.0 5.3 5.4 6.0 6.34(treatment) 6.4 6.5 6.6 6.5 6.05(control) 6.7 6.6 6.4 6.7 7.06(control) 6.4 6.5 6.6 6.0 6.47(control) 6.0 5.3 6.6 7.0 6.98(control) 5.6 6.7 5.9 5.7 5.6Overallaverage 6.0 6.1 6.3 6.3 6.3Note:Scalewasfrom1(Notatall)to7(Completely)Table84:TransparentinCommunicationAverageofSelf-ReportedTeamRatings,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 6.7 6.7 5.8 6.8 6.12(treatment) 6.4 6.0 6.7 6.2 6.73(treatment) 6.0 5.8 6.1 5.9 6.64(treatment) 6.5 6.3 6.6 6.5 6.85(control) 6.5 6.7 6.2 6.8 7.06(control) 6.1 6.6 6.7 5.4 6.27(control) 6.1 5.7 6.7 6.8 6.88(control) 5.3 6.6 6.4 6.0 5.1Overallaverage 6.2 6.3 6.4 6.3 6.4Note:Scalewasfrom1(Notatall)to7(Completely)Table85:AbilitytoDisagreeProductivelyAverageofSelf-ReportedTeamRatings,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 6.6 5.2 6.6 6.8 6.02(treatment) 5.2 6.0 5.9 5.1 6.43(treatment) 3.9 4.0 5.7 5.3 6.04(treatment) 6.4 5.5 6.6 5.9 5.85(control) 4.3 6.7 5.9 6.0 7.06(control) 4.9 5.9 5.7 5.1 6.47(control) 4.4 4.9 5.6 5.2 5.88(control) 5.3 6.4 5.6 5.9 5.9Overallaverage 5.2 5.5 6.0 5.7 6.1Note:Scalewasfrom1(Notatall)to7(Completely)


Table86:DefiningaCommonPurposeAverageofSelf-ReportedTeamRatings,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 5.8 6.1 5.8 6.4 6.12(treatment) 6.1 5.6 6.8 6.1 6.93(treatment) 6.0 5.8 5.9 6.0 5.94(treatment) 6.3 6.1 6.5 6.6 6.35(control) 6.2 6.6 6.0 6.6 7.06(control) 5.7 6.0 6.1 5.9 6.57(control) 5.6 5.1 5.6 5.8 6.98(control) 5.0 6.4 6.3 4.9 5.6Overallaverage 5.8 5.9 6.1 6.1 6.4Note:Scalewasfrom1(Notatall)to7(Completely)Table87:CreatingCultureofMutualAccountabilityAverageofSelf-ReportedTeamRatings,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 6.3 5.4 6.8 6.6 6.02(treatment) 6.3 6.0 6.3 6.2 6.93(treatment) 4.6 5.7 6.0 6.0 6.44(treatment) 6.0 6.0 6.4 6.4 6.55(control) 6.2 6.5 5.7 6.5 7.06(control) 5.2 6.1 6.7 5.9 6.57(control) 5.8 5.9 5.9 4.8 6.98(control) 4.4 6.3 6.6 6.0 5.3Overallaverage 5.6 6.0 6.3 6.0 6.5Note:Scalewasfrom1(Notatall)to7(Completely)Table88:ProductivelyManagesDisruptionAverageofSelf-ReportedTeamRatings,EarlyCareerSTEMProfessionalsTeam R1 R2 R3 R4 R51(treatment) 2.9 3.4 4.4 5.2 4.62(treatment) 3.1 4.9 5.7 5.7 5.63(treatment) 3.1 3.9 3.6 4.6 5.44(treatment) 4.0 3.9 4.0 4.5 5.05(control) 3.3 5.9 5.6 5.9 7.06(control) 4.9 4.6 5.4 4.9 6.57(control) 3.6 3.7 3.7 4.1 5.38(control) 3.9 5.3 5.7 4.7 4.4Overallaverage 3.6 4.4 4.7 4.9 5.5Note:Scalewasfrom1(Notatall)to7(Completely)


EarlyCareerSTEMProfessionalsBehaviorFindingsSummaryFortheearlycareerSTEMprofessionals,thereweredifferencesintheweeklytotalsforthetreatmentgroupsinabouttwothirdsoftheobservationbehaviors,whileonlyonebehaviorforthecontrolgroupshowedastatisticallysignificantdifference.Whencomparingthetwogroupsdirectly,therewere7ofthe11behaviorswherethefrequencyandpatternsofthebehaviorsdifferedsignificantlybetweencontrolandtreatment.In4ofthese7behaviors,thetreatmentgroupshowedtheunambiguouslystrongerperformance.Thesebehaviorswereactivefollower,mutualrespect,trustinmovingtowardasolutionandtransparentincommunication.In2ofthebehaviors,sharingleadershipandempathiclistening,thecontrolgroupshowedamarginally,butstatisticallysignificant,strongerperformance.Inemotionallyintelligentbehavior,thetreatmentgroupshowedamarginally,butstatisticallysignificant,strongerperformance.Fortheself-reportedteambehaviors,therewerenostatisticallysignificantdifferencesforeitherthecontrolortreatmentgroupacrossthefivesessions.TheR5comparisonallowedforacomparisonbetweenthecontrolandtreatmentgroupsduringtheirlastsessions,whenparticipantswerecompletingtheircourseofstudyandteamswerefinishingtheirworkonthechallengeandmakingalloftheirfinaldecisionswithrespecttobusinesscasesandpresentations.Thus,R5datagiveusasenseofthecumulativeimpactofthefulltwenty-hourinterventiononcollaborativebehaviorofcontrolandtreatmentgroups;ifbehaviordifferenceswouldbeexpectedtoexistanywherebetweenthetwogroups,itwouldbeduringthislastsessionattheculminationoftheproject.Statisticallysignificantbehavioraldifferencesfor2ofthe11behaviorswereobservedinR5:mutualrespectandtrustinmovingtowardsasolution.Forbothofthese,thetreatmentgrouphadasignificantlyhigheroccurrenceofthesebehaviors.


Hypothesis3Findings:TeamInnovationOutcomesHypothesis 3: Arts-based innovation training, compared to traditional innovation training, enhances the novelty, impact and feasibility of team innovation outcomes. WhileHypothesis1focusedonindividuals,andHypothesis2focusedontheteamastheunitofanalysis,Hypothesis3focusedontheinnovationoutcomesoftheteamsbyanalyzingtheprototypeproducts,processesandservicestheydeveloped.Theanalysiswasperformedbyadistinguishedgroupofexpertpanelists,composedofthreemembersoftheselectioncommitteeoftheProductDevelopmentManagementAssociation(PDMA)’sOutstandingCorporateInnovationAwards,includingthatcommittee’sfoundingandcurrentChairs.Thepanel,consultingwiththeprojectandresearchteams,developedanassessmentrubricidentifyingandweightingsevenmeasurestogaugethequalityoftheinnovationoutcomesincludingteaminnovationoutputsandworkproducts.Theysubsequentlyappliedtherubrictothenewproduct,processandservicesolutionconceptsdevelopedbytheteams.Allscoresarebasedonaratingscalefrom1-5points.Eachofthescaleswasgivenanoverallweight,inordertocalculatetherelativeimportanceofeachforthetotalscore.

1. InsightintoChallenge:Graspoftransportationneeds,conditionsandopportunities.[Overallweight=15%]

2. ClarityandRelevanceofProblem[Overallweight=15%]3. ProblemSolvingStrategy[Overallweight=25%]4. Impact:assessedbyvalueproposition,potentialmarketsizeandpenetration.[Overallweight=15%]5. DevelopmentalStrategy[Overallweight=10%]6. Feasibility:Market,technology,capacity,cost,competition,risk/barriersetc.[Overallweight=10%]7. Teamwork/Collaboration:Integrationofskills;distributedandcollaborativeeffort[Overall

weight=10%]

ThepanelistsmetinpersonovertwodaysattheKelleySchoolofBusinessatIndianaUniversitytodiscussandassesstheproduct,processandserviceinnovationscreatedbytheteamsinresponsetothecivicchallenges(seeAppendicesJandK).Thechallengesincludedtransportationalternativesforthehighschoolstudents,waterresourcesfortheearlycareerSTEMprofessionals.Aheadofthepanelmeeting(butafterthepanel’scompletionofworkontheassessmentrubric),thepanelistsreceivedthebusinesscasescreatedbyeachoftheteamsfortheiradvancereview;panelistswereaskednottodiscussthesematerialswiththeircolleaguesaheadofthemeeting.Duringthecourseofthemeeting,panelistsreviewedthreeadditionalcomponents:thePowerPointpresentationscreatedbyeachteamaboutitsinnovation,pre-recordedvideosofeachteam’s5-minuteconceptpresentation,andpre-recordedvideosofeachteam’sresponsestoastandardizedsetofquestions.Takentogether,theseelements(businesscase,PowerPointpresentations,videoofconceptpresentationandvideoofresponsestoquestions)formedthebasisforthepanel’soverallassessmentofeachteam’sinnovationoutcomes.


Theclassroomhostingthepanelmeetingwasequippedwithatwo-screensetup,whereeachPowerPointwasprojectedontheleftscreenwhilethevideoofthatteam’spresentationwasprojectedsynchronouslyontherightscreen,withaudioheardthroughtheclassroom’ssoundsystem.AprojectpartnerstartedthevideoontherightscreenandthenmanuallyranthroughthePowerPointslidesonthelefttomatchwheretheteamwasinthevideopresentation.Inthismanner,thepanelistswereabletobothwatchtheteamspresentandviewtheirPowerPointpresentationsatthesametime,approximatingtheexperienceofbeingintheclassroomwhentheteamsgavetheirpresentations.Thethree-personreviewpanelsattogetherinonerowoftheclassroom,tooknotesandindependentlyfilledoutratingsheetswhileviewingthepresentations.Afterthevideohadfinished,theycompletedtheratingsheetwithouttalkingtoeachother.Whenallthreepanelistsweredonewiththeirindividualratingsforthatteam,theyengagedinadiscussionaroundtheteam’soverallpresentation,itsapproachtotheproblem,theproposedsolution,andothertopicsontheratingsheets.Afterthisgroupdiscussionwasover,thepanelistswentbacktotheirratingsheetsandmadeanyadjustmentsorrescoringofthesheets.Thesheetswerecollectedafterthereviewofeachteam’spresentationwascompleted.Thissameprocesswasrepeatedforeachteam.Eleventeamswerereviewedinthismanneronthefirstdayandfiveonthesecondday.Thereviewersdidnotknowwhichofthe16groupswereinthecontrolortreatmentcondition,andtheorderofconsiderationofteamswithineachgroup(i.e.,adultsandadolescents)wasrandomized.

HighSchoolStudentsFindingsForhighschoolstudents,thetotalweightedteaminnovationoutcomescoresvariedgreatly,rangingfroma1.3toa4.4ona5-pointscale(seeTable89).Next,thescoresofeachitemforeachteaminthecontrolandtreatmentgroupswerecombined,andthetwosetsofcombinedtotalswerecompared(seeTable90).Treatmentoutperformedcontrolonallsevenindividualitemsscored.Fourofthesedifferenceswerestatisticallysignificant:insightintochallenge(2.0difference),clarityandrelevanceoftheproblem(1.7difference),problemsolvingstrategy(1.3difference)andthepotentialimpactoftheirproposal(1.3difference).Whilethedifferencesbetweencontrolandtreatmentontheotherthreeitemsscoreddidnotreachstatisticalsignificance,thetreatmentgroupdidhavehigherratingsthanthecontrolgrouponeach.Similarly,inthetotalweightedteaminnovationoutcomescore,whichusedanaverageweightedtotalscoreacrossallitemsforthecontrolgroupcomparedtotheaverageweightedtotalscoreacrossallitemsforthetreatmentgroup,thetreatmentgrouphadhigherratingsbutthedifferencedidnotreachstatisticalsignificance.


Table89:PanelScoresforHighSchoolStudentsTeamFinalProducts(InnovationOutcomes) Control Treatment Team

5Team6

Team7

Team8

Team1

Team2

Team3 Team4

Insightintochallenge 2.8 1.0 1.3 2.3 5.0 1.7 3.0 3.3Clarityandrelevanceofproblem 3.3 1.3 2.7 2.7 5.0 1.7 4.0 3.0

ProblemSolvingStrategy 2.0 1.3 2.5 2.0 4.8 2.0 2.3 2.7

Impact 2.7 1.3 1.7 2.3 4.3 1.7 2.3 3.0

DevelopmentalStrategy 3.0 1.0 3.5 3.8 3.7 1.3 3.3 2.7

Feasibility 2.3 1.3 3.0 3.3 3.3 1.3 4.0 2.0Teamwork/Collaboration 4.0 2.0 5.0 2.7 3.7 2.3 2.7 4.3

Totalweightedscore 2.8 1.3 2.6 2.6 4.4 1.8 3.0 3.0Table90:AveragePanelScoresforHighSchoolStudentsTeamFinalProducts(InnovationOutcomes),ControlandTreatment

Control Treatment

StatisticallySignificantDifference?

Insightintochallenge 1.8 3.8 YesClarityandrelevanceofproblem 2.3 4.0 YesProblemSolvingStrategy 2.0 3.3 YesImpact 1.9 3.2 YesDevelopmentalStrategy 2.5 3.2 NoFeasibility 2.3 3.1 NoTeamwork/Collaboration 3.2 3.6 NoTotalweightedscore 2.3 3.0 No

EarlyCareerSTEMProfessionalsFindingsForearlycareerSTEMprofessionalsthetotalweightedteaminnovationoutcomescoresvariedgreatly,rangingfroma2.2toa4.2ona5-pointscale(seeTable91).Next,thescoresofeachitemforeachteaminthecontrolandtreatmentgroupswerecombined,andthetwosetsofcombinedtotalswerecompared(seeTable92).Whiledifferencesbetweencontrolandtreatmentwerefoundforthehighschoolstudentsontheindividualitems,noneofthedifferencesbetweencontrolandtreatmentonthesevenindividualitemsscoredwerestatisticallysignificantfortheearlycareerSTEMprofessionals.Similarly,therewasnostatisticallysignificantdifferencebetweenthecontrolandtreatmentgroupsinthetotalweightedscoreacrossallitems.


Table91:PanelScoresforEarlyCareerSTEMProfessionalsTeamFinalProducts(InnovationOutcomes) Control Treatment Team

5Team6

Team7

Team8

Team1

Team2 Team3

Team4

Insightintochallenge 3.7 3.0 3.7 4.0 3.0 3.7 2.0 4.0

Clarityandrelevanceofproblem 4.0 3.3 3.3 3.7 3.3 3.2 3.0 4.3

ProblemSolvingStrategy 3.8 1.5 3.3 2.3 3.0 2.3 2.0 4.3

Impact 3.2 1.3 2.7 2.2 2.7 2.3 1.7 4.3

DevelopmentalStrategy 2.7 1.6 3.0 2.7 2.0 2.2 1.3 4.3

Feasibility 3.7 1.8 2.2 2.8 2.5 2.8 2.0 4.2Teamwork/Collaboration 4.5 3.7 3.3 2.7 3.7 4.7 3.3 4.0

Totalweightedscore 3.7 2.2 3.1 2.9 2.9 2.9 2.2 4.2Table92:AveragePanelScoresforEarlyCareerSTEMProfessionalsTeamFinalProducts(InnovationOutcomes),ControlandTreatment

Control Treatment

StatisticallySignificantDifference?

Insightintochallenge 3.6 3.2 NoClarityandrelevanceofproblem 3.6 3.5 NoProblemSolvingStrategy 2.7 2.9 NoImpact 2.3 2.7 NoDevelopmentalStrategy 2.5 2.4 NoFeasibility 2.6 2.9 NoTeamwork/Collaboration 3.5 3.9 NoTotalweightedscore 3.0 3.0 No Theroleoftheexpertpanelistswastoprovideanobjectiveassessmentofthenewproducts,processesandservicesconceptuallydevelopedbytheteamsovertheir5-weekcoursesoftraining(i.e.,teaminnovationoutputs).Assuch,panelistsconsideredtheextenttowhicheachteamwasabletosurfaceinsightfulthinkingabouttheInnovationChallenge;identifyaclear,relevantandproductiveproblemtosolve;findanovelandpotentiallyimpactfulwaytosolveitschosenproblem;outlineacompellingdevelopmentalandgo-to-marketstrategyforitsinnovation;andarticulateapotentiallyfeasiblepathfromconcepttoimplementation.Inaddition,10%ofeachteam’stotalweightedscorewasbasedonthepanel’sassessmentofitsintegrationskillsthroughdistributedandcollaborativeeffort;anassessmentthatwasnecessarilyinferential,sincepanelistshadextremelylimitedopportunitytowitnessdirectevidenceofteamcollaborativeprocesses,dynamicsanddecision-making.GiventhatthepanelistswerenotinvolvedinorpresentduringthetrainingsessionsinWorcesterorSanDiegoandthattheydidnotknowwhichgroupswerecontrolortreatment,theirscoringwasagoodtestofwhethertheintegrationofarts-basedlearningintoinnovationtrainingimpactedthekindsofnew


products,processesandservicesdevelopedbytheteams;andwhetheritimpactedthewaysteamscreativelyaddressedSTEMInnovationChallengesintheirthinkingaboutproblems,solutions,novelty,value,impact,feasibilityandotherkeyconceptsofinnovation.

TeamInnovationOutcomesFindingsSummaryItwasaveryimportantresultthattheexpertpanelistsratedthehighschoolproducts,processesandservicesofthetreatmentteamssignificantlyhigherthatthoseofthecontrolteamsintermsofinsight,clarity,problemsolvingstrategy,andpotentialimpact.Incontrast,therewerenostatisticallysignificantdifferencesbetweencontrolandtreatmentgroupsreflectedinthepanelistratingsoftheearlycareerSTEMprofessionalteams.AsmentionedearlierintheLimitationssectionofthisreport,itispossiblethatthislackoffindingsfromtheadultteamsmayresultfromusingacurriculumthatwasdevelopedspecificallyforadolescents.Furtherstudytodeterminewhetheradultfindingswouldchangewiththesubstitutionofacurriculumspecificallydesignedforusewithadultswouldbeveryuseful.Thefindingsinthissectionshowthepowerfulpotentialbenefitsofapplyingarts-basedapproachestoadolescentSTEMinnovationlearningandpractice.Theyalsounderlinethecriticalneedforfurtherstudytodeterminewhether,andinwhatways,adultfindingswouldchangewiththesubstitutionofacurriculumspecificallydesignedforusewithadults.


References

Basadur,M.S.(1993).Impactsandoutcomesofcreativityinorganizationalsettings.InTheemergenceofadiscipline:Nurturinganddevelopingcreativity.VolumeII.(Editors:Isaksen,S.G.,Murdock,M.C.,Firestein,R.L.,&Treffinger,D.J.),Chapter12(2),278-313.NewYork:Ablex.

Basadur,M.S.,&Finkbeiner,C.T.(1985).Measuringpreferenceforideationincreativeproblem

solvingtraining.JournalofAppliedBehavioralScience,Vol.21,No.1,37-49.Basadur,M.,Graen,G.,&Wakabayashi,M.(1990).Identifyingindividualdifferencesincreative

problemsolvingstyle.TheJournalofCreativeBehavior,24(2),111-131.Epstein,R.,Schmidt,S.M.,&Warfel,R.(2008).Measuringandtrainingcreativitycompetencies:

Validationofanewtest.CreativityResearchJournal,20(1),7-12.Hackman,J.R.(2002).Leadingteams:Settingthestageforgreatperformances.Boston,MA:

HarvardBusinessSchoolPress.Heifetz,R.A.,&Linsky,M.(2002).Leadershipontheline:Stayingalivethroughthedangersof

leading.Boston,MA:HarvardBusinessSchoolPress.Kahn,K.B.(Ed.)(2013).ThePDMAhandbookofnewproductdevelopment.Hoboken,NewJersey:

JohnWiley&Sons.Retrievedfromhttp://altmannpeter.com/PDMA%20Handbook%20of%20New%20Product%20Development%20(3rd%20Edition).pdf

Katzenbach,J.R.,&Smith,D.K.(1993).Thewisdomofteams:Creatingthehigh-performance

organization.Boston,MA:HarvardBusinessSchoolPress.Norton,R.W.(1975).Measurementofambiguitytolerance.JournalofPersonalityAssessment,39,

607-619.Runco,M.A.,&Basadur,M.S.(1993).Assessingideationalandevaluativeskillsandcreativestyles

andattitudes.CreativityandInnovationManagement,2(3),166-173.Sawyer,K.(2007).Groupgenius:Thecreativepowerofcollaboration.NewYork,NY:BasicBooks.Senge,P.,&Scharmer,O.(2001).Communityactionresearch.InP.ReasonandH.Bradbury(Eds.),

HandbookofactionResearch.ThousandOaks,CA:SagePublications.Thota,H.,&Munir,Z.(2012).Keyconceptsininnovation.Boston,MA:PalgraveMacmillan.


Appendices

AppendixA:PreWorkshopSurvey WelcometotheTransportationChallenge!Wearesopleasedtohaveaspartofthisproject.Asyouknow,thisprojectistheresearchportionofalargerNSFgrantonscienceandlearningininformalcontexts.Theresearchisanintegralportionoftheproject,andunderstandingwhoyouare,whatyouthink,andwhatyoudoisimportantinformationformakingtheprojectasuccess.Thisinformationisalsoimportanttounderstandwhetherprojectsofthistypeareworthyourtimeandtaxpayermoney.Thisparticularsurveyiscriticaltousgoingforward.Thissurveyshouldtakeapproximately20-30minutestocomplete.Thequestionsinthesurveyareaboutyou;therearenorightorwronganswers.Pleasebeassuredthatallofyourresponseswillbekeptconfidential.Noneoftheinformationyouprovidethatcouldidentifyyouindividuallywillbeincludedinthepresentationofsurveyresults.Wewouldliketoencourageyoutotakeyourtimeandanswerthesurveyquestionsbothasopenlyandhonestlyasyoucan.

We’dliketoknowyouabitbetterasyoujointhisproject.Thenextsetofquestionsarebackgroundquestionstobetterunderstandyouroutsideinterests,especiallyconnectionswithscienceandart.

1. Howoftenhaveyouparticipatedinthefollowingactivitiesinthelasttwoyears:

Activity Notatallinthelasttwoyears

Atleastonceinthelasttwoyears

Attendedalivemusic,theater,ordanceperformance Performedinorpracticedaspecificartform(e.g.,dance,singing,classicalmusic,etc.)

Attendedanartmuseumorgallery Tookaclassorlesson(whetherinoroutofschool)inanartformorartsubject

Emailed,posted,orsharedartwork(yourownorothers;includesphotos&music)

UsedTV,radio,ortheInternettoaccessartorartsprogramming

Attendedaprofessionalconferencerelatedtothearts Readanarts-focusedblog Visitedacraftsfairoravisualartsfestival Taughtanartclassorlesson Readascience-focusedblog Taughtascienceclassorlesson Attendedaprofessionalconferencerelatedtoscience


Attendedasciencemuseum Performedascienceexperiment(informallyorformally) UsedTV,radio,ortheInternettoaccessscienceprogramming

Visitedasciencefestival Tookascienceclassorlesson(whetherinoroutofschool)

Emailed,posted,orsharedscientificinformation(e.g.,quoteorarticleofinterest,etc.)

Participatedinaneveningeventatascience-basedinstitution

2. Onascalefrom1(Stronglydisagree)to7(Stronglyagree),towhatextentdoyouagreeordisagreewiththefollowingstatements?

a) OccasionallyIliketoworkonextremelydifficultproblemsb) Ionlyliketasksthathaveahighprobabilityofsuccessc) Idonotsharemyideaswithothersd) Ioftenreadbooksandmagazinesoutsideofmycoreinterestareae) Isometimesusemydreamsordaydreamsasasourceofnewideasf) Iamnotafraidoffailureg) Daydreamingonlywastesmytimeh) Idonotliketoworkonproblemsthathavenosolutioni) TherearespecialplaceswhereIgotothinkj) Ikeepsomethingbymybedatnight,torecordideask) Ienjoyworkingwiththesamegroupofpeopleallthetimel) Idonotneedanymorecolleaguesm) Iseektraininginnewareasn) Imakeanefforttomeetnewpeople

3. Onascalefrom1(Stronglydisagree)to7(Stronglyagree),towhatextentdoyouagreeor

disagreewiththefollowingstatements?a) Almosteveryproblemhasasolutionb) Iliketofoolaroundwithnewideas,eveniftheyareatotalwasteoftimec) Nothinggetsaccomplishedinthisworldunlessyousticktosomebasicrulesd) Usually,themoreclearlydefinedrulesasocietyhas,thebetteroffitise) Personally,Itendtothinkthatthereisarightwayandwrongwaytodoalmosteverythingf) Idon'tneedtofinishataskbeforestartinganewtaskg) Beforeanyimportantjob,Imustknowhowlongitwilltakeh) Inaproblem-solvinggroupitisalwaysbesttosystematicallyattackaproblemi) IdonotliketogetstartedingroupprojectsunlessIfeelassuredthattheprojectwillbe

successful


j) Inadecision-makingsituationinwhichthereisnotenoughinformationtoprocesstheproblem,Ifeelveryuncomfortable


a) Thereistoomuchwastedtimeinthecreativeprocessb) Brilliantideascomefromsingle‘AHA!’moments,notfromworkingthroughthecreative

processc) Emotionsdetractfromeffectivecommunicationd) Itisalwayscounterproductivewhenconflictarisesduringcollaborativeworke) Agoodcommunicatorcancommunicateeffectivelythesamewayineverysituationf) Collaborationisrarelyworththetimeittakesg) Noteveryoneiscapableofcreativityh) Thelesslifeexperienceyouhavethelessyouhavetoofferwhencollaboratingwithothersi) Peoplearemorelikelytoproduceeffectivesolutionsthroughcompetition,ratherthan

throughcollaborationj) Individualsaremorelikelythangroupstocomeupwithtrulyoriginalideask) Alotofpeoplewhothinktheyareeffectivecommunicatorsjusttalkalotl) Tobecreativeyoumustbeartisticm) Creativityissomethingdonebyindividuals,notsomethingthathappensonagroupleveln) Communicationispredominantlyverbalo) Peoplewhoareoutgoingarenaturallybettercommunicators

5. Onascalefrom1(Stronglydisagree)to5(Stronglyagree,pleaseratethefollowingsentences:a) Ishoulddosomeprejudgmentofmyideasbeforetellingthemtoothers.b) Weshouldcutoffideaswhentheygetridiculousandgetonwithit.c) Ifeelthatpeopleatworkoughttobeencouragedtosharealltheirideas,becauseyounever

knowwhenacrazy-soundingonemightturnouttobethebest.d) Onenewideaisworthtenoldones.e) Qualityisalotmoreimportantthanquantityingeneratingideas.f) Agroupmustbefocusedandontracktoproduceworthwhileideasg) Lotsoftimecanbewastedonwildideas.h) Ithinkeveryoneshouldsaywhateverpopsintotheirheadwheneverpossible.i) Iliketolistentootherpeople’scrazyideassinceeventhewackiestoftenleadstothebest

solution.j) Judgmentisnecessaryduringideagenerationtoensurethatonlyqualityideasare

developedk) Youneedtobeabletorecognizeandeliminatewildideasduringideageneration.l) Ifeelthatallideasshouldbegivenequaltimeandlistenedtowithanopenmindregardless

ofhowzanytheyseemtobe.


m) Thebestwaytogeneratenewideasistolistentoothersthentailgateoraddon.n) Iwishpeoplewouldthinkaboutwhetherornotanideaispracticalbeforetheyopentheir

mouth.

6. Onemoresetofquestionsabouthowyouliketotackleproblems.Onascalefrom1(Stronglydisagree)to7(Stronglyagree)onthesequestionsonyourpersonality:

a) Iliketogetthingsstartedbygettinginvolved,gatheringinformation,questioning.b) Ilikeimaginingthepossibilitiesandsensingallkindsofnewproblemsandopportunities.c) Icanseegoodandbadsidestoalmostanyfact,ideaorissue.d) Iamcomfortablewithsituationswerenoteverythingisclear.e) I’mwillingtoletotherstakecareofthedetails.f) Itendtoformquickassociations,defineproblemsandconceptualizenewideas,

opportunitiesandbenefits.g) Iamgoodatininductivereasoning,inpullingtogetherseeminglyunrelatedobservations

intoanintegratedsolution.h) Idon’tlikegoingforwarduntilIhaveasoundunderstandingofthesituation.i) Iwouldprefernottohavetoprioritizeamonggoodornotfullyunderstoodalternatives.j) Ipreferideasratherthanmovingtoaction.k) Idobestinsituationswherethereisasinglecorrectanswerorbestsolutiontoaproblem.l) Icansortthroughlargeamountsofdataandpinpoint“what’swrong”inagivensituation.m) Iamconfidentofmyabilitytomakeasoundevaluationandselectthebestsolutiontoa

problemn) Itendtolackpatiencewithambiguity.

Iprefernotspendingtoomuchtimethinkingaboutotherideasandpointsofview,orhowdifferentproblemsrelatetooneanother.

o) Ilikebecominginvolvedinnewexperiences.p) Iliketotrythingsoutratherthan“mentallytest”them.q) Iconsidermyselfarisk-taker:Idon’tneedtounderstandsomethingcompletelybeforeIact.r) I’mwillingtotryasmanydifferentapproachesasnecessaryuntilIfindonethatis

sufficientlyacceptabletothoseaffectedbytheproblem.s) Itendtobeenthusiastic,butcanbeimpatientasItrytoactonplans.

7. Onascalefrom1(Stronglydisagree)to7(Stronglyagree),towhatextentdoyoufeelyouarean

innovatorinyourschoolwork?

8. Onascalefrom1(Stronglydisagree)to7(Stronglyagree),towhatextentdoyoufeelyouareaninnovatorinyourprofessionallife?


innovatorinyourpersonallife,includinghobbiesandinterests?


10. Pleaseexplainyourratingsonwhetheryouseeyourselfasaninnovatorornot.

11. Workingwithotherclasses,we’velearnedthatindividualshaveverydifferentandpersonal

definitionsofinnovation.Whatisyourdefinitionofinnovation?Youarealmostdone!Thefinalsetofquestionsaredemographicinnaturethatwecollecttoknowthatadiversesampleofpeopleparticipatedwithinthisproject.Again,yourresponseswillremainanonymous.12. Areyou...

a) Maleb) Femalec) Other/Prefernottosay.

13. Whichethniccategorydoyoumostidentifywith?(PleasecheckALLthatapply)Thisinformationis

important,aspartoffederalgrant,weneedtoshowwearereachingabalancedaudience.

a) AfricanAmericanorBlackb) AmericanIndianorAlaskaNativec) Asiand) CaucasianorWhitee) HispanicorLatinof) NativeHawaiianorotherPacificIslanderg) Other:(pleasespecify)h) Prefernottoanswer

14. Whatisthehighestlevelofeducationthatyou'vecompleted?(PleasecheckONE)

a) Lessthanhighschool(I'mstillenrolledinhighschool)b) Lessthanhighschool(I'mnolongerenrolledinhighschool)c) HighSchool/GEDd) Communitycollege/technicaltrainingorcertificatee) Collegedegree(BA/BS)f) GraduateorPostgraduatedegree

15. Thankyouverymuchforcompletingthispre-survey.Wearelookingforwardtoyourparticipation

inthechallengeoverthecomingweeks!Ifthereisanythingelseyouwouldliketosharewithusatthistimepleasedosobelow.


AppendixB:PostWorkshopSurvey Wehavebeensopleasedtohaveyouparticipateinthischallenge!Nowforthefinalreflection.We’dliketoaskyousomequestionsaboutyou,yourteam,andyourthoughtsaboutthechallenge.Asyouknow,thisprojectistheresearchportionofalargerNSFgrantonscienceandlearningininformalcontexts.Theresearchisanintegralportionoftheproject,andunderstandingwhoyouare,whatyouthink,whatyoucandoisimportantinformationformakingtheprojectasuccess.Thisinformationisalsoimportanttounderstandwhetherprojectsofthistype(research)areworthyourtimeandtaxpayermoney.Thisparticularsurveyiscriticaltotheresearch.Thissurveyshouldtakeapproximately20-30minutestocomplete.Thequestionsinthesurveyareaboutyou;therearenorightorwronganswers.Pleasebeassuredthatallofyourresponseswillbekeptanonymous.Noneoftheinformationyouprovidethatcouldidentifyyouindividuallywillbeincludedinthepresentationofsurveyresults.Wewouldliketoencourageyoutotakeyourtimeandanswerthesurveyquestionsbothasopenlyandhonestlyasyoucan.1. Onascalefrom1(Stronglydisagree)to7(Stronglyagree),towhatextentdoyouagreeor

disagreewiththefollowingstatements?a) OccasionallyIliketoworkonextremelydifficultproblemsb) Ionlyliketasksthathaveahighprobabilityofsuccessc) Idonotsharemyideaswithothersd) Ioftenreadbooksandmagazinesoutsideofmycoreinterestareae) Isometimesusemydreamsordaydreamsasasourceofnewideasf) Iamnotafraidoffailureg) Daydreamingonlywastesmytimeh) Idonotliketoworkonproblemsthathavenosolutioni) TherearespecialplaceswhereIgotothinkj) Ikeepsomethingbymybedatnight,torecordideask) Ienjoyworkingwiththesamegroupofpeopleallthetimel) Idonotneedanymorecolleaguesm) Iseektraininginnewareasn) Imakeanefforttomeetnewpeople

2. Onascalefrom1(Stronglydisagree)to7(Stronglyagree),towhatextentdoyouagreeor

disagreewiththefollowingstatements?a) Almosteveryproblemhasasolutionb) Iliketofoolaroundwithnewideas,eveniftheyareatotalwasteoftimec) Nothinggetsaccomplishedinthisworldunlessyousticktosomebasicrulesd) Usually,themoreclearlydefinedrulesasocietyhas,thebetteroffitise) Personally,Itendtothinkthatthereisarightwayandwrongwaytodoalmosteverythingf) Idon'tneedtofinishataskbeforestartinganewtaskg) Beforeanyimportantjob,Imustknowhowlongitwilltake


h) Inaproblem-solvinggroupitisalwaysbesttosystematicallyattackaproblemi) IdonotliketogetstartedingroupprojectsunlessIfeelassuredthattheprojectwillbe

successfulj) Inadecision-makingsituationinwhichthereisnotenoughinformationtoprocessthe

problem,Ifeelveryuncomfortable


a) Thereistoomuchwastedtimeinthecreativeprocessb) Brilliantideascomefromsingle‘AHA!’moments,notfromworkingthroughthecreative

processc) Emotionsdetractfromeffectivecommunicationd) Itisalwayscounterproductivewhenconflictarisesduringcollaborativeworke) Agoodcommunicatorcancommunicateeffectivelythesamewayineverysituationf) Collaborationisrarelyworththetimeittakesg) Noteveryoneiscapableofcreativityh) Thelesslifeexperienceyouhavethelessyouhavetoofferwhencollaboratingwithothersi) Peoplearemorelikelytoproduceeffectivesolutionsthroughcompetition,ratherthan

throughcollaborationj) Individualsaremorelikelythangroupstocomeupwithtrulyoriginalideask) Alotofpeoplewhothinktheyareeffectivecommunicatorsjusttalkalotl) Tobecreativeyoumustbeartisticm) Creativityissomethingdonebyindividuals,notsomethingthathappensonagroupleveln) Communicationispredominantlyverbalo) Peoplewhoareoutgoingarenaturallybettercommunicators

4. Onascalefrom1(Stronglydisagree)to5(Stronglyagree,pleaseratethefollowingsentences:a) Ishoulddosomeprejudgmentofmyideasbeforetellingthemtoothers.b) Weshouldcutoffideaswhentheygetridiculousandgetonwithit.c) Ifeelthatpeopleatworkoughttobeencouragedtosharealltheirideas,becauseyounever

knowwhenacrazy-soundingonemightturnouttobethebest.d) Onenewideaisworthtenoldones.e) Qualityisalotmoreimportantthanquantityingeneratingideas.f) Agroupmustbefocusedandontracktoproduceworthwhileideasg) Lotsoftimecanbewastedonwildideas.h) Ithinkeveryoneshouldsaywhateverpopsintotheirheadwheneverpossible.i) Iliketolistentootherpeople’scrazyideassinceeventhewackiestoftenleadstothebest

solution.j) Judgmentisnecessaryduringideagenerationtoensurethatonlyqualityideasare

developed


k) Youneedtobeabletorecognizeandeliminatewildideasduringideageneration.l) Ifeelthatallideasshouldbegivenequaltimeandlistenedtowithanopenmindregardless

ofhowzanytheyseemtobe.m) Thebestwaytogeneratenewideasistolistentoothersthentailgateoraddon.n) Iwishpeoplewouldthinkaboutwhetherornotanideaispracticalbeforetheyopentheir

mouth.

5. Onemoresetofquestionsabouthowyouliketotackleproblems.Onascalefrom1(Stronglydisagree)to7(Stronglyagree)onthesequestionsonyourpersonality:

a) Iliketogetthingsstartedbygettinginvolved,gatheringinformation,questioning.b) Ilikeimaginingthepossibilitiesandsensingallkindsofnewproblemsandopportunities.c) Icanseegoodandbadsidestoalmostanyfact,ideaorissue.d) Iamcomfortablewithambiguity.e) I’mwillingtoletotherstakecareofthedetails.f) Itendtoformquickassociations,defineproblemsandconceptualizenewideas,

opportunitiesandbenefits.g) Iexcelininductivereasoning,indistillingseeminglyunrelatedobservationsintoan

integratedsolution.h) Idon’tlikeproceedinguntilIhaveasoundunderstandingofthesituation.i) Iwouldprefernottohavetoprioritizeamonggoodornotfullyunderstoodalternatives.j) Ipreferideasratherthanmovingtoaction.k) Idobestinsituationswherethereisasinglecorrectansweroroptimalsolutiontoa

problem.l) Icansortthroughlargeamountsofdataandpinpoint“what’swrong”inagivensituation.m) Iamconfidentofmyabilitytomakeasoundevaluationandselectthebestsolutiontoa

problemn) Itendtolackpatiencewithambiguity.

Iprefernotspendingtoomuchtimethinkingaboutotherideasandpointsofview,orhowdifferentproblemsrelatetooneanother.

o) Ilikebecominginvolvedinnewexperiences.p) Iliketotrythingsoutratherthan“mentallytest”them.q) Iconsidermyselfarisk-taker:Idon’tneedtounderstandsomethingcompletelybeforeIact.r) I’mwillingtotryasmanydifferentapproachesasnecessaryuntilIfindonethatis

sufficientlyacceptabletothoseaffectedbytheproblem.s) Itendtobeenthusiastic,butcanbeimpatientasItrytoactonplans.


innovatorinyourschoolwork?

7. Onascalefrom1(Stronglydisagree)to7(Stronglyagree),towhatextentdoyoufeelyouareaninnovatorinyourprofessionallife?



innovatorinyourpersonallife,includinghobbiesandinterests?

9. Pleaseexplainyourratingsonwhetheryouseeyourselfasaninnovatorornot.

10. Hasyourperceptionofyourselfasaninnovatorchangedduringyourparticipationinthisproject?Inwhatways?

11. Onascalefrom1(StrongNegativeimpact)to7(StrongPositiveImpact),towhatextentdidparticipatingintheTransportationChallengechange:

a) youroverallabilitytothinkcreatively?b) Whetherandhowyoudocumentideasastheypopup?c) thewayyoubrainstorm?d) originalityofthinking?e) howyoudescribeandelaborateonyourthinkingtoothers?f) howyouworkwithawiderangeofindividuals?g) yourcomfortinplacingyourselfintodifficultorunfamiliarsituations?h) yourcomfortinambiguoustasks?i) whetheryouthinkofyourselfascreative?j) howmanystrategiesyoucomeupwithwhenproblem-solving?

12. Nowthattheprojectisover,wewouldlikeyoutoratethecontributionofeachmemberofyour

grouptothecompletionofthetasksandoverallproject.Yourratingswillbeanonymousandwillnotbesharedwithothermembersofthegroupatanytime.Theywillonlybeusedforanalysispurposes,andcombinedwithotherresponses.

Overall,duringthisproject…

CONTRIBUTEDVERYLITTLE

1

2

3

4

5

6

CONTRIBUTEDALOT7

Member#1,Yourself: ! ! ! ! ! ! !Member#2,Name: ! ! ! ! ! ! !Member#3,Name: ! ! ! ! ! ! !Member#4,Name: ! ! ! ! ! ! !Member#5,Name: ! ! ! ! ! ! !Member#6,Name: ! ! ! ! ! ! !Member#7,Name: ! ! ! ! ! ! !Member#8,Name: ! ! ! ! ! ! !

Thankyouverymuchforcompletingoursurveyandthisproject!Wereallyappreciateyourtimeandthoughtfulness.Doyouhaveanyfurthercomments,thoughtsorquestionsforus?


AppendixC:ObservationRating TEAM:___Date(dd/mm/yy):_______ObserverInitials:_______________EachObservationSheetrepresentsonegroupobservedideallyfor20-30minutesduringanon-taskgrouptime.Onceobservationbegins,itdoesn’tstopeveniftheygroupveersofftask.GROUPACTIVITY#:________StarttimeofObservation:______EndTimeofObservation:_______

#1:

#2:

#3:

#4:

#5:

#6:

#7:

#8:

#9:

#10:

#11:

ParticipantContributiontotheTaskRating(1-7) __ __ __ __ __ __ __ __ __ __ __

• Sharesleadershipaccordingtoknowledge/skill

! ! ! ! ! ! ! ! ! ! !

• Active,engagedfollower ! ! ! ! ! ! ! ! ! ! !

• Emotionallyintelligentbehaviors,interpersonalrelationshipsanddynamics

! ! ! ! ! ! ! ! ! ! !

• Empathiclistening(Opennesstootherpointsofview,acknowledgesothersviews)

! ! ! ! ! ! ! ! ! ! !

• Mutualrespect ! ! ! ! ! ! ! ! ! ! !• Trustinmovingtowardsasolution ! ! ! ! ! ! ! ! ! ! !

• Transparentincommunicationsandinformation

! ! ! ! ! ! ! ! ! ! !

• Abilitytodisagreeanddisputeproductively ! ! ! ! ! ! ! ! ! ! !

• Definingacommonpurpose(particularlyatthesolution,buy-inasanindicator)

! ! ! ! ! ! ! ! ! ! !

• Creatingacultureofmutualaccountability ! ! ! ! ! ! ! ! ! ! !

• Productivelymanagesdisruptionwithinthegroup,reframingcontextordirection

! ! ! ! ! ! ! ! ! ! !


AppendixD:CollaborationRating Name:_______________Date(dd/mm/yy):_________GroupNumber:__________Pleaseanswerthequestionsbelowaboutyourgroupashonestlyaspossible.Theothergroupmemberswillnotseeyourresponsesandtheywillallbecombinedforanalysis.

Duringtoday’stask(s),ourgroup…

STRONGLYDISAGREE

1

2

3

NEITHERAGREENORDISAGREE

4

5

6

STRONGLYAGREE7

• Sharedleadershipbetweenpeople,basedonknowledgeandskill ! ! ! ! ! ! !

• Didagoodjobfollowingtheintentofthegroup,ratherthaneachperson’sindividualthinking

! ! ! ! ! ! !

• Wasawareofandtookintoaccounthoweachgroupmemberwasfeeling

! ! ! ! ! ! !

• Listenedtoandacknowledgedeveryone’spointsofview ! ! ! ! ! ! !

• Showedmutualrespecttoeachother ! ! ! ! ! ! !

• Trustedthatwewouldeventuallycomeupwithagoodsolution ! ! ! ! ! ! !

• Wastransparentwitheachotherwhenwecommunicatedandsharedinformation

! ! ! ! ! ! !

• Wasabletodisagreeanddisputeproductively ! ! ! ! ! ! !

• Definedthecommonpurposewewereworkingtowards ! ! ! ! ! ! !

• Createdacultureofmutualaccountability,wherewewereallresponsibleforcompletingthetask

! ! ! ! ! ! !

• Productivelymanagedanydisruptionwithinthegroup(ifapplicable)

! ! ! ! ! ! !

• Wasabletosuccessfullycompletethetask(s)assignedtoustoday. ! ! ! ! ! ! !


AppendixE:TransferabilitySurvey It’sbeenseveralmonthssinceyourparticipationintheWorcesterInnovationChallenge.Nowthatsometimehaspassed,we’dliketoaskyousomequestionsaboutyourparticipation.1. Towhatextenthaveyoubeenthinkingaboutwhatyoulearnedaboutinnovationandcreativity

duringtheWorcesterInnovationChallengePleasecheckoneonly.

Haven’tthoughtaboutitatall

Thoughtaboutitonceortwice

Havethoughtaboutitweekly

Havethoughtaboutitseveraltimesaweek

Havethoughtaboutitdaily

2. WhatwereyourinitialreasonsfordecidingtoparticipateintheChallenge.Whatmotivatedyoutodoit?

3. DidparticipatingintheChallengehaveanyeffectonhowyouhavebeenthinkingaboutsolutions

forothertopicsorissues?

1–Notatall

2 34–Tosome

extent5 6

7-Agreatdeal

a. Canyougiveanexampleofhowithaschanged?

4. Doyoufeelinthisworkshopyouhaddoactivitiesthatwereoutofyourcomfortzone?

1–Notatall

2 34–Tosome

extent5 6

7-Agreatdeal

a. Whydidyougivetheratingyoudid?

5. TowhatextenthaveyoubeenabletoapplytheInnovationChallengeexperiencetoyourcurrentworkorvolunteeractivities?

1–Notatall

2 34–Tosomeextent

5 67-Agreat

deal

NotApplicable

6. Whichpartsofthechallengehaveyoubeenabletouseinyourcurrentworkorvolunteer

activities?Pleasegiveasmanyexamplesasyoucan.


7. TowhatextentdoyouthinkyouwillbeabletoapplyyourInnovationChallengeexperiencetoyourfutureworkorvolunteeractivities?

1–Notatall

2 34–Tosomeextent

5 67-Agreat

deal

NotApplicable

a. Ifyes,inwhatways?

8. TowhatextenthaveyoubeenabletoapplytheInnovationChallengeexperiencetoyourcurrentschoolorextracurricularsituation?

1–Notatall

2 34–Tosomeextent

5 67-Agreat

deal

NotApplicable

9. Whichpartsofthechallengehaveyoubeenabletouseinyourcurrentschoolorextracurricular

activities?Pleasegiveasmanyexamplesasyoucan.

10. TowhatextentdoyouthinkyouwillbeabletoapplyyourInnovationChallengeexperienceto

yourfutureschoolorextracurricularsituation?

1–Notatall

2 34–Tosomeextent

5 67-Agreat

deal

NotApplicable

a. Ifyes,inwhatways?

11. TowhatextenthaveyoubeenabletoapplyyourInnovationChallengeexperiencetoyourcurrent

home/personallife?

1–Notatall

2 34–Tosome

extent5 6

7-Agreatdeal

a. Inwhatways?

12. Ifyoufeltyoubenefitedfromthischallenge,wereparticularattributesaboutyourbackgroundor

situationthatmadeitusefultoyou?We’reinterestedinwhetherthosethatbenefitfromthistraininghaveparticularcharacteristicsorbackgroundincommon.


13. WhatwastheonethingyouexperiencedintheChallengethatyouthinkwillbemosthelpfultoyouinthefuture,andwhy?


AppendixF:PreWorkshopCreativitySkillsTaskWorcester YourName:__________________________________Group: ☐ Morning �AfternoonWelcometotheWorcesterChallenge!We’resopleasedtohaveyouhere.Wehaveabriefwarm-uptaskthatweneedyoutocomplete.YourWarm-UpChallengeisasfollows:Worcester(substituteyourtownifyou’renotlivinginWorcester)needstransportationalternativestoenhanceitseconomicproductivity,connectitsneighborhoodsandcommunities,andimprovethequalityoflifeforitsresidents.Duringthenext15minutes,dothefollowing:A) Listupto5specificproblemsrelatedtothischallengethatyou'dliketofix.

1.2.3.4.5.

B) Fromthatlistabove,selecttheoneproblemyouwouldmostliketofix,andexplainwhy

youchosethatparticularproblemfromthelistyoucreated.Itmayhelptofocusontheparticularcharacteristic(s)oftheproblemthatmakestheproblemyouselectedeitherparticularlyimportantorparticularlyfixable.


C) Listupto5possiblesolutionstotheproblemyou'vechoseninanswer‘B’above.

1.2.3.4.5.

D) Fromthelistof5possiblesolutions,selecttheonesolutionyouwouldmostliketodevelop.Explainwhyyouchosethissolution,focusingontheparticularcharacteristic(s)ofthesolutionthatseemstohavemadeitmostappropriateforaddressingtheproblem,givingasmuchevidenceaspossibleforyourdecision.


AppendixG:PreWorkshopCreativitySkillsTaskSanDiego YourName:__________________________________

Group: �Morning �Afternoon

WelcometotheSanDiegoChallenge!We’resopleasedtohaveyouhere.Wehaveabriefwarm-upactivitythatwe’dlikeyoutocomplete.

SanDiego’sInnovationChallengeisasfollows:

SanDiegoneedsinnovationsthatwillreducethegapbetweenitsregionalwatersupplyandthedemandsofitsindustrial,agriculturalandresidentialusers.

Duringthenext7minutes,please:

A) Listupto5specificproblemsrelatedtothischallengethatyou'dliketofix.

1.

2.

3.

4.

5.

B) Fromthatlist,selecttheoneproblemyouwouldmostliketofixandbrieflyexplainwhyyouchosethatparticularproblem.Itmayhelptofocusoncharacteristicsoftheproblemyouselectedthatmakeiteitherparticularlyimportantorparticularlyfixable.


InnovationChallenge

January10,2015

YourName:__________________________________

Group: � Morning � Afternoon

Duringthenext7minutes,please:

C) Listupto5possiblesolutionstotheproblemyou'vechosenasanswer“B”onthepreviouspage.

1.

2.

3.

4.

5.

D) Fromthislist,pleaseselecttheonesolutionyouwouldmostliketodevelopandbrieflyExplainwhyyouchosethissolution.Pleasefocusonthecharacteristicsofthesolutionyouchosethatmotivatedyoutochooseit,givingasmuchevidenceaspossibletosupportyourdecision.


AppendixH:PostWorkshopCreativitySkillsTask Wehavesoenjoyedtheseconversationsoverthelastfewweeks.Asoneoftheclosingactivities,we’dlikeyoutoworkonanotherquicktask.YourWrap-UpChallengeisasfollows:ImagineatownsimilarsizeandcompositionofWorcester/SanDiego,wheretwentypercentofthepopulationisaffectedbyfoodinsecurity,lacksfairandequitableaccesstoasufficientquantityofaffordable,nutritiousfoodforallcitizens.Duringthenext7minutes,dothefollowing:A) Listupto5specificproblemsrelatedtothischallengethatyou'dliketofix.

1.2.3.4.5.

B) Fromthatlistabove,selecttheoneproblemyouwouldmostliketofix,andexplainwhy

youchosethatparticularproblemfromthelistyoucreated.Itmayhelptofocusontheparticularcharacteristic(s)oftheproblemthatmakestheproblemyouselectedeitherparticularlyimportantorparticularlyfixable.


Duringthenext7minutes,dothefollowing:C) Listupto5possiblesolutionstotheproblemyou'vechoseninanswer‘B’above.

1.2.3.4.5.

D) Fromthelistof5possiblesolutions,selecttheonesolutionyouwouldmostliketo

develop.Explainwhyyouchosethissolution,focusingontheparticularcharacteristic(s)ofthesolutionthatseemstohavemadeitmostappropriateforaddressingtheproblem,givingasmuchevidenceaspossibleforyourdecision.


AppendixI:CreativitySkillsTaskRubric SectionA:Listupto5problemsEachproblemwillbescoredeither0or1,asperthescoringtablebelow.Atotal,from0toamaximumof5points,willbeassignedforeachrespondent.SCORE1:SectionAScoringTable

0points 1point• Restatementofanotherproblem

onthelist• Answernotfocusedonthetopic• Listedonlysolutions• Commentsabouthowtogoaboutidentifyingproblemsorsolutions

• Commentssuggestingtheparticipantsdidnotreadorunderstandtheinstructions

1pointperdistinctproblemmeetingthefollowingfivecriteria(nopartialcreditperproblem).

• Theintentoftheproblemstatementisreasonablyclear• Theproblemstatementisnotjustarestatementofthechallenge• Aproblemisstated,ratherthanasolution• Theproblemisdistinctfromothersidentifiedbythesameperson• Theproblemrelatestothechallenge

Example:CarpollutionandfumesDamagedroadsSlowconstructionBikeandcaraccidentsSlowmovingtrafficMaximumpossiblescoreof5.

SCORE2:IdeaClusterTheIdeaClusterisacategoricalcode,withmultiplecategoriespossible.Foroptimumcodingpurposes,therewillbenomorethan10categories.Thiscodeisalevelhigherthanthedistinctsolution,sothatmultipledistinctproblemsmayfitwithinasingleideacluster.Forexample,ideaclusterswithinTransportationproblemsetsmayinclude,butarenotlimited:

• Safety• Poorenvironmentalconditions(suchaspollution)• Poorqualitypublicspaces• Availabilityofpublictransportation• Affordabilityofpublictransportation

SectionB:ExplainwhyyouchosetheproblemSCORE3:StrengthofProblemStatement0-2scoringtokeeppeopleinthegameasmuchaspossible.Inordertokeeppeopleinthegameasmuchaspossible,aproblemnotpreviouslylistedinSectionAisgivenequalweighttoonethatwaslistedinSectionA.Further,ascoreof1isgiventoanalignedproblemevenifitsexpressionisinferentialinnature,solongasitisclear,specificanddistinct.Iftheindividualreceivesaboveascoreofzero,moveontoscore4.

• 2=aclear,distinct,specific,explicitlywordedproblemalignedtothechallenge.• 1=aclear,distinct,specificproblemalignedtothechallengebutinferentialinexpression.• 0=aresponsethatfailedtomeetthecriteriafor1or2points.

SCORE4:Acountofeachvalid/logically-connectedreasongivenforchoosingtheproblem-onepointperreason.Thereisnolimittothepossiblescore.


SCORE5:Acountofthedifferentrationalesfortheproblemchoiceoffered(categoricalandmultiplecategoriespossible):impact,feasibility,personalengagement,originality/uniqueness,ancillarybenefits.Theseshouldalsobecodedbycategory.Note:ananswermighthavemultiplereasonscodedasthesamecategory,andshouldonlyreceiveonepointperdifferentrationalecategory.SectionC:Listupto5solutions(seenotebelowaboutmatchbetweenSectionCandSectionsA/BEachsolutionwillbescored0or1,asperthescoringtablebelow.SCORE6a:SectionCScoringTable

0points 1point• Restatementofchallenge• Statesaproblemratherthan

asolution• Restatesanothersolutionon

thelist• Answervagueornotfocused

onthetopic• Commentsabouthowtogo

aboutidentifyingproblemsorsolutions


1pointperdistinctsolutionmeetingthefollowingfourcriteria(nopartialcreditpersolution).

• Theintentofthesolutionstatementisreasonablyclear• Thesolutionstatementgoesbeyondthepurelyaspirational• Thesolutionstatementclearlyrepresentsa“how”responsetothe

problemidentifiedinB• Thesolutionisdistinctfromothersidentifiedbythesameperson

1pointperdistinctsolution(nolimit).Example:IncreasepublicbusscheduleDecreasepriceofbusticketIncreasenumberofbuses

SCORE6b

0points 1point• Restatementofchallenge• Statesaproblemratherthan

asolution• Restatesanothersolutionon

thelist• Answervagueornotfocused

onthetopic• Commentsabouthowtogo

aboutidentifyingproblemsorsolutions


1pointperdistinctsolutionmeetingthefollowingfourcriteria(nopartialcreditpersolution).

• Theintentofthesolutionstatementisreasonablyclear• Thesolutionstatementgoesbeyondthepurelyaspirational• Thesolutionstatementclearlyrepresentsa“how”responsetosome

challenge-relatedproblemotherthantheproblemidentifiedinSectionB.

• Thesolutionisdistinctfromothersidentifiedbythesameperson1pointperdistinctsolution(nolimit).Example:IncreasepublicbusscheduleDecreasepriceofbusticketIncreasenumberofbuses

SCORE6c–totalof6aand6bSectionD:ExplainwhyyouchosethesolutionScore7:SolutionStatementStrength0-2scoringtokeeppeopleinthegameasmuchaspossible.Inordertokeeppeopleinthegameasmuchaspossible,asolutionnotpreviouslylistedinSectionCisgivenequalweighttoonethatwaslisted


inSectionC.Further,ascoreof1isgiventoansolutionalignedwithadifferentproblem,solongasthatproblemisalignedwiththechallenge.Iftheindividualreceivesaboveascoreofzero,moveontoscore8.

• 2=aclear,distinct,specificpossiblesolutionthataddressesthechallengebysolvingtheproblempreviouslyidentifiedinSectionB.

• 1=aclear,distinct,specificpossiblesolutionthataddressesthechallengebysolvingsomeotherchallenge-relatedproblem

• 0=aresponsethatfailedtomeetthecriteriafor1or2pointsSCORE8:Countofthenumberofreasonsgiven,includingnumberof“evidence”statementswhentheyaretherewithoutreasonsSCORE9:TypeofRationaleofferedforSolutionChoice(categoricalandmultiplecategoriespossible):impact,feasibility,personalengagement,originality/uniqueness,comparativeanalysis,ancillarybenefits.SCORES10&11:SpecificityofSolution

• What(specifically)doestheparticipantproposetodo.(assign0pointsforunclear/incoherentproposal,1pointforclearbutwithoutdetail,2pointsforadditionallevelofdetail).

• How(specifically)doestheparticipantproposetodoit.(assign0pointsofthequestionisnotaddressed,1pointforageneric/non-specificwayofaddressingit,2pointsforadditionalspecificity).

SCORE12:IdeaClusteringTheIdeaClusterisacategoricalcode,withmultiplecategoriespossible.Thiscodeisalevelhigherthanthedistinctsolution,sothatmultipledistinctsolutionsmayfitwithinasingleideacluster.Forexample,ideaclusterswithinTransportationproblemsetsmayinclude,butarenotlimited:

• Safety• Improvedenvironmentconditions(suchaspollution)• Improvedpublicspaces• Availabilityofpublictransportation• Affordabilityofpublictransportation

Additionaldetails:

• StudentsareeligibletoreceivecreditforananswertoBorDeveniftheydidnotcompleteAorCiftheystatedorimpliedaproblem/solution.


AppendixJ:BusinessCase

Teamnumber:_____________PROBLEMSTATEMENT:SOLUTIONTITLE:1. SOLUTIONOVERVIEW–Describethespecificsolutionyouwanttodevelop.

2. RELEVANCE–Howistheproposedsolutionrelevanttothe[Worcestertransportationchallenge/San

Diegowaterchallenge]?

3. NEWNESS–Describespecificallyhowthesolutionisneworinnovative.

4. MARKET

a. Whoisthecustomer?

b. Whatisthevalueproposition?Whywillcustomersbuythisinnovation?

c. Howlargeisthepotentialmarket?


d. Howmanycustomerswillhavepurchasedthisafteroneyearonthemarket?

e. Howmuchwillyourcustomersbewillingtopayforyourinnovation?

f. Whatisthebasisforyourestimatestoc,dandeabove?

5. IMPACT–What’sthecaseforyourinnovationhavingasignificantdirectimpactonthechallenge?

6. FEASIBILITY

a. Whatkeystepsareneededtodevelopyoursolutionfromconcepttomarket?Howlongwouldtheytake?

b. Arethereobstaclesthatcallintoseriousquestionthefeasibilityoftheproposedsolution?

Howmightyouaddressthem?

c. Whatskillswillbeneededforthedevelopmentteam?

d. Aretherepotentialpartnerswhocouldhelp?

e. Howwillyourinnovationgeneraterevenuestosustainitselfinthemarket?


AppendixK:OutputScoringSheet Panelist/Reviewername:_________________GroupnumberID:_____________1.INSIGHTINTOCHALLENGE-Graspoftransportationneeds,conditionsandopportunities.[Overallweight=15%]

5 Outstanding(Conceptshowsexceptionalinsightintoactualconditionsandopportunities,andmeetssignificantunmetneeds)

4

3 Acceptable(Conceptaddressesthechallengebysolvingrelevantunmetproblems/needs)

2

1 Limited(Conceptdeficientinitslackofgroundinginactualconditionsorfailuretoaddressrelevantunmetneeds)

InitialRating:________

Ratingafterdiscussion:______________

Comments:2-CLARITYANDRELEVANCEOFPROBLEM[Overallweight=15%]

5 Outstanding(Problemisveryclearlydefinedandrelevanttothechallenge)

4

3 Acceptable(Problemdefinitionhasreasonableclarityandrelevance)

2

1 Limited(Problemasstatedisunclearorirrelevanttothechallenge)

InitialRating:__________


Comments:3-PROBLEMSOLVINGSTRATEGY[Overallweight=25%]

5 Outstanding(Solutionisahighlyoriginalandpotentiallyfruitfulwayofaddressingtheproblem)

4

3 Acceptable(Solutionhaselementsofnoveltyandseemslikelytopartiallyaddresstheproblem)

2


1 Limited(Solutionlacksnoveltyorisunlikelytomeaningfullyaddresstheproblem)

InitialRating:_________


Comments:4-IMPACT:assessedbyvalueproposition,potentialmarketsizeandpenetration.[Overallweight=15%]

5 High(Solutiondeliverscompellingvaluetoasubstantialnumberofclearlydefinedcustomers)

4

3 Medium(Teamsuccessfullyarticulatesacrediblevaluepropositiontoanidentifiedcustomerbase)

2

1 Low(Solutionfailstodeliversignificantvalueorcrediblyidentifyacustomerbase)



Comments:5-DEVELOPMENTALSTRATEGY[Overallweight=10%]

5 High(Clearevidenceofintegrationofskillsandeffort)4 3 Medium(Appropriatecollaborativebehaviordemonstrated)2 1 Limited(Evidenceofbadteamworkorcollaborativedysfunction)



Comments:


6-FEASIBILITY:Market,technology,capacity,cost,competition,risk/barriersetc.[Overallweight=10%]

5 High4 3 Medium2 1 Low



Comments:7–TEAMWORK/COLLABORATION:Integrationofskills;distributedandcollaborativeeffort[Overallweight=10%]

5 High(Clearevidenceofintegrationofskillsandeffort)4 3 Medium(Appropriatecollaborativebehaviordemonstrated)2 1 Limited(Evidenceofbadteamworkorcollaborativedysfunction)



Comments:


AppendixL:Inter-raterReliabilityScoringforCreativitySkillsTest Question ScoreCreativityScore1:1 1CreativityScore1:2 1CreativityScore1:3 1CreativityScore1:4 1CreativityScore1:5 1CreativityScore2:1 0.652CreativityScore2:2 0.652CreativityScore2:3 1CreativityScore2:4 1CreativityScore2:5 1CreativityScore3 CreativityScore5:Impact 0.75CreativityScore5:Feasibility 0.75CreativityScore5:PersonalEngagement 1CreativityScore5:Originality/Uniqueness 1CreativityScore5:ComparativeAnalysis 1CreativityScore5:AncillaryBenefits 1CreativityScore6a:1 1CreativityScore6a:2 1CreativityScore6a:3 1CreativityScore6a:4 1CreativityScore6a:5 1CreativityScore6b:1 1CreativityScore6b:2 1CreativityScore6b:3 1CreativityScore6b:4 1CreativityScore6b:5 1CreativityScore7 1CreativityScore9:Impact 1CreativityScore9:Feasibility 1CreativityScore9:PersonalEngagement 1CreativityScore9:Originality/Uniqueness 1CreativityScore9:ComparativeAnalysis 1CreativityScore9:AncillaryBenefits 1CreativityScore10:What CreativityScore10:How 1CreativityScore12:1 0.652CreativityScore12:2 1CreativityScore12:3 0.652CreativityScore12:4 1CreativityScore12:5 1


AppendixM:IdeaClusterCategoriesforCreativitySkillsTest IdeaClustersforPreCreativitySkillsTest,HighSchoolStudents

1.SocialConnectionbetweenneighborhoods2.Access–schedules/distancemostlyrelatedtopublictransportation,accessforthosewithdisabilities3.Safety–ofdriving,publictransportation,walking,biking,cleanlinessoftransportation4.Environment–pollutioncausedbytransportation5.Infrastructure–roads,sidewalks,bikelanes,construction,repair,design6.Alternatives–sharedtransportation,alternativefuelsources,bikes7.Publictransportation–neworrevampedbuses,subways,trollies8.Government–legislationabouttransportation,subsidies,grants,laws,taxes9.Traffic–relatedtodriving10.Economic–costoftransportation11.Misc.–technology,stress,resources,education,advertising

IdeaClustersforPostCreativitySkillsTest,HighSchoolStudents

1.Access–farmersmarkets,grocerystores,location,safety,availability2.Economic–foodcost,costofliving,discounts3.Education&Awareness-classes,campaigns,commercials,information4.School–foodprograms,changelunchplan,classesinschool(homeec,gardening)5.CommunityEffort–locality,charity,communitygardens,buyingorgrowingfoodtogether6.Agricultural–farming,crops,GMOs,chemicals,runoff,pollution,verticalfarming,labeling7.Health–obesity,nutrition,vitamins,etc.8.GovernmentAction9.InfrastructureofFoodSupply–transportandsupplyoffood10.FoodWaste11.Misc.–examples:overpopulation,watersupply,tech,research,jobsEarlySTEMProfessionals

IdeaClustersforPreCreativitySkillsTest,EarlySTEMProfessional

1.Collection(andLoss)ofEarth’swater–salt,rain,storm,desalination,general“runoff”comments2.Recycle/Reusehouseholdwater–greywater,shower,toilet,tap,washingmachine,drinking3.GovernmentAction–taxincentives,regulations,subsidies,laws,mandates,watervalue,emergency4.Conservation(andWaste)–waterfilters,productssuchasshowerheadsor½flushtoilets,communitycontests,householdmetering5.AgriculturalUse6.ResidentialUse–lawns,gardens,nativeplants,“landscape”insomeinstances7.IndustrialUse8.Infrastructure–commentsabout(in)efficiency,transport,piping,concrete,etc.9.EducationandAwareness–includescommentsaboutadvertisingandbranding10.Pollution(andPurification)–e.g.“contamination”and“cleanwatersupply”11.Misc.–energy,composting,oil&gasusage,“foodindustry”,environmentaldetriment(aquaticecosystems),selfishnessofpeople


IdeaClustersforPostCreativitySkillsTest,EarlySTEMProfessional

1.Access–farmersmarkets,grocerystores,location,safety,availability2.Economic–foodcost,costofliving,discounts3.Education&Awareness-classes,campaigns,commercials,information4.School–foodprograms,changelunchplan,classesinschool(homeec,gardening)5.CommunityEffort–locality,charity,communitygardens,buyingorgrowingfoodtogether6.Agricultural–farming,crops,GMOs,chemicals,runoff,pollution,verticalfarming,labeling7.Health–obesity,nutrition,vitamins,etc.8.GovernmentAction9.InfrastructureofFoodSupply–transportandsupplyoffood10.FoodWaste11.Misc.–examples:overpopulation,watersupply,tech,research,jobs

aosl research report - the impact of arts-based … · the impact of arts-based innovation training...

Documents