2022 Pioneer In Tech AwardWinner: Poppy NorthcuttThe NCWIT Pioneer in Tech Award recognizes technical women whoselifetime contributions have significantly impacted the landscapeof technological innovation, amplifying the importance ofcapitalizing on the diverse perspectives that girls and womencan bring to the table. In this session, we celebrate the 2022recipient, Frances “Poppy” Northcutt.

YouTube Video Link https://www.youtube.com/watch?v=Fnfk4NWBSiw

“A Queer Endeavor: InclusiveK-12 Education” With DoctorsBethy Leonardi and Sara StaleyWant to learn about creating cultures that are affirming ofgender and sexual diversity? This session focuses on the K-12space and will also touch on ways curriculum can be made moreinclusive.

YouTube Video Link https://www.youtube.com/watch?v=67JO279CO9k

“A Queer Endeavor: QueeringLeadership” With Doctors BethyLeonardi and Sara Staleyant to learn about creating cultures that are affirming ofgender and sexual diversity? This workshop series bringsattention to policy, climate, and social and cultural norms andpractices.

YouTube Video Link https://www.youtube.com/watch?v=eoT8HCCTlGA

“Navigating the New Normal:Renewal, Allyship, & JoyDuring Twin Pandemics” WithDr. Damon A. WilliamsNationally recognized DEI scholar and expert Dr. Damon A.Williams leads this discussion on Inclusive Excellence,strategic diversity leadership, and allyship in learningenvironments and in the workplace, and offers simple, yeteffective strategies for how leaders can level-up from bystanderto up-stander.

YouTube Video Link https://www.youtube.com/watch?v=W9Bs9UwOqw8

“Powertilt: Examining Power,Influence, and the Myth ofMeritocracy” With DoctorsCatherine Ashcraft and BradMcLainLack of influence in key innovation processes results in what weterm a powertilt phenomenon — that is, a differentialdistribution of power and influence along lines of gender, race,and other intersecting social identities. We present findingsfrom our study examining what counts as power and how itoperates on technical teams.

YouTube Video Link https://www.youtube.com/watch?v=xMECqxQwPEU

“Harnessing Power for PositiveImpact” With Dr. JulieBattilanaIn this live talk, Dr. Julie Battilana discusses thefundamentals of power, debunks the common myths surrounding it,and reveals how to harness power for positive impact in our

lives and in the world.

YouTube Video Link https://www.youtube.com/watch?v=5HgsROCrPtY

NCWIT 2012 Summit – Plenary 1,Nora NewcombeNORA NEWCOMBE: Okay, well thank you very much. It’s really apleasure to be here. I remember well that panel where I metLucy. And, actually, there’s a picture of me circulating on theinternet [laughs] from that panel, which I gave a talk inNiagara Falls a few months ago and the person who introduced meselected that picture because she said it was so much betterthan the usual headshot because I looked very animated. [laughs]I guess I was pretty animated at that panel. So, let’s see if I,yeah. So, I called my talk Spatial Skills and Success in STEM, Iguess I didn’t know that I was suppose to somehow fit a C intoSTEM, although I’m not sure where you wanna put the C, [laughs]Thinking About Gender Differences. Basically, what I want to dois to give you, perhaps at greater length than I could onCapitol Hill and also updated by seven years, a version of thekinds of things I say when people ask me, “Are there in factsome “cognitive barriers to women succeeding in science, “andmath, and engineering, and computer science?” I think you getfurther by really going into the science here than you do byjust sort of indignantly saying, “Of course not.” Because thenthey can say, “Well, but what about this and what about that?”So the truth is a little bit more nuanced than just, “Oh, ofcourse not.” But the bottom line is of course not. [laughs] Sothe Spatial Intelligence and Learning Center that I’m PI of is

funded by the National Science Foundation. And I heard in theoverview that Lucy gave that 50% of NCWIT’s money comes fromNSF. So I guess we owe a big debt to the National ScienceFoundation here. It’s ending year six of a 10-year grant. Andthat will be it, it’s not the kind of thing that they’replanning to continue. They want to sort of seed the field andsee where it goes from there. But our purpose is to develop thescience of thinking about spatial thinking and to actually usethat to change educational practice. So we do think that, as youdo, that STEM skills are really important in the global economy.We do think that spatial skills are relevant to them, and I’lltell you a little bit about all that before I really dive intogender. Lucy found this difficulty, too. So I want to first saywhy spatial thinking is a sort of component of intelligence.When people say intelligence in the singular, some other peoplesay multiple intelligences, and there can be a lot of academicdebates about that that I don’t wanna go into. But a lot ofpeople, even people who want to use intelligence in thesingular, really acknowledge that there is a kind of thinkingthat you would want to label spatial. And I actually think thereare two rather different kinds of thinking that you would wantto label spatial, and they both have evolutionary roots. So it’simportant to see them in this kind of biological context. Onekind of spatial thinking is the kind that supports navigation.How do we get around in the world? How do we avoid predators? Wedon’t think much in terms of predators these days, but inevolutionary adaptation we had to think about that. How do wefind our food? How do we find our home? And this is a problemthat we share with any organism that moves around in the world.So ants have this same problem, and they actually are quite goodat spatial navigation. Birds, they do incredible jobs ofmigrating and so forth. So in that context, homo sapiens hasinvolved certain particular kinds of adaptation that aredifferent from the adaptations of other species. For instance,

I’ve often wished that I had the magnetic compass that a birdhas. But humans, sad to say, don’t have a magnetic compass intheir nervous system. But we have our own sort of set ofprinciples for doing this. So Barron’s stressing though thecross-species comparison, because lots of species have to dothis. But there’s a different kind of spatial thinking that isinvolved with tool use. And tool use is really one of thecorners that our species has on the evolutionary market. So Ihave a picture there of a chimp using a stick to fish out atermite. And, yes, they do do that. And that’s kind of culturaland it seems to be culturally transmitted. But it’s really veryhumble compared to what we do. They don’t, for instance, fashionthe stick to be a better termite piercer than it could benaturally. So the kinds of things that we do when we chip awayand make an arrowhead and attach it to a shaft and figure outhow to launch it, this is really unique. And I use the termadvisedly in its proper meaning. It involves things that we dowith our hands to objects. And different parts of the brain, byand large, are involved in doing this kind of spatial thinkingthat are involved navigation. Actually, people sometimes talkabout gender differences in both these kinds of areas, but mostof the data that I’m gonna be talking about focus on the,focuses, focus, yes, because data is plural, [laughs] on thetool use kind of thing. This continues to be true in modernlife. It’s not just true for those long ago ancestors. We stillneed to figure out where to go. Now we use symbolic means. Wetry to interpret these very complicated signs. In terms oftools, we’re not using arrows that much anymore, but we do use,for instance, power Tools to put together furniture. And weeducate our children using spatial tools. If you’re learningabout the various layers in the canopy in the jungle, you make adiagram, and if you look at it upside down, you may or may notbe able to read the label as that girl is doing. But these arethe kinds of things the are supporting our learning about the

world. And our use not only of tools for navigation, tools thatare tools, but also tools to learn about other things such aslayers in the jungle canopy. Not only is it important and sortof general everyday life, but spatial skills are really veryrelevant to the STEM disciplines. Here, what I have up is just apanoply of diagrams and figures from various of the sciences,whether from biology or from physics or from engineering andtechnology, or from geoscience, or from geography. We layout alot of information in spatial ways. And we use this kind ofthinking to learn from others. And also it’s relevant indiscovery. So when Watson and Crick came up with the structureof the DNA molecule, they were working with flat X-raydiffraction patterns that they got from a woman, that they gotfrom Rosalind Franklin, but they were trying to imagine howthose flat patterns gave them information about a three-dimensional structure. This is the kind of thing that peoplecommonly cite when they say that spatial thinking is important.But psychologists always want data. We actually do have somereally impressive data, I think, about how spatial skillspredict success in the STEM disciplines. These data are takenfrom a longitudinal study that was launched in the 1950s whenhundreds of thousands of American high school students took awide array of cognitive tests. And they’ve been followed overtime. I mean, most of them are now retired or retiring. We knowquite a lot about what disciplines they went into and what theircareers look like. Now, because there are lots of kinds ofintelligence, and verbal and mathematical intelligence aredistinct kinds of intelligence that obviously contribute tosuccess, there was a statistical analysis done in which the teamremoved the contribution of verbal and mathematical intelligencein order to leave behind the extra contribution that spatialintelligence made in predicting whether or not these high schoolstudents would go into various disciplines. And what you cansee, whoops, when I don’t want that to happen, it does happen.

What you can see in the bottom two bars is that both forengineering, so in terms of SWI, and for math and computerscience which got lumped together, you were much more likely togo into those disciplines if you were high in spatial skill.That was also true for physical science. And then it reallydiminishes as you go out towards biological science andmedicine. Distressingly enough, education goes to the left. Sothe people who are teaching K-12 or, a lot of those are probablyfrankly carried by K-8, are people who are not as confident inspatial skill. And that may be part of the puzzle of what kindsof support they’re gonna need in introducing this kind ofthinking to our children early. So the kinds of spatial teststhat were given to all these teenagers in the 1950s were thefollowing. There were four of them. The top one, which is kindof small, is a paper folding test where you have to imagine apiece of paper being folded along certain dotted lines, and thatit can form one of those three-dimensional pictures that you seeon the right. And you have to check the box that shows which oneis the correct answer. There’s mental rotation. The second rowshows a flat, two-dimensional mental rotation task where youtake the design on the left and you have to imagine which of thedesigns on the right could be made by a rotation in the plane.You’re not allowed to flip it. There’s also three-dimensionalmental rotation tests, and I’ll show you one of those in aminute. Then there’s a mechanical reasoning test where you haveto think about gears, and if one moves, how will the other onemove? And then there’s an analogies task where if one shape isto the other shape, is the other shape is to the other shape,then the third shape is to which of the bottom ones? So that’sinference, but it’s about spatial material. Now, interestingly,two of these tests show sex differences. Two of the four. Sowhen anyone tells you, “Oh, there’s spatial differences.” One ofthe first things to say is, well, sometimes yes and sometimesno. So it depends on the test. And this is actually one of the

scientific puzzles here. Why do some of the test show sexdifferences and others don’t? There aren’t any sex differenceseither on the bottom test, the inference task, or on the toptask, the paper folding task. On the middle two, the mentalrotation and the mechanical reasoning, there are. And they’requite large, too. Mechanical reasoning can be written off to,you know, there’s a lot of content here, maybe boys have spent alot more time with actual gears, you know, taking apart thingsand putting them back together. It’s a little bit moremysterious when it comes to mental rotation. I wanted to laythat out. And then I wanted to tell you about some research thatwas actually done here at the University of Chicago that tellsus that there are similar findings even back in early childhood.That is, that the spatial skill of young children predicts theirmathematical skill, not decades later as was true in the ProjectTalent study which is the one I just told you about, but severalyears later. So the children took that test that you can see onthe left where you imagine the two pieces that are at the toprotating and moving together. And then you have to select whichof those four designs could be made by moving those four piecestogether. And then, there. So the spatial skill measured at agefive does predict the math task, it was something calledapproximate symbolic calculation. You don’t really have tosuccessfully do the calculation, you have to be able to estimateit three years later at age eight. Now, that relationship is,actually, mediated through knowledge of the number line. So lestyou think that this is completely occult and how would thiswork, let me remind you that one of the most basic mathematicaltools we have is the idea that numbers are arrayed on a line,that addition moves things rightwards, that subtraction movesthings leftwards. And that’s a spatial skill as well as anumerical skill. So being able to think that way is themediating factor that allows you to predict the approximatesymbolic calculation task. In fact, when you do the proper

statistic, if I could ever get it up, there it is, the directline from spatial skill to approximate symbolic calculationdrops out. So it’s all mediated through the number line. Interms of gender differences, on the same task, the one I justshowed you, we once thought that they didn’t begin untiladolescence. And if they didn’t begin until adolescence, somepeople thought, “Oh, that shows it’s hormonal.” Other peoplesaid, “Oh, no, that’s because of intensifying “sexual pressureswhen you enter adolescence “and you start to date.” But forbetter or for worse, I mean, those were two explanations, butneither of them can possibly be true because it turns out thatgender differences, actually, are evident early. So I show herethe full distribution, not just the average, because I do wantto emphasize that any particular girl may well be better thanany particular boy. So there’s a great deal of overlap in thesedistributions. It’s not the case that you can make predictionsabout any individual person. And that’s a very important fact tokeep in mind. Nevertheless, on the whole, it is true that theboys tend to score higher and the girls are overrepresented atsome of the lower scores. And we also have found this in secondgrade, second and third garde on another mental rotation test.And this is, actually, pooled with a map reading task, but theresults were identical for both of them. Now, one of the thingsthat this graph shows you is that there are sex differences inthe high SES group, socioeconomic status, and in the middlegroup, although they’re a little smaller but they are stillsignificant. There, actually, are no sex differences in thelower SES group. So it’s interesting that that’s true. We don’tknow quite why that is. But it’s a fact that does lead one tothink that some of this may be environmental. But I also want toemphasize that the sad fact here is that the low SES kids, thosedata are from the spring of third grade. Whereas, for the highand middle kids, those data are from the fall of second grade.So what this suggests is that it takes a year to a year and a

half before lower status kids catch up with their higher statuscounterparts. So there are social status differences in thisskill to worry about as well as gender differences in terms ofsocial equity. Now, going along with the theme that, yes, thereare some sex differences, but, no, there is not a sex differencein everything, another area of spatial thinking where peoplehave found very robust substantial sex differences ishorizontality and vertically. Now, you may realize that themiddle photo is photoshopped. [laughs] You couldn’t actuallytake that photograph. But not everyone really seems to realizethat that’s true, that the beer stein on the left, whichactually has a weird sort of bottom is why it can bephotographed stably without a hand holding it. But if you askeven adult men and women to draw a horizontal water line in aglass, the women, it’s not that they totally don’t know where itgoes, but they draw it more than five or more than 10 degreesoff. So they’re unable to disregard the surrounding sort ofinfluence of the glass to sort of orient themselves to the truesort of gravitationally defined, this is vertical, this ishorizontal. And this is also true, my colleagues and I havefound recently, for perception of slope under your feet. So whenwe put you in a room whose floor is tilted by five degrees,which is a fairly appreciable slope. It’s about what awheelchair ramp is sloped at. And there aren’t any other cuesfor telling your way around the room. And we spin you aroundblindfolded and disorient you. Women are less able than men touse the feeling of the ground under their feet to reorient, totell, basically, what’s uphill and what’s downhill tells mewhere something is hidden in the room because we had hiddensomething in the room that we had them look for. So it’s a realsex difference. But there’s another interesting absence of a sexdifference to something that I think is very, verytechnologically relevant, namely cross-sectioning. This is veryimportant in the geosciences which are very relevant to things

like finding oil, that people in Washington, I think, careabout. And both with a test that we devised to give to youngerkids which you see on the left and a more complicated task whichyou see on the right, which can be given to undergraduates,people are not always good at this. In fact, one of the problemsis that people can quite often be quite bad at them. But thereare no sex differences. So men and women are the same. Sothere’s a very interesting picture here. From a scientific pointof view, it’s really pretty intriguing. Don’t ask me why we havethis picture, it’s really a work in progress. And I will say alittle bit more about this. But I think people who either saythere aren’t any sex differences, or they’re everywhere, they’reboth wrong. One of the most important things that I wanna getacross in this talk is that when Lucy used the word innate,innate often, to a lot of people, means fixed. Now, it doesn’tactually technically mean fixed. You can have something presentat birth and you can change it. But a lot of people think innatemeans fixed. And if you think that spatial skills are fixed,that is not true. Spatial skills can be improved. I did a meta-analysis which is a statistical literature review thataggregates findings a long time ago that showed this. Subsequentto that, I’ve done a number of training studies that show howmuch, in this case, mental rotation is what I’m graphing there,how much it can be improved. Now, the top line is in blue, andthat’s men. And they can improve a lot. I mean, they start outhigh, and that doesn’t mean they’re at ceiling performance. Ifyou give them continued training, they still get better. So thisis not a skill that we’re maximizing in our society. The solidpink line is for women, who start high, we selected some womenwho start high. And then the dotted pink line is for women whostart low. The women who start high also improve. And the womenwho start low also improve. Now there is a difference in therate. So the people who start high, the men and women, improvemore quickly at first and then a little less quickly later.

Whereas, the women and men who start low… the women, we,actually, for some reasons that I won’t go into couldn’t get menwho started low. Damn. They improved slowly at first and thenthey get better much more quickly. Oh, you know, the thing isthat you have to be here, not where I thought, I think. Yeah,okay. So the big deal here is that if you were a woman who havelow spatial skill, why did you come back and do this? You’re badat it, you are bad at it a week later, you’re bad at it twoweeks later. Well, the answer is we paid you to do this.[laughs] That’s what happened in the study. But the message for,you know, situations where we can’t necessarily pay people totry to get better is that the motivation is very important. Andthe faith that indeed they’ll reach the sweet spot of theirlearning curve and that they will start to get better is veryimportant. Now, we’ve recently done another meta-analysis, whichis now in press, which continues to show that overall there arethese big effects. And which shows that they are present formental rotation, which is the second bar from the left, and alsofor the horizontality, verticality which is the bar on theright. In fact, the bar on the right is the highest bar thereis. So even the two areas that I told you about that have sexdifferences could definitely be improved. Now, in terms ofpossible effects, I’ve illustrated this for engineering, but, asyou saw, the effect size for spatial skill fostering going intocomputer science is just about as large as for engineering. Sothis is a hypothetical set of data that put together two datasets. First of all, we took from the longitudinal project talentstudy the thresholds for where we set, “You have enough spatialskill, “you could become an engineer.” It doesn’t mean you haveto become an engineer, but were you to want to, you wouldprobably be gifted enough at this kind of thinking that youwould succeed. Then we right shifted that distribution by asmuch as we know from the meta-analysis. We can actually rightshift it. So how much could be actually improved. Therefore, at

the light area under the curve is the proportion of thepopulation eligible to become engineers if we don’t foster theseskills. Whereas, the dark gray area is the proportion that wecould add to the potential pool of people who could go intoeither engineering or computer science were we to foster thesespatial skills K-12. And since we, actually, do need moreengineers and we do need more computer scientists as Lucymentioned, I think this is of national importance. And we alsoknow this experimentally. Those of you who are fuzzy aboutexperimental design are probably only 2% of the audience, butjust for those of you who are, you may have realized that I’vepresented correlational data. And even though it’s longitudinaldata and it’s controlled for a bunch of stuff, it’s still notexperimental data. We’re beginning to accumulate findings whichdo the very, very, very toughest test, which is to take people,randomly assign them to being trained in spatial thinking or tobe doing something else, and see if we can get them by virtue ofhaving the spatial training to have hard endpoints like bettergrades in chemistry, or geoscience, or physics, or calculus. Andalthough there aren’t as many studies as one would like, it looklike that is actually true. And there are also similar suchexperimental findings in children. But you’ll note that both ofthese are under review. So this is a new enterprise and one wewanna press on just to make sure of our facts, but we think it’sgonna turn out to be true. So now we want to get to the issue ofwhether sex differences are biological. And I have a photo ofLarry Summers whose become very convenient as a whipping boy.[laughs] Still on the slides seven years later. Because he didsuggest, inspired, I think, by Steve Pinker who was at Harvardand fed him a reading list, that spatial skills were biological.And for him that was correlated with immutability, which I justargued was false, and was supported, he thought, by hormonal andneurological evidence, and which is supposedly predicted byevolutionary theory. So I think I just disputed the first of

those points. And I want to tell you a little bit about thesecond and then more about what I think are some incrediblecontradictions in the third point. So just briefly in terms ofthe second point, I’m not even gonna talk about the neurologicaldata because they’re very weak. I really don’t think there’s ashred of clear evidence for neurological differences that wouldlead to these kinds of differences. There are some interestinghormonal findings, and I would not want to sweep them under thecarpet. But they’re very, very messy. You see them, then youdone see them, it depends on the methodology. But moreimportant, and this is actually in a way also anticipating whatI’m gonna say about evolution, they’re odd from the point ofview of, if you’re a woman, it seems that spatial skill isstronger at times, for instance in the menstrual cycle, when youhave higher levels of testosterone or testoronizing type malehormones. I don’t wanna get into technicalities of hormones.Basically, when you’re menstruating, some of these skills tendto be a little higher than when you’re ovulating. On the otherside, among men, it’s actually better to have less testosterone.I mean, I don’t know if this is like a clue for people hiringmale computer scientists that, you know, [audience laughs] heavybeards, forget it. [laughs] The point I’m trying to make is,it’s very odd from the point of you of, if you’re trying to usethis to predict sex differences in an evolutionary framework,why would it go up with testosterone within the range oftestosterone that you see in women and down with testosteronewithin the range that you see in men? You’d have to explain thatin order to have a really consistent picture. And it isn’tthere. So the explanations really get to be very convoluted. Sothe remainder of the talk, I mainly wanna talk about anevolutionary approach and the idea that that somehow implieshardwired sex differences. And I am not a person who denies thatthere are some real sex differences. I mean, clearly humanfemales lactate and human males don’t lactate. And there can be

indirect effects of that. So in some cases, anthropologicallyand cross-culturally, that has led to some sex type divisions oflabor that are less in our society because we have fewer kidsand we don’t always nurse them, and so on and so forth. But insubsistence societies, it can be correlated with certain kindsof sex type divisions of labor. But what is true for spatialability? There are, basically, two families of evolutionarytheories that are pretty popular. And you probably alreadythought of them. One is that Man the Hunter. Well, aren’t menhigher at these skills because they have to hunt and they haveto aim and they have to track, and they have to make tools inorder to have something to aim with. So there’s several thingsthat are kind of correlated with hunting. And there’s also adifferent story which is actually a little bit more consistentwith sociobiology. Although Man the Hunter, well, they sort ofinteract. But sociobiologists mainly care about how many genesyou leave to succeeding generations. So how many kids do youhave. The idea is, spatial ability must somehow help men toleave more genes in the succeeding generations. You must somehowhave more children if you have higher spatial ability. So thisis a little weird-sounding. But let me tell you about theresearch that leads people to think that it’s actually true.There are two kinds of a certain rodent, that you may or may notheard of called the vole. That’s a picture of a vole. It looks alittle like a rat, but it doesn’t have a tail. So it’s cuterbecause rat tail [audience laughs] is really obnoxious. Andthere’s two kinds of voles, the prairie vole and the meadowvole. The prairie vole is pair bonded. They just find a mate andthey live together and they make kids, and that’s great. Andthey have equal spatial ability, which is tested, by the way,using a navigation test. Obviously, voles don’t use tools. Soyou put them in a maze and see how they can get around, andthey’re equal. The meadow vole is polygynous, which means thefemale vole stakes out a territory during the mating season very

separate from the territory stake out by some other female vole.And the male voles wander around trying to mate with as many ofthe female voles, within a limited period of time while they’reovulating, as they possibly can. So this is actually anavigational challenge because you have to basically calculate[laughs] the least distance to get you to as many female voles,within a limited time, as possible. And they do have higherspatial ability during the mating season only. And they actuallyeven have a larger hippocampus, which is the part of the brainthat supports navigation, at the mating season only. So it’sactually a really interesting set of data if you care aboutabout voles, but [audience laughs] the issue is, really, youknow, to what degree this can be generalized. So let me firsttake Man the Hunter. And the idea of aiming, well, we saw thatmental rotation is one of the biggest sex differences there is.Another one is this horizontality, verticality thing. Anotherone is slope perception. But aiming has nothing to do with that.In fact, people have actually tried to correlate aiming withmental rotation, and they aren’t correlated. So you could justas easily say that the things where women are equal to men arerelated to aiming. But aiming, actually, it’s basically asensory motor kind of capacity, which has nothing much, as faras I can tell, to do with spatial skill. There’s also the ideathat it has to do with tracking where you go around in the worldand you figure out, follow the animals, and you end up maybequite a ways away from where you started. And then you also haveto figure out how to get home. But the fundamental assumptionhere is that gathering, which is what women do in these kinds ofhunter-gatherer societies, doesn’t involve having to get around.So the sort of mental picture that people have of these groups,I think, is that women kind of roll out of the huts in themorning and the blackberries or whatever are right there andthey just kinda gather them. [laughs] Well, the blackberriescould be a long way off, too. In fact, studies of primeval

hunter-gatherer societies today suggest you may have to wanderquite a lot to be able to gather. Furthermore, we don’t knowthat our ancestors hunted in the way that’s envisioned this way.And there’s actually quite a lot of evidence that they huntedinstead by, you know, dig a hole, wait for an animal to fallinto it. [laughs] Go to the stream, wait for them to drink thenshoot them. These are smart things that homo sapiens is good at,but they don’t involve spatial skill. And for toolmaking, all Ihave to say is that women in hunter-gatherer societies maketools, too. They weave, they make baskets, they produce pottery.So there is some sex typing in what you produce, but I can’t seethat making a pot is less spatial than making an arrowhead. So Ikind of don’t get what’s going on here. In terms of thehypothesis, the man who gets around, the vole data are fine.But, as I said, there’s the issue of generalization. So the bigdeal here, I think, is that humans live in social groups. Sosidestepping the issue, up to what extent we are or are not pairbonded, to which the answer actually seems to be, you know, it’san enduring tragedy that we’re kinda, sort of pair bonded butsort of imperfectly. So there’s a lot of extra mate pairings.[audience laughs] 5% of children are estimated not to befathered by their official father. But whatever the constraintis on fathering children out of wedlock, which is asociobiologically good thing to do, it’s not finding the woman.If you live in a village, finding her is not the problem that[mumbles]. Getting her to agree could be a problem, or evadingthe husband who’s jealous could be a problem. But not findingher. So I don’t think that that makes sense. In fact, as I said,we’re probably only mildly pair bonded. So unlike the gorilla,we don’t have a real polygynous mating system. And there’s alsosome sociobiologists who talk about, well, there’s femalechoice. So we might select someone to father our children ifthey’re better hunters or if they can beat out other men, and,you know, bring home the mammoth or whatever the primeval

equivalent of bringing home the bacon was. But that just getsback to me, back to my story about why Man the Hunter doesn’twork. So, basically, there’s no real reason to suppose thatspatial ability is confined to men for evolutionary reasons.Most things that are sex differentiated in this way. So the liongrows a mane, the male lion grows a mane. The peacock grows atail. Having a mane is a drag if you live in Africa. There’s acost. Having a tail is a real drag literally if you’re fleeingfrom an enemy. So, typically, these kinds of displays that maleshave also have costs. It’s an interesting fact. But sincethere’s no obvious cost to spatial skill, this also doesn’t fitthe evolutionary picture. I thing I’m gonna skip this because itgets a little bit too much into the sociobiology stuff and justconclude that if sex differences in mental rotation have abiological cause, their evolutionary function is either notcurrently understood or maybe it’s nonexistent. I’m not rulingit out, but I think the evolutionary people have a lot to do toshore up their story before it would really be believable. Butwhat I wanna leave you with is the following. The idea that thecausation of sex differences. Well, I agree that it’sscientifically interesting. And, on occasion, I have doneresearch relevant to it. It’s not the socially importantquestion. The socially important question is not whether thisdifference is in any way innate, or in any way biological, or inany way hormonal. The important fact is that it is not a fixedability, that there is potential to benefit from experience,that there can be vast improvement for both the sexes. So theimportant fact is that we can bring both men and women more intothat band that I showed you before where they are perfectly ableto choose a technological discipline if they choose. Now, one ofthe barriers to that is anxiety. Now, I just wanted to mentionthis because I showed you that leftward bar for the elementaryschool teachers. Elementary school teachers have a lot ofspatial anxiety, but one of the things we have done in my center

is to show that it can be reduced. So this is the kind of areawhere cognitive psychology meets social psychology. And I knowlast year you had a plenary address from Josh Harrison. So thesetwo fields are both part of the jigsaw puzzle. So overall,spatial skills are important to STEM learning, they can beimproved, the causes of sex differences are not well understood,and more research is needed but it’s not, I don’t think relevantto the socially important fact that improving spatial skillscould help to reduce gender and SES inequities in the STEMpipeline. And that’s not to say that other factors aren’trelevant as well, such as work-family balance. I think this ispart of the picture. Thank you. [audience applauds] I left alittle less time than I would have liked, but, yeah.

AUDIENCE MEMBER: I just got the microphone, hi.

NORA NEWCOMBE: Oh, there you are, hi.

AUDIENCE MEMBER: Right here. Thank you for a really fascinatingsharing of your research. So it’s about this same time, 1989,when one of your first analysis, I think, were on the slidesabout gender differences and spatial cognition. One of mycolleagues at MIT Media Lab ran a study on gender differencesand spatial cognition and correlated it to elementary schoolboys and girls, low income, African Americans and Hispanic kidsin their ability to do 3D programming. And I’m bringing this inbecause I think it really relates to what we are about here. Andso what she showed is that indeed where some things that had nodifferences between boys and girls, but on some test did show adifference. But she also demonstrated, and I know, it was 30years ago. I don’t really remember the exact details, but shedid demonstrate that learning how to program, how to code, itwas Logo programming language but also 3D space that she createdto program in 3D, actually was a form of training to eliminatethe gender differences. And so while we see in your slides this

pretty fascinating correlation between people who have strongspatial cognition with their attraction to engineering andcomputer science, I think that research from back then alsodemonstrated, her name is Judy Saf-ter. Her research at MIT, herPhD thesis showed that if we will get kids very early on toprogram, especially 3D programming, we can actually increase notjust their engagement in computing and programming but alsotheir spatial cognition that will eventually make them betterprogrammers, which is kind of like what we’re saying. Do youknow anything about that research? Did anybody carry on that?

NORA NEWCOMBE: I don’t know about that particular study, and ifyou could remember the name or send me the reference, I’d loveto get it. Part of the work of our center is in fact to come upwith a variety of means to improve spatial skills both in littlekids and in undergraduates and everything pretty much inbetween. And we have a lot of good tricks, and, you know, if Ihad yet another talk to give, I could talk about, you know, justsome homely things like play with blocks and do jigsaw puzzlesand so on and so forth. But the idea that you get kids to do 3Dprogramming, that’s fascinating, that’s great.

AUDIENCE MEMBER: This is the area that I’m interested in also isthe, you’re defining spatial skill as relating to the actualenvisioning of objects or molecules or mechanical drawings thatrepresent reality. Is that correct?

NORA NEWCOMBE: Mmm hmm.

AUDIENCE MEMBER: So aside from the use of the number line,you’re not using visual aids to try to model a mental processlike programming. So 3D programming aside, I’m interested in thecorrelation between spatial skills as you define them and theability to produce programs, computer programs, knowing thatfrequently the representation of an algorithm is a visual

representation. And in my experience of teaching computerscience, generally I find I’m a tremendous visual learner. Manyof my best students are visual learners. And whether it’s a URLor a flowchart or a data transfer diagram or whatever, a visualrepresentation of an algorithm generally shows the mastery ofthat algorithm, so the correlation between spatial skills andthat ability.

NORA NEWCOMBE: Right. Now that I understand what you’re saying,I think I totally agree. One of the slides that I sometimes showand didn’t show in this presentation actually makes the pointthat the spatialization of non-spatial information is a veryimportant learning tool. So every time we produce any kind ofgraph or any kind of flowchart, we can be putting downinherently non-spatial information such as, you know, economicinformation, how much people earn over time, or this kind ofthing, and spatializing it. And it’s often much easier to graspwhat’s happening from those kinds of representations than it isfrom words. I’m actually a great believer in those kinds of datadisplays. I don’t know much about programming or teachingprogramming, but I do think that the fact that, in thelongitudinal study there was a prediction to computer science,tells us there must be some kind of link of that kind becausewhy else are you getting it? So I would like to know more aboutthe linkage you see in teaching computer science, but yes.

AUDIENCE MEMBER: I enjoyed your talk very much. I actually havea quick comment and a question to follow up. I’m Catherine Di-de-an, and I was actually at the talk that Larry Summers gave.And he had actually three points. One was this is yourmathematical ability, and that the fact that young men tested atmuch higher level, at the higher levels than young women did.But his other two arguments were that women did not want highpower jobs because they did not want to work more than 40 hoursa week and that there were social and cultural differences that

excluded them. But what’s interesting is actually if you look atthe data that was presented at that meeting in January 14th,that there, actually, even the women who test high in thoselevels of mathematical or spatial, roughly less than a third ofthem go on to computer science and engineering. So actuallyhaving that spatial ability and testing well doesn’t necessarilypredict you going into the field. And there actually are ethnicand racial differences among certain groups. Like Asians, youwon’t see the differences within gender than you do inCaucasians. And sometimes you don’t see them in other ethnic andracial group. So I think this is a very interesting topic, but Ithink we need to acknowledge that there may be some cultural andsocial expectations that can influence people even who have thatinterest that go beyond that. And I just wanted your response tothat.

NORA NEWCOMBE: Well, two points. One, just about the ProjectTalent data. It actually does work in Project Talent when youseparate the men from the women. So you find the same kind ofthreshold effects and the same kind of prediction that you findfor women as for men. Now, the more important point that youmake, I think, that actually Larry Summers was also making in asomewhat ham-handed fashion, is something that I eluded to atthe end when I mentioned work-family conflict. I do think thatthe threshold that’s posed by spatial skills is really just akind of threshold or gatekeeper kind of effect. There are othergatekeeper effects that I did not mention, one of which I reallybelieve in although I haven’t personally examined the data, butI think there are data there that women tend to see thesedisciplines as not helpful and not social. And it’s veryimportant, I think, and you guys probably know more about thisthan I do, to contextual, you know, if your bridge falls down,people die. This is like socially bad for people or, I’m notquite sure how you do it, but. So there are other gatekeeping

factors. In other words, you don’t wanna say that any one factoris the only factor. And, recently, Steve Ceci and Wendy Williamsat Cornell have been making the argument that the work-familyconflicts are really the key. And I don’t think that’s correct.I think they’re very important, I just don’t think they’re theonly thing that’s going on.

AUDIENCE MEMBER: My first impression is, first of all, why ismental rotation important? And when I think about it, I wonderif there’s data and whether or not people who do well on mentalrotation, or don’t do well on mental rotation, do well whenthey’re dealing with the kinesthetic problem. So when they havea 3D thing in their hand, is there data that shows that they domuch better with spatial knowledge when they’re working with themodel that’s in 3D rather than doing something mentally? And iteven made me think of even auditory spatial awareness, thinkingof the distance between musical notes or something like that andhow those types of things might be different if they were lookedat across genders.

NORA NEWCOMBE: Right, right. I don’t know about the auditorypart, but people in general do much better at these tests ifthey do have kinesthetic information. Interestingly, that’s notjust because when you turn something, oh, you know, the answerpops out at you. You can give something to somebody to hold intheir hand and it’s under the table. So you can’t actually seeit. But the fact that you are turning it sort of drives part ofthe motor cortex of the brain to help you to mentally turn itand to get the correct answer. In fact, when people do thesemental rotation problems, in echo MRI setup so we know whichbits of their brain are lighting up. You often, although notalways, get motor cortex involvement. In fact, the kinestheticcomponent of doing mental rotation or any other of these mentaltransformations is actually very important. But the first partof your question, which is why is it important at all? I didn’t

go through the many, many correlational studies, I just lookedat the Project Talent one because it’s the most impressivebecause it’s really longterm longitudinal. But there’s a lot ofcorrelational studies that do suggest that it’s correlated withsuccess not only in the STEM disciplines but also, you know,things we might care about in, you know, a different kind of waylike being a good dentist or being a good surgeon, or thesekinds of things. So yeah, it is correlated.

AUDIENCE MEMBER: Fantastic talk, thank you for it. You didn’tspend much time talking about transfer. As I understand spatialabilities, for a long time we thought you could practice, let’ssay Tetris, it would improve your score on Tetris, but itwouldn’t transfer to any other spatial. What was thebreakthrough on that and what have we learned about creatingtransfer from one domain to another from your work?

NORA NEWCOMBE: Well, the breakthrough was… I’m not sure theconclusion that you couldn’t get transfer was ever correct.[laughs] In other words, I think you always could get transfer,there were just some studies that were widely cited that failedto find transfer. In those studies, there are actually reasonsfor that. In some cases, they didn’t train very much, so it justwasn’t enough training. An interesting reason, which you’llprobably appreciate more than most other people in the audience,has to do with the nature of your control group. So it has to dowith what your quote, unquote non-spatial activity is. And italso has to do with how many spatial tests you give as pretestand your post test. Because taking the test itself is a form oftraining. So it’s hard to get that kind of transfer if what youhave to beat is a control group that is also getting trainingthrough taking the test. And we actually rigorously show this inour meta-analysis that’s in press. So I think that’s animportant methodological point. Retesting effects in this areaare actually enormous. A psychologist called Tim Salthouse has

compared, psychologists are always worried about retestingeffects, and they’re bigger in the spatial domain than in anyother domain. So you have to worry about it more there. Okay,well, thank you very much. [audience applauds]

Vimeo ID 45873134

2013 NCWIT Summit – PlenaryIII, “Habit Disruption: InSearch of How People Change”by David Neal[upbeat music]

JANE MARGOLIS: David Neal is a social and behavioralpsychologist specializing in behavior change. His academicresearch focuses on understanding basic mechanisms of habitualcontrol in daily life including the influence of environmentalcues, planning, and self control. He received his PhD from theUniversity of Melbourne and completed his post doctoral trainingat Duke where he was the director of social science researchlaboratories. David is published broadly in the areas ofbehavioral change, attitudes, and motivational processes andconsumer and social decision making. His academic research hasbeen featured in the New York Times, Time magazine, the BBC,NPR, The Independent, and The Australian. Additionally, he’sbeen working with Microsoft on habit-changing and technologyuse. He’s worked with a surgeon general on health behavior in

the U.S. military and with the Australian government oncommunity based health behavior change campaigns. I’m reallylooking forward to hearing his talk. It very much builds on theinvestigation that we’ve been doing on stereotype threat and inthe SSAB, the Social Science Advisory Board, we’ve been lookingat the whole issue of social change, how does change happen andwhat is the relationship between systemic change that musthappen and individual change and how do the two converge? Welook forward to this talk and just know that a lot of us in theaudience are wrestling with that kind of tension of these twodifferent approached to social change. So welcome, David.[applause] [upbeat music]

DAVID NEAL: Wow, hi everybody. I feel like I should sing now.Thank you so much for the invitation to come and talk to youtoday. As Jamie mentioned, I am a psychologist and the issuethat I, and Wendy Wood is my mentor, my collaborator in most ofthe work that I’ll tell you about. The thing that we’ve beenreally interested in is how people can successfully change,especially really entrenched behaviors, things that they’ve beendoing for a long time, things that are like second nature tothem. What I’m gonna try and do in this talk is sort of expand alittle bit back from gender and IT and look more broadly at thesort of the psychological machinery if you like that determineswhether or not human beings at the individual level and then atthe group level are successful at changing their behavior. Letme start by putting it in context with some kind of data that Ifind kind of paradox going on, I’ll see whether you feel thesame way. Gallop has been, since the 1950s, been doing thissurvey where they ask whether you’d prefer to have a man or awoman as a boss. Here’s the data from the 1950s. Oop, sorry.What you can see is a steady decline over time in the number ofpeople who say they would want a man for a boss and a steadyincrease in the number of who said they would rather have a

woman as a boss, and now the largest single group is people whosay they have no opinion, no preference. Obviously haven’treached the state of full parity, but it’s pretty evident thatover the past 60 years, there’s been a significant positivetrajectory in people’s attitudes to women in leadership. Nowlet’s look at outcome data. Let’s look at the actual percentageof women and men in these leadership roles in the United States,and this is 2013. We’ve reached about parity for all managersand professionals, this is using the Bureau of Labor Statisticsnumbers. With managers there’s still only 39% women. With allCEOs, 27%. 16% for F500 board members. 14% for corporateofficers, 8% for the top earners, and 4% for CEOs. As you know,other domains of leadership like politics, the data is similar.I think that the U.S. currently rates 77th in women elected tofederal legislatures around the world which puts the UnitedStates I think just slightly ahead of Madagascar and the UnitedArab Emirates. [audience laughs] The thing that is weird to meabout this as a psychologist and also as a human being is whywould it be the case that there’s been such positive shift onthe sort of attitude, value metrics but the outcomes haven’tshifted much at all? There’s a kind of a disjunction between thetwo. Of course there are many things that no doubt contribute tothis. First of all obviously, the Gallop data is people sort ofself report it, express conscious attitude and maybe in fact wedo know that people have unconscious attitudes as well that theymay not be reporting on in those kind of surveys. Anothercontributor might be the fact that those data from the generalpublic, but maybe the attitudes of decision makers who areactually making these employment decisions, maybe the attitudesof those people are different. Of course there are no doubtgenerational effects where maybe it takes the outcomes longer tocatch up. An attitude can change quickly, someone’s movement upthe corporate ladder is a much more time lagged phenomenon. Thenof course there’s other sort of structural variables like

traditional gender roles and workplaces and organizational rolesthat are structured better to men’s lives than to women’s lives.All of those things no doubt are important and I don’t want togloss over those in any way but what I wanna do today is tofocus on what I think might be one other contributor to thisdisjunction. That is the idea that organizations are made up ofindividuals, individuals are creatures of habit, and we havemyriad sets of routines, rituals that have accumulated over timeand may partly explain why we have such a hard time marrying upour attitudes and our values with the behavioral outcomes thatwe care about. Here’s a kind of little graphic that sort ofbrings this into relief. The sort of basic thesis that I’m goingto argue today is that there is this large gap between ourattitudes and our intentions, our values and the behavioraloutcomes and that one of the things that causes that is habitsthat we’ve accumulated over time at an individual level, at aninstitutional level, at an organizational level. To get a littlebit more psychological about this, I want to introduce to youthis idea that in a sense we have two selves and I’ll kind ofrun with this idea throughout the talk. One self we have is thekind of what I’m gonna call the intentional self. This is thepart of our self that is guided actively by our attitudes, ourgoals, our values. It’s the more conscious side of ourselves.It’s the side of ourselves that has knowledge that we can easilytalk about and verbalize, and it’s the sort of control, sort ofknowledge that can change pretty quickly just my giving someonea role or by choosing to take on a role yourself. That’s thekind of intentional system. Then on the flip side, we’ve gothabitual system which we also rely on from time to time to driveour behavior. That system is guided very differently from theintentional system. It’s driven by cues, we’ll talk about whatthose cues might be later on. It’s less conscious, most of thetime we’re not thinking about what we’re doing when we’re actingout of habit. It’s not verbalizable, just like all of us perhaps

can ride a bicycle but if someone said “Could you explain howyou do it?” it’s very hard to verbalize that kind of implicitknowledge. The final pieces is that habit knowledge changes veryslowly and only through experience. Whereas you can give aperson a role to change the intentional self, the habitual selfreally only changes in the long term through sort of slow,gradual, accumulated experience. One of the really importantthings to note is that these two sort of sides of ourselves areoften locked in a battle to control behavior and the sort ofbalance of power switches the more we’ve done something. Soinitially, if you’re in a new role, in a new place, doing a newthing, the intentional system will be dominating control of yourbehavior but then the more you perform a behavior, the morefamiliar you are in an environment, the more opportunities thereare for you to have accumulated habits and the balance of powerwill switch over, so more of what you do is controlled by thishabitual self. This I think underlies some of the problems wesee in getting individuals and groups to bring their behaviorsin alignment with their attitudes and intentions. I want to usea little visual analogy to make this point. You may have seenthis visual illusion before and so if I asked you which one ofthose tables is longer, which would it be? Can anyone seeanything other than the one on the left being longer than theone on the right? Okay, they are the same, they are the same,but what do you see, what’s your visual experience? Doeseveryone see the one on the left as longer? Okay, let me helpyou out by putting a ruler on them and now let me put the rulernext to the other ruler to try and convince you that actuallythey’re the same length and then take that away and unlessyou’re different from me, you still can’t help but see the oneon the left as longer than the one on the right. This I think,as an analogy, is a way of thinking about this battle of willsbetween the intentional self and the habitual self. That in thiscase here, your sort of rational, conscious mind knows that the

tables are equal length, but the visual processing system is soautomated, is so reflexive, and it’s the one that’s controllingwhat you actually see. You can’t use your sort of conscious,rational self to make yourself see these tables as they actuallyare. You’re forced to see them as that unconscious, habitualsystem, vision system, forces you to see them. What I’m going toargue is that this to some degree is analogous, the situation’snot quite so bad, but at some degree analogous to the problem wehave when we try to just focus on changing hearts and minds whenpeople’s behavior is stored in this other system. Let me justspend a little time convincing you that this, that phenomenonisn’t something that just works with visual illusions, itactually plays out in real people’s lives. I’m gonna show yousome data, this is from a big study called a meta analysis, it’sbasically a study where the authors get together a whole bunchof other studies and compute the average effect from thosestudies. The studies in this meta analysis all had one thing incommon and that is they were interested in trying to predictwhat a person would do in the future based on what they saidthey wanted to do, their intentions, their preferences, and thehabit strength of the behavior, meaning how long have they beenperforming that behavior, how frequently do they perform it?You’ve sort of got these, you can think about this as we wannasee whether the habitual self or the intentional self, who hascontrol of the reigns in predicting what someone does in thefuture? What you can see here is that first of all for behaviorsthat people don’t perform very often, so things like getting aflu shot or registering for class or donating blood, theintentional self controls what people do. These are correlationcoefficients so that basically tells you that there’s a muchstronger relationship between intentions and future behavior forthese actions that we don’t do very often. But now let’s look atthe pattern for behaviors that people perform really frequentlyand especially if they do them in the same environment. So

things like exercise and seatbelt use. Suddenly the balance ofpower gets flipped around. The habitual self, and that basicallyjust means the sheer number and contextual stability with whichyou’ve done the action in the past, all right, have you done ita lot and when you do it, do you do it in the same environment?That is the thing that is predicting whether you’ll do it in thefuture and whether you say “Yes, I’m absolutely going to go anddonate blood,” or “No, I’m definitely not gonna donate blood,”that contributes very little predicatively to whether you go anddonate blood. This is also as you can imagine, a major problemwhen you wanna try and get in either with an individual or witha group of people to change behavior. What I’m gonna show younow is data from intervention studies that have taken a behaviorand tried to convince people through a sort of intentionalpathway, so providing lots of information, lots of facts, andtrying to get people motivationally on board with the idea ofchanging their behavior. In this study, what Webb and Sheerandid was broke up the studies into ones that were trying totackle things that we don’t do very often, so things like classregistration or blood donation versus things that we do reallyoften, like exercise. Then what they asked was well howeffective were these interventions, the ones that were trying totarget people’s motivations, their intentions, their knowledge,how effective were they for these two different classes ofstudies? What they found was that for the really infrequentbehaviors, stuff people don’t do very often, intention basedinterventions work. This is what’s called an effect size andanyone who would run an intervention and get an effect size of.74 would be really happy, they’d be like “This worked, “thisintervention changed people’s behavior.” On the flip side, forbehaviors that people perform really frequently in the samecontext, the effect size was dramatically smaller. The basictakeaway is that trying to tackle people’s intentions, armingthem with information, getting them motivationally on board will

work if the thing that you’re trying to change is something thatpeople do relatively infrequently but if they do it all thetime, been doing it a long time and in the same environment,changing intentions, providing knowledge will often have apretty small effect. You should be thoroughly depressed at thispoint about the scope to change people’s behavior, it was mygoal. What I wanna pivot to now is well, is there anything youcan do about it? The balance of power gets switched to habit andthen you can’t talk to people anymore, you can’t provide themwith information, you can’t motivationally engage them, it haslittle impact, so what can you do? Fortunately, there are thingsthat you can do and that’s what most of our research has focusedon. The first idea I wanna introduce to you is that contextchange, and these can be both small changes and large changes,can upset, can disrupt the cues that keep people locked in todoing what they’ve done in the past and create a little windowof opportunity for intentions to regain control of behavior.Here’s a little suggestive evidence to support this. This wasstudy that was done in Europe and what they were interested inwas the, this is a sort of environmental study looking atwhether or not people’s concern about the environment predictedwhether they used their car or not or the percentage of the timethat they used their car. These were the results they found forthe average person which is kind of sad, basically what it showsis that actually people who had high concern for theenvironment, that’s the green bar, had slightly higher car usethan people who had low concern for the environment. Not a veryhappy making result. That’s the average person. But then theyfound this other group who had the pattern flipped around whereif you have high concern for the environment, you’re in a greenbar there, you use your car much less than if you have lowconcern for the environment. So who are these people? Well turnsout they’re people who are the same as the average people, butpeople who recently moved house. The mere fact of moving to a

new environment, disrupting all of those cues associated withwhat you’ve done in the past was enough in this study to movepeople’s behavior to be more in alignment with their values andtheir attitudes. This study’s really interesting, it doesn’tspecifically look at habit so we’ve been, in our work, trying todrill down into this a little bit more sort of scientifically ina controlled way and I wanna share a couple of studies with youon that. I want you to engage in a little sort of imaginaryexercise with me and see whether you do the same thing that I dobut imagine that you’re, you’ve gone to the movies, it’s aSaturday night and you buy some popcorn on your way in and yousit there and the trailers start and before the trailers arefinished, all the popcorn is gone because if you’re like me,you’ve been mindlessly shoving it into your face and all thepeople around you are doing the same thing and ask yourself thequestion, well why on earth would we, why do we do that? Who’scontrolling our behavior when we engage in that? Is it theintentional self? Is it just that we’re so overwhelminglyinfluenced by the delicious, irresistible taste of the popcornthat we’ve got no choice but to eat or is it instead somethingabout the environment and the ritual and the practice of beingin that setting and association with eating that particularfood? That’s kind of what we were interested in in this study.What we did was got a bunch of people and then we randomlyassigned them to do one of two things. Watch movie trailers in acinema, just like the scenario I just described to you there orthe control group, watched music videos in a large, darkenedroom. Think about these two groups of people, they’re bothsitting in large, darkened rooms, but one of them, it’s anenvironment that the mind associates with consuming popcorn,that is a movie cinema, and in the other it’s not. Then associal psychologists like to do, sort of deceptive, manipulativemean things to people, we gave, we randomly assigned people toget a box of popcorn and for some people, it was beautiful, it

was just been popped a half an hour earlier, it was warm, it wasbuttery, it was delicious, and the other half of the people, thepopcorn had been sitting in my office for a week in sanitaryconditions I hesitate, I quickly add. People are randomlyassigned to get this fresh or this horrible stale popcorn, andthey’re in one of these two environments. Then at the end of thestudy, we got a measure of whether you are the sort of personwho always buys popcorn when you go to the movies or rarely dothat, so there’s a measure of kind of are you a habitual popcorneater or not? First of all just to reassure you that themanipulation kind of worked, people do indeed find fresh popcornnicer than stale popcorn, I won’t be winning any Nobel prizesfor that finding but it checks the box that the people noticedthat the stale popcorn was not very nice. But then the questionis, okay people disliked the stale popcorn but what did they dobehaviorally? Let’s look first at that group of people who arein that environment that for some people will be associated withconsuming popcorn. So sitting in a cinema watching movietrailers. Let’s look first of all, these are the people who arethe non-habitual eaters, who said “No I don’t actually buypopcorn that often “when I go to the movies. “Sometimes yeah,sometimes no.” Well they acted as you would expect rationally.They ate most of the popcorn if it was fresh and they didn’t eatmost of the popcorn if it was stale. Now let’s look at thehabitual eaters. They ate exactly the same amount, exactly thesame amount regardless of whether it was fresh or stale. But nowthe interesting thing is what happens when you take those peopleand you put them in an environment that isn’t for themassociated with eating that food? Let’s look at the controlcondition now. First of all, here’s the non-habitual eaters,they behave basically the same as they did in the other study,ate it if it was fresh, didn’t eat it if it was stale. But nowlet’s look at the habitual eaters. So now suddenly, the habitualeaters are eating it if it was fresh and not eating it if it was

stale. Just shifting these people out of the environment that isassociated with their routine was enough to restore control totheir preferences, their attitudes, and they were like I’ll eatif it’s fresh and if it’s not, I won’t eat it. That’s contextchange, but often in an organization, it’s not possible to goand make everyone move their desks or move to a differentlocation, context change is not always a practical thing to doeither at an individual level or at an organizational level.What are some of the other things that you can do? Well if youcan’t change the context, we’ve also looked at the idea of doingsomething to disrupt the action sequence. One of the otherthings that fuels habits is that people end up with a kind ofsequence of actions that get tied together, and one step in theprocess triggers the next step in the process. Imagine yourselfgetting into the car in the morning. There are actually a wholebunch of different actions that you perform totally mindlessly,but you do them perfectly and in sequence, one thing triggeringthe next in sequence. If you can’t change the environment, youcan do certain things to alter the decision process in some way,add a step in, mess up the normal sequence, and at least wehoped in this study to find, to see whether that would also helppeople overcome the influence of habit enact out of intention.What we did was again got a bunch of people, put them all in themovie cinema, and then randomly assign people to eat either withtheir dominant hand or to eat with their non-dominant hand.You’re literally just moving the box of popcorn from one hand toanother, that’s all you’re doing, so you’re using a differenthand to eat but it was enough we thought maybe to just mess withthe action sequence enough to give people a little window ofopportunity to do something different. Again, we gave them staleor fresh popcorn. First of all, here are people eating withtheir dominant hand, so this is the sort of the normal, non-disrupted way of eating and we replicate the effect we saw inthe previous study that is that if you’re a habitual eater, you

eat, and in fact in this study, people ate slightly more of thestale popcorn than the fresh popcorn. But again, they’re actingout of habit. There’s no influence coming from the freshness ofthe popcorn. Then here’s folks who switched the hand they usedto eat. Now suddenly you’ve got a rational, sensible gap betweenthe fresh and the stale popcorn, people eating it if it’s freshand not eating it if it’s stale. The sort of interim summary ofthis would be that there are ways to put the intentional selfback in control. One of the things we can do is get in andsomehow mess with the environment, alter the environment, removecues that have been associated with triggering the old way ofdoing things. Just to sort of think practically for a secondabout ways in which this might be scalable, we often think aboutavoiding making changes to organizational practices when manyother changes are taking place but I think one of theimplications of this type of thinking is that actually, it canbe a good thing to try and make new changes either with or justafter other major structural changes have taken place,especially if they involve moving people out of environmentsthat they’ve been stuck in for a long time, physically changingthe environment in some way. Perhaps timing interventions tooccur shortly after those kinds of major structural events mightactually be a good thing rather than a bad thing which might bethe sort of standard current thinking. Or instead we can dosomething to disrupt the familiar action sequence. That couldbe, obviously here we were thinking about very simple low-levelbehaviors but within a team, it could be doing something toalter the normal sequence of decision making, adding a personin, changing the order of a step might be enough to dislodgepeople from just running through the normal habitual pattern ofthinking and instead do something else. One I think importantnote here is, throughout this research is that these sort ofhabit disruption techniques only work if the intentional self isalready aligned with the desired change. We saw in the popcorn

studies for example that making people, moving them to a newenvironment or making them switch hands didn’t drop the totalamount of food people consumed, it just made them sensitive towhether the food was fresh or stale. In other words, if people’smotivations and intentions are still pushing in the quite wrongdirection, then messing with the environment, messing with theaction sequence won’t help. We really are, we’re reallyemphasize in this work that not that these techniques areintended to replace strategies based on information, motivation,and intention change, goal setting, but to augment them. If youcan successfully change people’s intentions and disrupt the oldways of doing things, that’s the optimal sort of strategy totake. In the time I have left, I wanted to just share with youthree more simple and interesting ways that we’ve come across,either in our own work or in other people’s work that people cankind of escape the tyranny of this habitual self when it’s notworking for them. The three ideas are, the first one is, thefirst one is the question of whether or not just by beingmindful, just by being vigilant in our thinking, consciousnessraising, can that have any positive effect on habits and if so,in what conditions? The second idea is to look at the value offorming very specific what are called if-then plans, and I’lltalk to you about what those are. Then the third and final oneis the idea of using kind of creative perspective taking as away of getting outside of the well worn ways of doing things andcreating space to do something new. Let’s tackle the first oneof these which is sort of simple awareness and vigilance.Earlier I gave you that example of the visual illusion and thatwas a case where consciousness was basically useless, youcouldn’t just by knowing that the tables were the same length,you couldn’t force yourself to see them as the same lengthbecause that automatic visual system is so strong and sooverpowering that it dominates what you see regardless of whatyou consciously know to be true. Fortunately the same is not

true for habits. There is more scope to use mindfulness and touse what we call sort of vigilant monitoring or vigilantmindfulness to create a little bit more space to overcome oldways of doing things. In summary, in these studies, what we’venormally done is gone out into the field and recruited a wholebunch of people who were trying to change behaviors of differentsorts and then just measure the different strategies that peoplenaturally use. The top ones, when people are trying to change abehavior, so maybe reduce snacking or making themselves exercisemore or giving up, reducing alcohol consumption or cigarettes,there are sort of three main strategies that people use. One isdistraction, so they try to do everything possible to dosomething other than the problem behavior. The second one isstimulus control, so they avoid any exposure to the cue, so theygo and hide the ice cream, for example, or make sure they don’tbring it into the house in the first place. The third strategyis what we call vigilant monitoring or vigilant mindfulness, andthat’s basically a strategy where you, instead of trying toforce the unwanted behavior out of mind or avoid exposure to thecues, instead you actually direct as much mindfulness to it asyou possibly can and you sort of cultivate an orientation ofvigilance towards whatever you think is the cue that istriggering the unwanted response. All those strategies work tosome degree. Distraction doesn’t work very much, stimuluscontrol does work, so making sure that the cookies are hidden atthe very back of the cupboard works to some degree, but the mosteffective strategy is this vigilant mindfulness. What we foundin these studies is that it’s really important that you’remindful about the right thing. The right thing is knowing whatare the cues, and that could be the people, the places, thetimes, the decision points that are triggering you to go downthat well worn path that you decided you don’t wanna go down anymore and then there’s two other things that I think are reallyimportant to direct consciousness to. One is that we know not

just from our research but from 40 years of research inpsychology, that humans have a very powerful tendency to mistakethings that are familiar with things that are good or to infer,another way of thinking about that is, we have a very compellingtendency to assume that because we’ve done something before,that it must reflect our choices and our preferences and ourvalues. Simply being aware of that, that we have this sort of,this inherit bias to partly infer our preferences by lookingbackwards in the mirror and saying what did we do in the pastand that may actually be a sort of cognitive bias rather than avalid insight I think can be another thing that gives you alittle bit of scope to do something different. Then the thirdone is that we know that when people are acting out of habit,when they’re highly routinized in a domain, it also forces themto shut down searching for new information and so that newinformation, information that counters what you already know ismuch less likely to be processed because you don’t direct anyattention to it. I’ll give you a little demonstration of this.What I’m gonna do is flash up some words on the screen all atonce, it’ll be about 10, and I want you to see if you canremember as many of them as possible. I’m just gonna put it upfor about two seconds, so be ready. Who thinks the word sandwichwas on the list? Okay, maybe about a fifth. Who thinks the wordbread was on the list? Okay so more, it seems like about halfthought that bread was on the list and maybe a third thoughtthat sandwich was on the list. Sandwich was on the list, breadwas not on the list. [audience laughs] Yeah, so. What’sinteresting, I’ll go back just to, so you’re sure, that’s theoriginal list, just so I didn’t cheat. I didn’t cheat. Why onearth do we do that? That’s very weird. We do that becauseobviously this set of terms is so familiar, it makes up a verywell worn comfortable sequence of events that we’re all veryaccustomed to. What some of you did, the majority of you did wasfilled in what was actually a missing piece of information by

seeing bread as having been on the list when it actually wasnot. Obviously this is just a metaphor but we do exactly thesame thing every day in decision making when we’re acting out ofhabit. That is, we see, we do two things. We see information asbeing there that is actually not to support the old way of doingthings and we also miss information that might point us to doingsomething else. I think to sort of tie those, this sort of ideaabout how does mindfulness, how does vigilance, how doesconsciousness help in this process? I think it’s really thesethree things. The critical things are having a process where youunderstand well what might be the points in this process wherewe’re being cued to do, to just repeat the same old way of doingthings? The second is to be aware that there will be a tendencyto assume that because something was done in the past, that it’sthe way things should be done in the future and is the optimalway of doing things. Then the third one is that we’re probablygoing to ignore information that is there and maybe seeinformation as being there that actually is not. Okay so thesecond idea is in terms of overcoming habits, there’s some verygood research now on something called implementation intentions,you may have heard about these before. This is essentially amemory phenomenon. What’s being discovered by researchers incognitive science is that we do a much better job carrying outfuture plans if we don’t just set an intention like I’m going toeat more healthily, but we set an intention that is connected toa specific environmental cue. For example, if a person is tryingto lose weight, it’s going to be much more effective for them tosay “Whenever I go to a restaurant and open a menu, “I’m onlygoing to look at salad options,” than for them to walk into therestaurant and say “I’m going to eat healthily now.” There’ssomething very powerful about tying the intention to anenvironmental cue. This is an area of active research at themoment. There is some data showing that this works for habitstoo and for the very reason that, especially if you can find the

right cue that’s triggering you to do something, triggering youto act in the old, well worn way, if you can kind of re-appropriate that cue and link it with an intention to dosomething else, that’s a really effective way to go. Just sothat you know, implementation intentions have been shown to beeffective for a whole bunch of things, but here’s just someexamples. Certainly for making yourself exercise, doing thingslike recycling, and there’s also very good data showing thatimplementation intentions work for helping people control genderand race stereotyping. It’s a little bit, just to sort of tellyou practically what that would involve, it sounds a littleheavy handed but basically in these studies on gender andstereotype control, people would, I know you’ve heard aboutthese studies before where people are looking at vitaes ofapplicants for jobs and they’ll sort of randomly change the nameto make it female or male, it’s actually exactly the same vitae,but they just randomly rotate it and it’s a way of working outwhether someone has a gender bias because they keep giving thejob to the male even though the genders were randomly assigned.In these studies, those sort of biases can be reducedsignificantly by just having a process where people haveformalized and signed onto the idea of ignoring race as a cue inthe decision process. Okay so the final idea that I wanted totell you about is this sort of idea of creative perspectivetaking. To sort of bring this point alive to you, I’m gonna doone little final exercise with you and so I’m gonna randomlyassign people in the room and I want you if you were bornbetween January and June to imagine how you might solve aproblem that I’m about to give you, but if you were born betweenJuly and December, I don’t want you to imagine how you wouldsolve it, I want you to imagine how the person sitting next toyou might solve it. Is that clear? I’m gonna describe thisproblem and if you’re January to June, you need to imagineyourself in this situation and how you would solve the problem

and if you’re July to December, you’re not thinking aboutyourself, you’re thinking about the person next to you and howthey might solve it. You ready? Here’s the problem. A person isin jail, they have a piece of rope, but the piece of rope ishalf the length required to climb out the window and down tosafety. The person cuts the piece of rope in half and climbs tosafety. How did they do it? Take 10, I’ll repeat it, okay? Sothis person is trapped in jail, it’s either you or the personsitting next to you, trapped in jail, has a piece of rope. Thepiece of rope is half the length required to climb out thewindow and to safety so what the person does is cuts the pieceof rope in half and climbs to safety, how did they do it? I’lltell you in a second what the answer is, but I’ll just give you10 seconds. Thinking about yourself in this situation or theperson next to you. All right, does someone wanna, I saw, yeah,what do you think the answer is?

AUDIENCE MEMBER: [mumbles] they split it so the rope has to be[mumbles] so he ties it [mumbles]

DAVID NEAL: Right, that’s right. Cutting it in half meanscutting it length ways, not cutting it across, okay? Cuts itlength ways, ties the two pieces together and climbs to safety,that’s the answer. I wanna ask now of the people who wereimagining themselves, so if you were born January to June, canyou tell me how many, put your hand up if you got that solution.Okay, gonna do a highly scientific poll here. Okay and if youwere thinking about someone else doing it, put your hand up ifyou solved it. Okay, I can’t tell at all. [audience laughs] ButI can tell you it looks like about half, but what I can tell youis what the data shows and the data shows that people are muchbetter at doing this kind of insight problem if they imaginesomeone else grappling with the problem than if they imaginegrappling with it themselves. Why would it be that this be thecase? Well, the argument is that stepping outside of the self,

literally engaging with someone else’s consciousness, projectinginto someone else is a way of escaping the familiar, well worn,uncreative ways of solving problems. It’s amazing to think thatsomething as simple as instead of, you know if you’reconfronting a problem and trying to act in a novel way, come upwith some new solution, to think that one way you can do that isas simple as not thinking about how am I gonna solve thisproblem but thinking about how the person next to you mightsolve it. I’m close to the end of my time. I wanna just finishby summarizing this and saying I hope I have convinced you inthis talk that habit is one of the main factors that explainsthis all too common misalignment that human beings have betweentheir attitudes, intentions, and values on the one hand, and thebehaviors that they enact on the other and that we have reasonto be optimistic about our scope to intervene in that process.Some of the things we can do are get in and mess with thecontext, the people, the places, the action sequence, thedecision points, mess with that process in a small way to createa window of opportunity to do something else. We also saw thatmindful vigilance can help. This isn’t as bad as those visualillusions. Being conscious can help, especially if you’refocusing on the right cues and you’re aware of the other biasesthat habits involve, like making you feel like the familiarthing is the best thing and making you ignore information. Thenfinally, we can make progress by forming these very specific if-then plans and by sort of creatively departing the self andprojecting into other people as a way of thinking outside thebox and doing something new. With that, I’ll thank you very muchand very happy to open it up to questions. [audience applauds]

AUDIENCE MEMBER: I read a study in Science sometimes last yearand it, I wonder if this applies. It had to do with when you gointo a room that’s different that you sort of reset, and thiswould be a case where, you know, you forget what you went into

the room for. Since I read it and I’m not rereading it, I maynot have all the details right, but would that have anything todo with … It would be a habit and not having the familiar, butthen when you go back into the other room, you can figure outwhy you were going into the other room.

DAVID NEAL: That’s exactly right, yeah. Yes, that’s a veryinteresting question. That research is about, it’s called eventboundaries and the basic idea is that people, you know, if wethink of ourselves in computational terms, people time stampbehaviors and link them to specific places. So you are much morelikely to create an event boundary when you move from one roominto another which is why you can, you know, have known why youwere going into the garage before you went to the garage, youcross over that event boundary and suddenly you’re in a newspace and you’ve got no idea what you were doing. It’s a similarphenomenon and it makes, I think it reinforces that point thatspace and memory and space and action are very tightly bound upand we don’t give the, we don’t give the environment the creditthat it deserves as helping us sometimes and hindering us atother times in doing what we wanna do.

AUDIENCE MEMBER: If you’re in a setting where you’re trying tomake a change to habitual behavior, I wonder if it’s helpful,like suppose you also have a context change going on, you know,something’s happening at the workplace, is it helpful to presentpeople with data that explains that context change is a goodtime to do habitual change?

DAVID NEAL: That’s a really good question but a complex one. Thebasic challenge is that, you remember I said earlier on in thetalk, I said that our sort of intentional self is a sort of rulebased, verbalizable, declarative memory is the word we use forit, system, and that’s not the same system that our habits arestored in which is why I can’t, if I was a brilliant concert

pianist, I couldn’t give you five rules that would immediatelymake you able to play a Chopin piano concerto, right? To alesser extent, the same problem arises here and that is that canyou, informationally, convince people to change their habits? Ithink you can do multiple things. First of all, as I saidthroughout, you’ve gotta get people motivationally in the rightplace so they’re interested in changing in the first place, andso that data that you’re talking about could be part of thatcase. I see those things as sort of interventions to make surethe intentional self is on board first and then people alsounderstand why you’re doing this weird thing of making them, youknow, deal with additional change while other change is alreadytaking place. That could be part of building the case for theintentional self but those interventions by themselves are notgonna disrupt the habitual self, so that wouldn’t be enough inand of itself. In addition, you’d need to be doing things toalter the environment, mess up the normal decision makingprocess in some not too irritating way, but enough to create alittle space for people to do something different. That’s theway I would tackle that.

AUDIENCE MEMBER: I’m a department head and I’m thinking of thisvery practically. If I always have my faculty meetings in thesame room and people sit in the same seats and they have thesame bad behaviors, can I move to a different room and break upsome of that and should I tell them why I’m doing it or just doit and see what happens? [audience laughs]

DAVID NEAL: That’s great. Well as an experimentalist, I thinkyou should do it without telling them and just observe theresults but no absolutely I think that, again coming back tothis issue though of if there’s … Context change is great, butif the underlying ill will is so overwhelming, context changeitself won’t do anything for that. That’s why we’re reallytalking about a parallel, like a push on two fronts in parallel,

so an intervention that addressed all of those tensions and thenpaired that with environmental change would be the optimal thingto do because you’ve made an intervention with the intentionalself and an intervention with the habitual self which reallygives people the best opportunity to press restart.

AUDIENCE MEMBER: I guess I’m also thinking about the practicalside and I wondered if you had some advice for us about what dowe do now in terms of fixing our situation, women in IT andattitudes and things? It sounds like people have to really wannachange to change.

DAVID NEAL: Yeah that’s true although the, I mean you guys knowthis area much better than I do, but I take seriously thoseGallop data. I know they probably overstate the case. I know twoof other very good scholarly work looking at the leadershipmodel and the gender neutral drift that has taken place in theleadership model over time. It started to be the case thatpeople’s stereotype of a leader and people’s stereotype of amale basically overlapped almost 100%, right? So stereotypicallyfemale traits were not really represented in the leadershipstereotype at all and there are great studies showing how theleadership stereotype has expanded and now is much more genderneutral, some studies even showing that it has more femalestereotypic traits than male. To me that’s affirming in that itsuggests that the intentional side of the equation has madesignificant progress, not that it’s where it needs to be, butthat there has been progress there and so there’s lots to feelgood about. The question is now how do we structurally roll thatout and make sure that it gets reflected in outcomes and I thinkthat’s where some of this work and a whole range of other morestructural interventions and policy type interventions are gonnabe necessary because I think that this shows pretty clearly thathearts and minds is one thing but that by itself is not enough.So drawing on these multiple fields, including this work, to say

how can we go beyond just changing hearts and minds?

AUDIENCE MEMBER: Learning is good with our students but thereare many habits, bad habits, that interfere with their learningsuch as texting or being on Facebook while doing your homework.Have you investigated student learning through the lenses ofyour framework?

DAVID NEAL: That’s a great question, I have not. Others havedone that and there’s some very interesting study, actually somevery interesting work on gender differences in sort ofattentional control and multitasking and you may have seen thisdata but there’s some good cognitive neuroscience data showingthat women are better at juggling multiple tasks and showingthat at the level of sort of underlying memory systems that …Yeah, so that’s an area that’s under active investigation. Ihaven’t been involved in that work but I think it’s reallyinteresting.

AUDIENCE MEMBER: I was just wondering how you would take yourframe of looking at things and apply it to for instance thecivil rights movement? Prior to the voting, there was somepeople who maybe thought it was changing the hearts and mindsand how does someone like you, from your perspective approachissues of social change?

DAVID NEAL: That’s a great questions, it’s a horribly difficultquestion, but thank you for asking it. [audience laughs] I thinkto be honest what we, we don’t think about it so much in termsof social justice, we think about it, although that’s a verylegitimate way to ask the question, we think about it more aspolicy and what kinds of policy interventions. Our work in thisarea has made us very convinced that top down policyinterventions wherever possible are really, are really the bestway to go. What I mean by that is things like making cigarette

packs out of view, things like banning the huge soda containersbecause just, and the reason we’ve reached that view is thatlooking at the data, we just know that this habitual dimensionof life controls so much of what we do that it really does meanthat people by themselves are often not able to change and so ifyou can make policy interventions that really make it muchharder, the people have to sort of effortfully opt in to doingthe wrong thing, that that is a much more powerful way to go.There’s wonderful data looking at organ donation data acrossEuropean countries, some of you might’ve seen it, and there’sthis weird effect where some countries in Europe have organdonation rates that are close to 100% and others have theaverage is about 12% and the top is like 27%, and that was aftera massive kind of public information campaign to get people tosign onto donating their organs. So why is there this huge gapwhere some countries have 100 and some people have like 10? Thereason is simply that on the license registration forms, thosecountries have opt out, the countries that have 100% have it optout and the countries that have 10% have opt in. To me that’sjust, you would think that something as fundamental as am Igonna donate my organs after I die would be something thatpeople would make an active effortful conscious choice about butthey don’t, so these little structural nudges, whether it’ssomething like that on a form or whether it’s a policyintervention, just much more effective in moving people in theright direction.

MAN: On your left.

AUDIENCE MEMBER: I was curious to know if your research isgender equal or does it have a difference? Because then we couldapply some of this to educational settings.

DAVID NEAL: That’s a great question. Just let me clarify tounderstand what you mean. Do you mean do we have a fine, did we

include both males and females in the research and then did wefind any gender differences?

AUDIENCE MEMBER: If you found gender differences, if females aremore of this type or that type.

DAVID NEAL: Yeah, that’s a great question. We have not … Wealways test for gender effects and we basically never find them.We have … And certainly they’re never systematic. Moregenerally, it’s very difficult to find any sort of demographicvariables that determine whether you’re a more habitual orientedperson or intentional person, we pursued that idea for a longtime, and found basically no evidence for it and there’s goodreason for that which is that that memory system that we have,that implicit learning memory system, it’s so old inevolutionary terms that the idea that it would vary, forexample, it’s uncorrelated with IQ. Your ability to learninformation in a habitual way, totally uncorrelated with IQ.It’s really such a sort of, it’s almost like your lungs, theidea that there would be big demographic differences acrosssomething as fundamental as that, so we haven’t found them.

JANE MARGOLIS: Everybody please join me in thanking– [applause]

DAVID NEAL: Thank you, thanks.

Vimeo ID 70358913

2013 NCWIT Summit – Plenary

II, Educational Disruptions byBen Eater[upbeat music]

VALERIE TAYLOR: So now I’ll take the opportunity to introduceBen Eater. Ben is the Lead Exercise Developer at Khan Academy,which is a non-profit education platform dedicated to providinga free world-class education for anyone, anywhere. He leads ateam of several developers who together are responsible forcreating all of the interactive exercises and assessment contentfor Kahn Academy. Ben joined the Khan Academy as a volunteer,but after six months he joined the team as a full-time employee.So with that I welcome Ben Eater. [audience applauds]

BEN EATER: All right, very exciting to be here. Just to give youa quick introduction what Khan Academy is. Khan Academy wasstarted by Sal Khan who was a hedge fund analyst in Boston whohad a cousin in New Orleans who was having difficulty in math.She was in middle school, had been placed into a slow mathprogram and he offered to tutor her remotely over the internet.And had started tutoring her and was actually able to get her toa point where she was ahead of her class. And he did this bycreating some interactive software for her to use and workingwith her over the phone. And once word got around in the familythat free tutoring was going on, he started tutoring a number ofhis relatives. Basically it got to the point where he wasworking with 20 or so different relatives all over the countrytutoring them in math and having some difficulty getting this toscale. And so [chuckles] a friend of his suggested well whydon’t you put some videos on YouTube? He’s like no, no, noYouTube is for cats playing the piano, [audience laughs] it’snot for serious mathematics. But eventually he kind of got over

the idea, it wasn’t his idea, put some videos up on YouTube andthe feedback he got from his family members was actually thatthey preferred him on YouTube than in person. [laughing] Whichsounds a little backhanded, but it actually makes a lot ofsense. On YouTube they could pause the video, they could rewindit, if they were stuck they could go back and watch previousvideos. And they didn’t have to feel embarrassed that they wereasking for him to explain something over and over again, theycould do this in the privacy of their own home. And somethinginteresting happened, the videos ended up taking off. And theywere getting thousands and tens of thousands, hundreds ofthousands of views on YouTube and people started commenting onYouTube thanking him for these videos. And they just sort oftook on a life of their own and he became just more and moreobsessed with creating these videos. And eventually convincedhis wife to quit the hedge fund and do this full-time not reallyhaving a plan for where it would go. And spent about nine monthsdoing this, kind of funding it himself just out of his ownsavings. And thankfully about nine months into this a number ofpeople started noticing this, most key among those is Bill Gateswho apparently discovered Khan Academy and was using it with hiskids and was starting to talk about it publicly. And so meetingwas setup with the Gates Foundation and the Gates funded theKhan Academy, turned it into a real organization and startedhiring a team to build out that initial software platform. Butjust to give you kind of a flavor of what these videos look likeI’m just gonna show a quick montage showing some of the videosthat Sal made as well as we have some more video contentcreators that have created videos as well.

SAL KAHN: We could integrate over the surface and the notationusually is a capital Sigma. All these interactions are justthrough the gravity over interstellar, almost you could call itintergalactic. This animal’s fossils are only found in this area

of South America, a nice, clean band here. Notice this is analdehyde and it’s an alcohol. Is their 30 million plus the $20million from the American manufacturer. They create theCommittee of Public Safety, which sounds like a very nicecommittee.

DR. STEVEN ZUCKER: This is not Eve.

DR. BETH HARRIS: No, Bottiecelli’s portrait, the ancient goddessof love.

BRIT CRUISE: This is six times six, times six, or 216.

LEBRON JAMES: I’m told the humidity makes it feel hotter, why isthis?

SAL KAHN: Excellent question Lebron. [audience laughs]

JESSICA LIU: Let’s just like make it 11.

SAL KAHN: Play with the pendulum and get a feel for how itmoves.

KARL WENDT: Function as a bridge rectifier.

VI HART: One, two, three, four, five, six, seven, eight.

SAL KHAN: If this does not blow your mind, then you have noemotion. [audience laughs]

BEN EATER: Can always judge the intelligence of a crowd by howmuch they appreciate Euler’s identity. You guys passed that Iguess. So where we are today Khan Academy now has scaled, we’vehad over 85 million users visit the website to date. Over 260million videos viewed. That interactive software that I justtalked about that Sal started using with his cousins, we’ve nowhad over 1.1 billion problems solved with that software. Andwe’re continuing to grow. There’s over 29,000 classrooms that

are using us either formally or informally as part of theircurriculum around the world. So very exciting where this isgoing, but I just wanna step back for a moment and just give alittle bit of a backstory about how I came to Khan Academy.Because it was sort of an interesting start. I actually neverdid particularly well in school. I was kind of a C Student mostof the way through high school and always particularly struggledwith mathematics. Which was always sort of frustrating to me,because I was always very interested in figuring things out andI was interested in programming and all of these things thatsort of use math. But the content in classes just never appealedto me, I never got it or understood it. And this got even worsewhen I got to college when I got into a computer science programand the first class I had to take I college was calculus. AndI’d never seen calculus in high school, because back in middleschool I was tracked into a remedial math program and so nevermade it to that point. And so when I got to college I tookcalculus and I ended up getting a D in the class. And againdidn’t quite understand, I actually felt like I understoodcalculus. I mean it’s one of the science fair projects I did inhigh school, I was interested in computer speech recognition.And so I was doing some signal processing where I was takingwave forms in and doing Fourier transforms and then trying to dopattern matching with markup models and all of the rest. And soI felt like I had some kind of understanding of the concept ofcalculus. I was able to actually apply them to do things Iwanted to do that were interesting to me, but I got to the classand it just didn’t make sense to me, very confusing. So I got aD in calculus, which is a passing grade so I moved on tocalculus two the next semester. [audience laughs] Of course. Andpredictably failed that. So at this point I thought well, maybeI oughta go back and take calculus one again and see if I cantry this again. So my third semester I took calculus one andactually failed it that time as well. [audience chuckles] And at

this point the university and I came to a mutual decision topart ways, which is to say they kicked me out [audiencechuckles] and I was kind of okay with that at this point. And Iguess I was actually very fortunate in a sense that the timingworked out very well for me. This was kind of at the peak of thedot com bubble and so it was actually very easy for someonewithout much in the way of a credential to get a tech job, whichis what I enjoy doing. And so that’s what I did and I startedlearning a ton in the real world and actually really thrivedthere. And was able to build a very successful career as aresult in the networking industry and went on to work as anengineer as an ISP and then at a large equipment manufacturerand so on. And finally got to the point in my career where I waslike okay the next step is for me to start a business. And sothat’s what I did. I started a business building some networkanalysis equipment. And I ran into a problem which was that inselling my stuff to my customers, my customers didn’t quiteunderstand the benefits of my product and I was trying toexplain it to them. And it turned out these were all engineersthat I’m selling to, but I guess it had been awhile since they’dtaken their statistics classes. And so I found myself trying toexplain Poisson processes to them and having some difficulty’cause I wasn’t explaining it very well. So I went online andsaid are there some resources that might help me explain thisbetter to my customers so that they understand what it is myproduct does? And I stumbled on some YouTube videos from KhanAcademy. I’m like these are amazing, this explains stuff very,very clearly. And I found the website and I found thousands andthousands of these videos. And I got to thinking, I was likeI’ve always been frustrated that school never made sense to me,I feel like in some sense the system failed me. I felt like Iwas a smart person, but somehow the material didn’t connect withme. Here I am 20 years later, let me go back and see where Iwent wrong. Maybe I can reengage with this calculus content on

Khan Academy, this guy seems to speak to me. Let me see how thisworks out. And so I started watching calculus content and Ifound the same things. There were things I didn’t quiteunderstand. So well you know they have trigonometry content, letme go check that out. And there were some things I got stuckthere too. And so well let me try the algebra stuff [chuckles]and I went back and when you get on Khan Academy we have thiswhat we call knowledge map. And each of these nodes is a conceptand it starts at the top with basic addition, one plus one, andyou can move your way down. And it works just like a video game.You master one concept, you move on to the next. And this is theway video games work, this is the way you learn a martial art,this is the way you’d learn a musical instrument. You master oneconcept before moving on to the next. And so what I did was Istarted at the top and I said okay give me basic addition, oneplus one. And I moved very quickly through some of these things,but then I found that there were these concepts really frommiddle school that somehow I missed. And I realized that I hadthese gaps in my knowledge and when I went back and went throughthis, I ended up completing all of these skills. And I finallyrealized why I kind of hit a wall in calculus, which was therewere all these little gaps in my knowledge that has compoundedover time. And the way that the system worked, there was no wayto go back and fill in those gaps. The analogy we use is imagineyou’re building a house and you hire a contractor and you sayokay, the state has given us two weeks to build the foundation.Do whatever you can in two weeks. Contractor builds a foundationand the inspector comes in in two weeks. Inspector says well youknow, it’s not quite dry over here, there’s some cracks overhere. This is maybe an 85% foundation. Well that’s a B, great.Build the first floor. So you hire another contractor, you saydon’t worry about the foundation. Need you to build the firstfloor, spend two weeks on it, doesn’t matter if the supplies arelate, doesn’t matter if it rains, just do what you can in two

weeks. Contractor goes builds the first floor. Two weeks laterthe inspector comes in. The inspector says well this isn’t quiteup to code, you forgot to build one of the walls. We’ll callthis a 65% first floor. Great, that’s a passing grade. Build thesecond floor. [audience laughs] And this is exactly what washappening to me in my education. And I built floor after,[clears throat] excuse me, floor after floor. You get to thefourth or fifth floor, the whole building collapses and everyonewants to blame the contractor. Or they say oh we need moreinspections, we need better inspection. But no, the inspectionswere fine. The inspections were identifying these gaps and justthe system, the process, didn’t allow us to go back and fixthose gaps. And so basically in my experience I was goingthrough a lot of these exercises on Khan Academy and these aresome examples of what those exercises look like. This one kindof sort of a Montessori for calculus, allows you to kind of getthe intuition for a derivative. So I went through all of thesethings and I felt very passionate about what Khan Academy wasdoing. And I discovered that actually this exercise platform forcreating these things was opensource. And so in my spare time Istarted building some and actually all the ones I’m showing youare things that I built as part of that opensource project. AndI started sending these things in and I got positive feedbackfrom Khan Academy. I got an email that says would you like ajob? [audience laughs] And I was like no, I have my ownbusiness, but I really love what you guys are doing, keep it up.A couple weeks later I got another email with like here’s abunch of statistics on how many people have been using yourexercise and thousands of people, do you want a job? I was likeno, I’ve got my own business. I’m happy, things are going well.And then Sal called me a couple weeks later and was like do youwant a job? I was like no. [chuckles] At that point they’re veryclever, they said well we feel like we should be paying you foryour work. So why don’t we set you up with a contract, we

realize you’re busy, you have this business. We’ll set you upwith a contract, you don’t have to work at all or you can workas much or as little as you want. Do whatever you want and justbill us. I’m like well okay, I can’t turn that down. [audiencelaughs] Turned down the job a few times, but I can’t turn thatdown. So I went ahead and started doing contract work and justover the course of a few months I just became more and moreobsessed. I just found I couldn’t help but spend more and moretime creating these exercises for Khan Academy. And I had a bitof a decision to make. I had this business that was reasonablysuccessful and everyone was telling me it would be ridiculousfor me to just leave my business and go work for a non-profit.I’ve been sort of a capitalist my whole life. [chuckles] Soeveryone’s telling me this and I guess I had to make thisdecision. And then eventually I ended up at a party with somefriends that I hadn’t seen in awhile that knew me in college andkind of knew what I went through. And they were like oh ofcourse you should do this. I guess that was kinda the validationI needed and so I picked up, moved across the country and joinedKhan Academy. So that’s kind of what got me to Khan Academy. AsI was showing you these exercises, once I got to Khan Academyone of the, this thing keeps going forward too many things, onceI got to Khan Academy one of the things that really was kind ofmind blowing for me was not just so much I had my own personalexperience of going through it and realizing that I had allthese gaps in my knowledge and if I went kind of at my own pace,I could fill in these gaps and the videos were there to help me.But what was really mind blowing is how Khan Academy’s beingused in classrooms. And one of the things that we’re doing is wego into a classroom where we start to give teachers theseanalytics. And so basically what this chart is showing is thisis a graph we give to teachers, the rows are students, thecolumns are each of those concepts that we saw on the knowledgemap earlier. Green means a student’s proficient. Blue means the

student’s still working on it and red means the student isstuck. And so what the teacher can do is the teacher can walkaround the classroom, this thing updates in real time, studentsare all working at their own pace at their computers and theteacher sees someone who turns red, maybe this student isstruggling in exponents three. There’s another student therewho’s proficient in exponents three and the teacher can start tosetup a peer tutor kind of an arrangement there. Or if thatdoesn’t work out, the teacher can intervene one on one. Or maybethe teacher can see that hey, a bunch of students have masteredthis concept so now we can actually do a project together as aclass that involves that and knowing that all the students aregonna get something out of that. One of the other dashboardsthat we give teachers, we have a bunch of different dashboardsthat teachers have access to if their students are using KhanAcademy in the classroom, is this. And what this is showing isalong the bottom is the number of days that a student has beenworking on the site and then the Y-axis is the number of skillsthat that student has completed. And each of these lines is adifferent student. And so what you see when you start off isthat some students race ahead, they get in, they really engage,they get a bunch of skills. Some students are kind of in themiddle and then some students, the flatter lines there, are alittle slower. And those are the students that you might say ohthese are the remedial students, these are students we’re gonnapull outta class. We’re gonna put them into these remedial andtrack them separately. But what we find time and time again, andyou can never identify these students ahead of time, is that ifyou let everyone work at their own pace, some of the studentsthat you thought were remedial, they might struggle on a conceptfor a little while, but once they get unstuck on that concept,if you let them work at their own pace, they race ahead. Andthat highlighted example of a student there is an example of oneof those. And it turns out she ended up as the first in her

class. And we see this time and time again. So this is great forkind of the math concepts. One of the other things that wewanted to kind of push is how can we express creativity or howcan we use creativity? And basically the way that we do this iswe came up with this computer science platform. And a lot ofpeople say well you know I was asking about creativity and likecreative writing, that sort of thing, how does computer sciencerelate to that? Well of course you guys know computer science isfundamentally creative, computer programming. And so we createdthis platform that was designed to inspire creativity. And thisis just an example. You see the code on the left is creatingthis animation that you see on the right. And you can update thecode in real time. So if I change that number, you see like thelittle rainbow thing changes shape, you can change it to adifferent number, now they get bigger. You can even drag thenumber around and kinda change the size. And you see what you’redoing in real time so you get real sort of sense of what thecode is doing. And so we have a bunch of these differentexamples that people can work through. And then what you can dois if you make changes to this, you can save your copy as aspinoff and it shows up underneath the program. And so you cansee these are all just different things that different usershave created and sort of gone in their own direction. Andthey’ve just gone and explored the code, explored hey what doesthis variable do if I change this? What does this do? Andthey’ve come up with all these different things and they cancreate a spinoff. And then that shows up on their profile then.This is just what someone made of that. And of course you canmake your own things from scratch as well. And then if you getstuck we have a series of videos that work within this platformand I’ll just show you an example of one of them just to kind ofgive you a sense of what that looks like.

JESSICA LIU: And here’s where the magic comes in. Somewhere in

the draw loop we’re gonna change the value of X to be a littlemore than it used to be, like this. X gets the old value of Xplus let’s say one. Yay, it works. Except, aha it’s so smeary.If you’re wondering why it looks that way, it’s because weforgot to draw this background inside the draw loop. So it’sdrawing the car over and over again, but you can see all the oldcars underneath the new one. So if I just pull this line intothe top of the draw loop like that, and then press restart so Ican see my car again. Yay, it’s so perfect. And if we wanna makethe car go faster, we can just change how much we increase X byevery time. So if we make it 10, [cheers] it’s off the screen.

BEN EATER: So this is just one example, we have a whole bunch.This talk is obviously about animation. We have a bunch of thesevideos that were all made by Jessica Liu who is a phenomenalintern that was with us last summer. We hope we will see heragain. But just gives you an example of some of the things youcan do. All of the code, everything runs within a browser. Sothat’s the other great thing in terms of accessibility to this.You don’t have to setup a whole dev environment to get started.You just point your browser to Khan Academy and you can juststart writing code or you can start pulling one of the examplesthat we have and start making your own spinoff from that. So itreally inspires creativity. There’s questions, comments, tips,feedback, and everything that goes underneath there so you cancollaborate with other people, you can have comments, people cantell you hey did you try this or think of this? So veryexciting. So just kind of to wrap up, one of the things that youmay have noticed is everything I have been talking about so farhas been in the English language or focused on the US. And oneof the really exciting things that’s been happening is therehave been a number of different groups, non-profits mostly, somegovernments, that have been taking Khan Academy and taking itaround the world. And these pictures are just some examples of

Khan Academy being used around the world. And one that reallyexcites us is in the top left corner there. We used to kindajoke like hey, maybe someday Khan Academy will be used inMongolia. Well, [chuckles] not too long later we got an emailfrom Mongolia. [audience laughs] And it was from this girl Zaya,she sent in a little YouTube video thanking us for Khan Academy.We assumed you know she has access to YouTube, the internet,Khan Academy, she’s probably a middle-class girl and all that.But we read the email a little bit more and it turns out thatactually there’s this group of engineers in Silicon Valley thaton their vacation they go out to Mongolia and they’re setting upcomputer labs in orphanages. And that picture in the top rightis our Mongolian girls in a Mongolian orphanage using KhanAcademy. And Zaya was one of those orphans. And if that’s notcool enough, Zaya got really into it and she started translatingsome of the videos into the Mongolian language. And she’s 17years old now, so this 17-year-old Mongolian girl is now theprime producer of Khan Academy content in the Mongolianlanguage. She’s become the teacher for her country. So just togive you kind of a flavor of what these videos look like, I’llshow you another quick montage and it ends with a piece of thatoriginal video that we got from Zaya.

ZAYA: [speaking foreign language] [speaking foreign language][speaking foreign language] [speaking foreign language][speaking foreign language] [speaking foreign language][speaking foreign language] [speaking foreign language][speaking foreign language] [speaking foreign language]

ZAYA: I am Zaya from Mongolia. Your regents are so interestingand funny. Make more lessons.

BEN EATER: We watch that whenever we’re feeling a little lazyaround the office. [audience laughs] One of the amazing thingsis our team right now we have 40 people, but over the last year

we average about 34 people and with the scale of the internet wewere able to reach over 50 million unique students in 216different countries. And I really think we’re kind of at thisGuttenberg moment, not even a once in a lifetime moment, reallyone in a millennium moment almost to kind of push the scale ofthis forward. The original tagline on this was just thenecessities. [audience chuckles] I guess just to kinda wrap up,education has always been kind of seen as this scare resourceand this thing that was very expensive, this thing that sort ofseparated the haves from the have nots. And even when people aredoing philanthropy they would say well what do the rich have?Well that’s expensive, let’s create a cheap approximation ofthat and we’ll give that to the poor. The poor had nothingbefore, so a cheap approximation is better than nothing. But noweducation is sort of becoming no longer a scarce resource. Andso we’re almost getting to the point where I honestly believeeducation is really going to become a fundamental right. In 20,30 years people are gonna view education as a fundamental rightjust like clean drinking water, or shelter, or electricity,something like that. So that’s all I have. I thank you all.[audience applauds]

Vimeo ID 70358911

NCWIT 2012 Summit – Plenary 2,Shelley CorrellSHELLEY CORRELL: It’s great to be here with you today. NCWIT isan organization that’s objectives and missions very muchresonate with the objectives of my own research as I think

you’ll see very quickly as we dive into that. What I want totalk with you about today is the way that gender stereotypesintroduce biases into the workplace that can be disadvantagingto women, so I’ll spend the first maybe 15 or 20 minutes of mytalk laying out how these biases come about, and then I want tospend the remainder of the time thinking about ways that wemight actually be able to reduce them. I have a lot of researchI want to share with you so I want to jump right in, and I wantto talk to you about a study that when it came out received alot of attention in the press, so this might be a study thatyou’ve heard something about. In this study the context of it isthe hiring of musicians into orchestras. In the ’70s and ’80smajor orchestras all around the country began making a seeminglyminor change to the way they auditioned musicians for hire, andthe reason for this is at the time in the ’70s and ’80s womenmade up only 5% of all the musicians in orchestras around thecountry, so it was a very male-dominated space. The orchestraswere worried that perhaps a bias was creeping into theevaluation process, and was reducing the number of women thatwere being hired. So what they decided to do was to introduce ascreen in front of the musician so that now when the musicianauditioned for a position he or she was not visible to thejudges, and, therefore, the gender was unknown. This smallchange had a dramatic effect. The odds that a woman moved onpast the first round of auditions increased by 50%, so there’s a50% increase in the movement past that first round of auditions.Today 25% of orchestra musicians are women so we’re not toparity yet but that’s a big change from 5%. The verysophisticated analysis that was done by the people I cite on theslide shows that this screen really was a major contributingfactor to the increase of women. I like this study because itillustrates really two of the main points that I want to try todrive home today. The first is that gender stereotypes bias theevaluations of individuals in ways that we might refer to as

male advantaging, and what I mean by male advantaging is thatmen’s evaluations come out to be more positive then they wouldbe if we did not know men’s genders, and women’s evaluationscome out more negatively than they would if we did not knowtheir gender. So this is something what we call a maleadvantaging bias. So in the case of orchestras switching to thisblind audition removed this male advantaging bias. Once youcouldn’t see the woman her evaluations went up. This leads to mysecond point, and that is these biases can be reduced oreliminated, they’re not inevitable. So after explaining how theyemerge I’m going to talk about ways to reduce or eliminate them.Now in the case of orchestras the way they reduced them was toput a screen up in front of the person auditioning for hire.Clearly, that’s not a solution for everyone. We could not orwould not want to do all of our work life behind a screen, sowe’re gonna have to be a bit more creative than that but thereare tools and strategies that we can employ. Now before movingon to some of the other studies I want to just say a bit aboutthe use of the word bias. A lot of people don’t like this word,it sounds really negative. It has this kind of ugly connotationlike we’re pointing our finger at someone, and accusing them ofbeing sexist, but that’s not at all how I’m using the word biastoday, and it’s not how social psychologists use it in general.Instead, when we talk about bias we’re simply talking about anerror in decision-making. So if we think about the orchestrasagain we can probably assume that most of the judges evaluatingthe musicians were well-intentioned people who truly wanted tohire the best musician out there, but yet gender affected howthey saw the musician, and how they rated the musician’squality. For reasons I’m soon going to explain all of us, men,women, whites, people of color, young people, old people, areall prone to these sorts of biases, but the good news is witheffort, and proper procedures in place these biases can bereduced. So this is a no blame message, but a high

responsibility one. So why are these biases so ubiquitous, andwhat can be done about it? If we want to understand how toreduce the biases I think it’s important to first understand alittle bit about why stereotypes lead to bias in the firstplace. So one of the main things that we know is that genderstereotypes function as what we might call cognitive shortcutsin information processing. So to return to our orchestraexample, again, imagine that the orchestra is auditioning like100 musicians, and they only have one or two slots that they’rehiring for. Processing all the information associated with 100different musicians would be very cognitively demanding. That’sa lot of information coming in, especially, in the time usuallyallotted for such decisions. So usually we have weeks perhaps tomake hiring decisions, not months or years. In these informationheavy context, especially, under time pressure, individualsunderstandably look for shortcuts, some way to help themnavigate through that sea of information. Unfortunately, andthis is where gender stereotypes come in, gender stereotypesfunction as a shortcut. What I mean is implicitly orunconsciously we often rely on what we know about categories ofpeople such as men or women when evaluating individual people,individual men and women. So what do we know about men andwomen? What’s the knowledge that we have about men and women?Here the knowledge that I’m talking about is the knowledgethat’s encoded in widely shared stereotypes that are availablein our society. Stereotypes are simply that, they’re sort ofwidely held beliefs. We may not personally endorse them, butwidely held beliefs out there in our culture about what men andwomen are like, and these then are what’s available to lead tothe kind of errors in decision-making that I’ll draw out.Psychologists have differentiated between two types ofstereotypes. On the one hand we have what are called descriptivestereotypes. Descriptive stereotypes are beliefs about thetraits and capabilities that men and women are thought to

possess. They’re how we think men and women are. Research on thecontent of descriptive stereotypes finds lots of things, but oneconsistent finding is that we still find in modern day Americansociety that most people hold the belief that men are morecapable and competent at a whole array of tasks than women. Sowhether or not there’s any truth value to this or nor thesebeliefs persist. Now often these differences are small. Peoplethink men are a little better at something than women, but theycan be more pronounced, and they are more pronounced for the so-called male type tasks, and these are things that I think peoplein this room care a lot about. Nora’s comments had us thinkingabout things like spatial ability, mechanical reasoning and thelike. For these type tasks the beliefs are actually considerablystronger with people thinking that men are considerably betterthan women. Women, by contrast, are stereotyped at being betterat female type tasks such as those requiring nurturing abilityand caretaking, so if we were in a place where we wereevaluating people’s abilities in those domains we would likelysee a female advantaging bias. However, the thing to know isthat nurturing and caretaking themselves are often devalued inthe workplace, so those are descriptive stereotypes.Prescriptive stereotypes are widely held beliefs about how menand women should be, so they have that prescriptive shouldelement to them. The content of these beliefs are that women areexpected to be, they should be nice, women should be warm, theyshould be concerned about others, and not dominant or assertivewhere men are expected to be agentic, by that we mean sort ofaction oriented, get things done, assertive and certainly notmodest or weak. As we will see later with some of the studies Iwant to show you, when people violate these prescriptivestereotypes they tend to be disliked. So what I want to do nowis to show you what the consequences of these stereotypes arefor men and women in the workplace. How do they play out? One ofthe findings from my research, and the research of others is

that descriptive stereotypes affect the standard that we use tojudge the performances of individuals. So it turns out that ourstandards shift depending upon whether we’re evaluating a man ora woman. Why would this be? Well, if we think about it when awoman performs well at a task, and especially a male type task,so let’s say we find out she has a lot of spatial ability, forexample, this runs counter to our stereotypical expectationsabout women as a group. As a consequence we tend to morecritically scrutinize her performance. It wasn’t what weexpected so it says we’re trying to look at it and make sense.How did this come about? When a man performs equally well at thetask this by contrast is consistent with what stereotypes wouldlet us to expect, therefore, we don’t scrutinize his performanceas closely. What this means then is that we end up in manycontexts judging men by a more lenient standard than women, andbecause we’re judging men by a more lenient standard, again,this is out of awareness, the performances of men come to beseen as higher quality. Women’s performances, by contrast, beingjudged by a harsher standard come to be seen as lower quality,so I think this research again can help us understand what wasgoing on with the orchestra study. Women musicians were beingjudged when their gender was visible they were being judged by aharsher quality so their musical performance was not being seenas sufficiently high to warrant them a position in theorchestra. This is a graphic that I like to use to illustratethis. Here we have a figure trying to go over a high jump bar,and it says if the bar is a little bit higher for women than itis for men meaning that for any given woman she’s going to haveto jump a little higher to get over this bar than acorresponding man would, or in the aggregate if we want to thinkabout it, what this means is that on the other side of the barwe can expect to see more men than we see women. Now in thepast, of course, these differences were larger than they aretoday. Stereotypes 100 years ago were way more stark in

differentiating men from women, but even then, of course,exceptional women got over the bar, Marie Curie. We could nameother sort of exceptional women that have always cleared thebar, but in my work what I’m really interested in is thinking ofways to bring these standards closer together so that theaverage woman is rated in the same way that the average man is.I want to now turn to our second research study that shows ushow stereotypes affect judgments, and how differential standardsplay out in that. This study comes from the field of psychology,and what the authors of this study did is they conducted anexperiment where they created a resume for a job candidate whowas ostensibly applying for a position to become an assistantprofessor in psychology. This applicant had really goodcredentials, and he or she had just completed his PhD. So theycreated this resume, and then what they did is they sent theresume out to psychology faculty all over the country, and theyasked them to make judgments about this assistant professor onall kinds of domains. Importantly, they asked people to say howhireable this person would be in their own psychology departmentfor a tenure track position. So these psychology faculty getthese resumes, and make these judgments. A randomly chosen halfof the participants got the resume on the left with a man’s nameon it, Brian Miller, while the other half of the psychologyprofessors out there got the very same resume with the exactsame publications, the exact same grants, teaching evaluations.Everything was the same except the resume had a women’s name onit, Karen Miller. So now what the question, of course, is doesgender bias these job evaluations? People have the exact sameresume. The only thing differs is whether one is presented as aman or one is presented as a woman. The differences werestriking. 79% of the people who got the resume with Brian’s nameon it said that he would be worthy of hire for a tenure trackposition in their department while the other randomly selectedhalf of the participants who got the very same resume, but with

a women’s name on it only 49% of those deemed her worthy ofhire. A second phase of the study really points to this kind ofidea that women are being judged by a harsher standard. So in asecond phase of the study what they did was analyze the commentsthat people wrote on the evaluation forms, and what they foundis that the participants wrote four times more doubt raisingstatements on the women’s evaluation forms than on the men’s. Sothey wrote things like this first quote, I would need to seeevidence that she’d gotten these grants, and publications on herown. So notice they’re scrutinizing the information that’scontained on her resume. The second one, it would be impossibleto make such a judgment without teaching evaluations. I mean,teaching evaluations are a perfectly valid criteria if you’reevaluating a professor, but that this was being raised only onthe women’s evaluation forms I think is quite telling. So tosummarize to this point what we see is that gender stereotypesaffect the standards that we use to judge men and women and,therefore, can affect the extent to which we come to see them asbeing sort of higher or lower quality. This is, of course,unfair to women who have to perform at higher levels than men tobe seen as equally qualified, but I also want to point out thata lot of the things we’re talking about today are also going tobe unproductive for organizations who presumably would prefer tomake more accurate judgments about the people they’re hiring,and not being influenced by gender stereotypes. In addition tostereotypes influencing the standard that people use to judgeindividuals gender stereotypes also affect the criteria that weuse to judge individuals. A cool study that shows this isanother experiment, but this time what’s happening is people arerating applicants for a police chief position. As you may know,police chief is a very male type job. In this study participantsalways rated two applicants, and one of the two always had moreeducation, and the other one had more street smarts, or moreexperience we might say, so that was the key difference between

them. They tried to create people who were sort of equallyqualified, but one was better on one dimension, and one wasbetter on another. In the first phase of the experiment a set ofparticipants evaluated these two applicants with no names on thefile, so there’s no gender that’s salient here at all, and whatthey found is that participants overwhelming preferred theapplicant that had more education, and they justified theirchoice by saying this person has more education, and I thinkthat’s more important for being a police chief. So educationseemed to be the criteria that really mattered to theparticipants. In the second phase of the study they used thevery same resumes, but now what they do is put men and women’snames on them so now one of them is a man, one’s a woman, andimportantly the one that has more education is the man, and theone that has more experience is the woman. So now they have adifferent set of participants evaluate these two applicants.What they find here is that people overwhelmingly preferredBrian, the male applicant, and the reason they gave is that hehad more education. So no problem here, education was whatmattered in the first condition when we didn’t know people’sgender. Now with a different set of participants we know thegender people are still saying that education is what matters.What’s interesting is what happened in the next condition, andyou can probably guess this by now. So now what they did, adifferent set of participants, same resumes, they just switchedthe names around. So now Karen has more education, and Brian hasmore experience. So if education matters Karen should be chosen,but that’s not what happened, or I wouldn’t be standing heretoday. What happened is that Brian was actually chosen, and whenpeople were asked to justify their choice they said it wasbecause he had more experience. So notice how the criteriashifted to justify what was probably their gut feeling about whowas more appropriate for a job as a police chief, the man inthis case. So to summarize to this point we see that women, even

if highly qualified, can find their past performances discounteddue to shifting standards, and their suitability for a jobdiscounted by shifting criteria. Now what we’ve seen across allthe examples that I’ve told you, the orchestra musicians, thepsychology professors, and the police chief, is that stereotypesaffect hiring decisions. So you might be wondering, yeah, butdoes this matter once we get on the job? Once you’re on the jobyou’re able to sort of demonstrate your competence to people. Dostereotypes continue to produce this male advantaging bias? Theanswer here is clearly yes. Research consistently finds thatwomen have less influence in group settings at work. So what Imean by that is in a group setting if women offer a contributionto the group people are less likely to use that information whendeciding how to proceed. Women tend to have their contributionsjudged less positively in the workplace. Very similar kind ofexperiments that show this. Women are less likely to get creditfor their ideas. This third point when I talk to women’s groupsthis is the one that seems to really resonate with women atwork. I don’t know if you’ve ever had this experience, but it’san experience where you make a suggestion in a group and no onehears you. It’s just kind of like you said nothing, and a fewminutes later someone else in the group, perhaps a man, makesthe very same suggestion, and they get credit for it. Sowhenever I talk about this as I said it usually resonates withpeople, and one of the women in a group I spoke to recently sentme a card with this on it, which sort of humorously depicts thevery phenomenon that we’re talking about here. So cartoons arealways welcome. It’s not just cartoons research bears this out.So I’ll tell you about a study here. This is a classic socialinfluence kind of study that was done back in 1995. The authorsof this study created a four person group with two men and twowomen, and what the group was charged with doing is todeliberate and come up with a group solution about who should beawarded the custody of a child. So what happened is each of the

participants was given a packet of information to read about thecase, and they came and deliberated and came up with the answer,who’s gonna get custody of this child. The key feature of thestudy, though, is that one person in the group was given aunique piece of information that no one else in the group had,and it turns out that this unique piece of information wasactually really key to deciding the case. Half the participantswere in groups where the person who had the key piece ofinformation was a woman, and the other half were in groups wherethe person with the key piece of information was a man, so thequestion is who do you listen to? What they found is thatparticipants were twice as likely to use the key informationwhen it was introduced by a man. This is a very common findingin these studies. Now why would this be? Well, generally, we’remore likely to be influenced by people that we view as beingmore competent, so if gender stereotypes lead us to sort ofunconsciously expect that men are going to be more competent orcapable then that means that their ideas are probably going tomore easily sound good, right or convincing to us. Just one morecurrent study in this domain that I want to mention to youbecause I think it’s really important a very similar kind ofdesign, but what the authors of this study did is in addition toone person having more information than the other people in thegroup they actually had one of the people in the group labeledas the expert, so you think you would listen to the person thatis labeled the expert in your group. What they found is thatparticipants were more likely to listen to men experts than towomen experts, and this had consequences. What it means is forthe groups that had men experts they actually performed betterthan the groups that had women experts, but it wasn’t becausethe experts were any better, or knew anything more it was simplythat people were more likely to listen to the expert when it wasa man. What I like about this example is I think it really showsthat the negative effect that stereotypes have not only on women

but on workplaces. Presumably a workplace would want othermembers, employees in the firm, or what have you to be listeningto the experts that they, in fact, hire. Okay, so these are someof the kinds of ways that stereotypes can introduce bias.Sometimes, when I talk about this work people want to know arethese biases big enough that we should be concerned about them,or are they trivial, I mean, does it matter that someone doesn’tgive you credit for your ideas? Are we making mountains out ofmolehills here? But what I want to argue is that mountains aremade out of molehills, and that the effects of genderstereotypes while small in any one instance, actually,accumulate to create increasing disadvantages for women. I wantto show you a study that sort of shows how this can happen. Thisstudy is a computer simulation study where what the authors didis they created a hypothetical firm that you see representedhere, and it’s got that traditional kind of pyramid structure.At the top of the pyramid you see 10 people at the top. Theseare our leaders. These are the people who’ve moved up to thetop. At the bottom in the lower level entry positions you seewhat we have are 500 employees, 250 men and 250 women, so thisfirm does not have a pipeline problem, and we’d love to be inthis position in computing, right? So 250 men and 250 women inthe firm. The way you move up in this firm is if you have betterperformance evaluations than other people, so if you performbetter you move up it’s a meritocracy. It’s the only thing thatmatters in this computer simulation. In the first condition ofthis experiment what they do is they give everyone of thesehypothetical men and women a performance evaluation. Some havehigh evaluations and some have low evaluations, but the keything is they fix it such that the mean performance evaluationis the same for men and women, and the distribution is the samefor men and women, so while some men might perform better thansome women, or vice versa there is no pattern to the difference,okay? There’s no systematic difference. Not surprisingly given

this what they find is when they run the simulation over andover again on average the top positions are 50% women, okay?They call this the no bias condition with equal performanceinformation, and decisions based solely on performance 50% ofwomen at the top. The next condition what they do is theyintroduce a 1% negative bias onto women’s performanceevaluations. In other words, they take that performanceevaluation, and they discount it by 1%. That’s a trivial amountof bias I think most people would agree, very, very small amountof bias. The question is does it cumulate over the eight levelsof this organization as people are being evaluated? The answerhere is yes. With only a 1% bias we go from having 50% women atthe top to only having 35% of women at the top. Make the bias alittle bigger 5%, but still very small in terms of what we findin a lot of the studies that I’ve reported, and what we see nowis less than three and 10 of the positions at top are now heldby women, so what this I think shows is that even a small amountof bias if it happens day in and day out as people are beingevaluated can cumulate, and negatively affect women’s careers.Now, so far I’ve talked about sort of bias in general, andbefore turning to what we can do to make it better I want toactually talk about when the processes that I’ve described areworse, so knowing when things are worse helps us I think,sometimes, understand the origins of the bias. So, basically,what we want to try to understand is why is it that undercertain situations people rely even more heavily on stereotypesas a shortcut? Well, research has shown that these biases areworse when tasks are male-typed. When there are tasks for whichwe have pretty strong stereotypic beliefs that men are betterthan women these processes are more pronounced, so things liketechnical competence, spatial ability, things like that, thoseare the tasks that are being considered they’re worse. It’sworse when positions are higher in an organization, so thehigher up you move the worse these biases are, and that’s

because our stereotypes about leaders, and our stereotypes aboutmanagers if you say to people what’s the typical leader like,for example, those stereotypes overlap almost perfectly with ourstereotypes about men, so leaders and men stereotypes are verysimilar, but the stereotypes about leaders, and the stereotypesabout women hardly overlap at all, so I’ll show you someexamples of that here shortly. My own research has shown thatthese biases are worse when women are mothers. The stereotypesthat we have about mothers turn out to be stronger than ourstereotypes we have about women in general. In particularthere’s the stereotype that mothers are not as committed to paidwork is pretty strong, and has really pretty profoundconsequences. It’s worse when evaluation criteria are vague.When people don’t know how to evaluate someone they’re just toldhire someone, and they don’t know what to do that’s when theymore need this sort of cognitive shortcut, and they tend to relyon stereotypes more heavily. Finally, they’re worse when peoplemaking decisions are tired, rushed, or otherwise what we mightthink of as cognitively burdened, when we’re distracted thinkingabout something else, we’re tired in the kind of ways we almostalways are at work, and we’re really looking for a shortcut,stereotypes are readily available. So that’s that the bad news.Let’s spend the last bit of our time talking about what can bedone to reduce these things. Here what I want to do in mycomments is differentiate between survival skills andorganizational solutions. Both are gonna be useful in some way,but I want to differentiate between them. Survival skills arethe advice and training that we provide to women to help themnavigate a workplace where bias is always possible. If you wantto keep yourself busy for days and days on end Google careeradvice for women. Career advice for men is a much shorter numberof hits career advice, so there’s no shortage of things that wetell women in terms of survival skills about how to betternavigate their careers, and a lot of these things as I said turn

out to be quite useful, so for example, be sure that othersnotice your own accomplishments. Toot your own horn. One of theproblems with stereotypes is people they’re discounting yourperformance, so self-promote let people know what you’re doing.Be confident, negotiate, ask for things that you want. Speak upat meetings. Don’t wait for people to call on you. Volunteer forleadership positions. Don’t wait for people to come knocking onyour door, and because of the negative effects that can childrencan have don’t draw attention to your children. Don’t putpictures of them up, or if your children are sick say you’resick. I mean all of these things can be found in handbooks forwomen. Now as I’ve said these survival skills can often behelpful, but they’re rarely going to be enough, and they canoften, also, cause a backlash reaction. That means they cancause an unintended negative effect, so let me show you a studythat shows this. This study comes from psychologist LaurieRudman’s work, and what she does in a series of experiments shehas people examine how self-promoting behaviors affect thejudgment of individuals. In general what we know is that men aremore likely to as the saying goes toot their own horn than womenare they’re more likely to self-promote, so given this we oftenadvise women to be more self-promoting arguing that if womenself-promote they will be more likely to get credit for theirideas, more likely to be hired and so on. So what Rudman does inher study is she has people evaluate job applicants who areinterviewing for a job, and these job applicants as you can seefrom the bathroom figures here are either male or female, andsome of them are either self-promoting the ones on the rightwith the horns, or they’re more modest, so if you are aparticipant in her study you would be assigned to one of thesefour quadrants where you would see one of these applicants whois either male or female, self-promoting or modest. Now thething that the participants don’t know is these people areactually trained actors that are part of the study and they’re

enacting a script, so whether it’s a man or a woman self-promoting they’re saying the exact same thing, so that’s notwhat’s going on here. So what happens what does she find? Well,what she finds is that self-promoting works for both men andwomen the more they self-promote the more competent they’re seenin their job interview process, so self-promotion does enhancethe competence ratings of both men and women, however, women whoself-promote were also seen as being less likable. People didn’tlike the woman who self-promoted. They didn’t mind the man whoself-promoted, but they didn’t like the woman who self-promoted.Why would this be? Well, when women self-promote, or when theynegotiate too hard, or when they give directive orders in aleadership position they violate those prescriptive stereotypesthat we talked about earlier those shoulds, okay? These are thestereotypes that say that women are supposed to be nice, warm,concerned for others, not assertive, not dominant and notaggressive, so women are violating those by self-promoting. Now,sometimes, when I talk about this I’ve had women say to me,well, I don’t really care if people like me at work or not I’mjust trying to get my job done, but likeability matters,especially, when you’re trying to get a job. What Rudman showsin her paper is that the man who self-promoted was more likelyto be hired than his modest counterpart, but for the woman thatwasn’t true. The woman who self-promoted was judged to be morecompetent, but people didn’t want to hire her because theydidn’t like her. The modest woman by contrast wasn’t very likelyto be hired either because they didn’t think she was verycompetent, so you see the double bind it’s hard for women to bejudged as simultaneously likable and competent. So what advicedo we give women in this mess that we find ourselves in? So thesurvival skills here really involve trying to establish yourcompetence without being threatening, without being seen asaggressive or assertive or dominant, so one piece of advice thatI just love that you find over and over again is to be

relentlessly positive and pleasant. If you’re relentlessly, Imean, I love that, relentlessly, if you’re relentlessly positiveand pleasant than perhaps you’ll be seen as likable, but alsostill be seen as competent, so notice the tightrope we’rewalking here. Express group-oriented motivations when makingsuggestions in groups. For example, you might say something likeit would be best for all of us if we did X, Y and Z, rather thanjust saying let’s do X, Y and Z, the directive that we mightexpect of a leader. Use humor to coax subordinates along, and inthis last strategy push forward, pull back. I’ll read you aquote to give you a sense of what I mean. This comes from AliceEagly’s recent book on women leaders. She’s interviewing a WallStreet female executive, and this woman says you have to bestrong and assertive without offending people, so you push alittle and then back off. You’re always testing the waters tosee how far you can go trying not to get angry, trying to thinkof other ways to say you’re not right without attacking theperson. It’s getting more and more difficult the higher I go. SoI think as we look at this list, and especially as we listen tothat quote it becomes clear that we’re not solving the problemof bias. Instead we’re working around it. If we want to solvethe problem of bias we’re going to have to look upstream to morethe root of the problem, and we’re gonna have to come up withorganizational solutions, so in my last few minutes I’ll talkabout what some organizational solutions that I might recommendare, and I have six of these I want to share with you, butthey’re all going to sort of share a common underlyingprinciple, and I think whenever you’re trying to enact changeit’s really important to think what’s the underlying principlethat really could affect the change. Here what the principle isis we need to break the tendency of people to use stereotypes asa cognitive shortcut. So the first thing is actually teachingpeople about the effects of stereotypes. Now I do this a lot andpeople would always say to me you’re preaching to the choir, I

mean, you go to a group like this this is a group that’s aboutwanting to help women. Of course, people here are all on thesame page, we want to do this, we’re not the ones that need thismessage, but my pushback here is to say I’m not preaching to thechoir, hopefully, I’m arming the choir, and what I mean by thatis that we’re giving well-intentioned men and women the tools tobe able to avoid bias themselves, and also the tools to be ableto think about changes within their organizations. The secondsolution is to establish clear criteria for evaluations. Tons ofresearch of all kinds now shows that when formal criteria are inplace, when people know what they’re supposed to be doing whenevaluating job applicants the percentage of women, and thepercentage of people of color who are hired goes up. It’s a veryclear finding, and you see it in that experimental study withthe police chief that I mentioned earlier. In a fourth phase ofthis experiment what they did is before participants saw theapplicants they told the researcher what criteria mattered tothem, so they came into the study and they said we’re gonna haveyou evaluating police chief what criteria do you think isimportant people overwhelmingly said education was moreimportant. Once they said that they committed themselves to thatcriteria, which meant that Karen now with more education wasactually more likely to be hired, so what people committed tothat criteria Karen when she had more education was more likelyto be hired. The next closely related suggestion is to evaluatethe criteria you’re using in the first place, and to be sure itis the right criteria. A lot of times our criteria came aboutthrough historical means. We look around and say whose beensuccessful here in the past, and what are they like, and that’swhere we get our criteria. If women weren’t part of theorganization in the past we may have criteria that historicallymen were better at than women. If that criteria is not validthan perhaps we need to change it. Here’s an example that peoplein the room know very well I think that will illustrate that.

Carnegie Mellon increased the percentage of women in itscomputer science department from 7% to 42% in just five years.Now, since then the number has gone down some, and that speaksto a different issue, and that is how to sustain organizationalchange, but, nonetheless, the gain is impressive, and it’s worththinking about what they did to produce that gain and they didmany things, but one of the things they did was to change theiradmissions requirement, so that they no longer required highlevels of prior computer experience. As with special reasoningability this is an ability that could be taught by teaching itin the first semester of school more women were eligible forthis particular major. I mean, it didn’t affect the quality ofgraduates they produced out the other end. The next solutionhold decision-makers accountable. This is commonly recommended,but how it works in the case of stereotypes is very interesting.If I have to explain my decision to someone else I have to goand say here’s why I preferred Brian Miller knowing that I’mgonna have to explain the decision to others that I’m gonna beaccountable forces me to slow down, and more carefullyscrutinize the hiring decisions that I’m doing, so it’s thatmore careful scrutiny that causes me to not use stereotypes thanas a cognitive shortcut. Be transparent track numericalprogress. Organizations manage what they measure, and I knowlots of people here in the room do this. When you’re constantlyposting the numbers about where you are in terms of women, andother forms of diversity what you’re doing is signaling thatthese are things we need to be thinking about, and in so doingthis also helps break the tendency for people to use gender andother forms of stereotypes in making their judgments. Finally,legitimate women leaders. Those higher up in the organizationcan do a lot to ensure that women leaders are getting theappropriate amount of credit for their expertise, so a study Ilove that was done way back in 1984 found that female graduatestudents were rated more positively by undergraduate students

after a faculty member vouched for their experience andexpertise. This turns out to be important because as many of youmay know in higher education female TA’s in science andengineering classes often are rated more negatively by thestudents they’re teaching. Female faculty in those fields oftenalso experience this, so what they found is that if a professorintroduced the graduate student, and didn’t just say here’sSusan she’s gonna be our TA, but said here’s Susan she’s gonnabe our TA. Susan’s area of expertise is this, she’s writtenthese papers that what they found is this actually led thefemale TAs to have higher course evaluations having been soendorsed. I’ll wrap up there with just a summary, and a briefconclusion, so what we see here is that gender stereotypesnegatively affect women in multiple ways, and that these thingscan accumulate into larger disadvantages. We’re all prone tothese biases, so the goal here is not to blame, but to empowerwell-intentioned people. Survival skills as we’ve seen can help,but they’re rarely enough and they can actually cause sort ofthese backlash reactions as well, so to more fully reduce andweed out bias what we need really are organizational solutionsthat intervene in the tendency the natural tendency to usestereotypes as shortcuts. The goal I think is really to createmore organizations that look like this one from the computersimulation. We’d love to solve that pipeline problem at thebottom, and I know lots of people are working on that, but alsolooking at the way that we can evaluate men and women morefairly without ideas about gender influencing our evaluations.This will obviously be good for women, and here I’m gonna soundlike NCWIT a lot, but it’s also gonna be very good forbusinesses in that it would allow firms to be able to betterharness the full range of talent in their firms. Finally, it’salso as we know this is why the National Science Foundation hasbeen so interested in promoting diversity in science andengineering it’s good for the nation as well as our nation is

able to more fully harness the talent of all of our citizens.Okay, I’ll stop there. [applause] So happy to take somequestions, and I think there will be microphones coming. Yep,here come microphones down the center aisle. AUDIENCE MEMBER:Yes, I was wondering if you could speak to the mix ofparticipants in a lot of the studies you were in terms of gendermix, and underneath that are women less, more, or equally likelyto commit the male advantaging even when it’s not in their ownself-interest to do so? SHELLEY CORRELL: Yeah, that’s a greatquestion I’m glad you asked it. In the studies that I presentedthere’s a mixture of different kinds of participants. A lot ofthe experimental studies are done on undergraduate samples, butyou saw the psychology study was not. That was done on actualfaculty that were out there, and then the orchestra study wasdone on people that was actually people who were actually makinghiring decisions, so there’s a large diversity in the types ofparticipants from which these conclusions are drawn. I like thequestion about sort of are men or women more likely to engage inthese kinds of processes, and the key answer the most commonfinding you find across what are now just these hundreds ofstudies is that men and women both exhibit bias, and they do soto about the same extent. That’s the number one finding is thatthe bias is about of the same magnitude between men and women,and that were both prone to these biases, and this makes senseif you think about it because what we’re doing is drawing onwidely shared cultural stereotypes. They’re in the air we’re allsharing the same air, so I think it makes sense that we wouldsee sort of similar amounts of bias. When you do finddifferences, and occasionally in some of these studies you findthat men are more biased against women than women are.Sometimes, you find women are more biased against women than menare so, occasionally, you find sort of the other possiblefindings, but that’s far less rare. It’s interesting when Iteach this stuff in my class I always ask the students to guess

what the answer to that question is before I tell them theanswer. The students seem to guess that women are more biasedagainst women than men are. That’s always their guess is thatwomen are harder on women, so that’s what they think. It’sactually the least common of the findings, but it’s interestingthat we would have that perception. I think that perceptioncomes about to some extent because we expect more of women,right? We expect women to be more fair in hiring women than men,so perhaps we’re holding them to a higher standard even thereit’s a great question.

AUDIENCE MEMBER: Hi, I was wondering if you had any likeinternational comparisons like is the United States more or lessbias than countries we might think is offering women moreopportunities like northern European countries versus countriesthat we think of male dominated like Japan or even like SaudiArabia?

SHELLEY CORRELL: A lot of this research has come out ofsociology and social psychology which have disciplines,especially, psychology that it’s origins are in the UnitedStates. We have a lot more studies, especially, the experimentalstudies in the United States, but one of my recently graduatedPhD students did a study where she had people evaluating startupbusiness plans for entrepreneurs trying to receive venturecapital funding, and she finds a bias against women that is ifyou describe the project with a women’s name on it it seems lessworthy of funding than if it has a man’s name on it. This issomething that maybe isn’t surprising to this group, butinterestingly and to your point she did the same experiment inthe United States and in the UK, and the biases were stronger inthe UK than in the United States. That was the prediction goinginto the study because the stereotypes about men and women aremore differentiated in the UK than they are here, so that’s anexample. We have fewer experiments that are truly comparative,

and we need that kind of stuff, but what we do know, forexample, I’ve done a lot of work on the bias against mothers inthe workplace, and we do know with actual data that mothers earnless than childless women, and this is controlling for all kindsof things, what kind of job you’re in, how long you’ve been onthe job that there’s a pay penalty that mothers experience. Thatdoes very cross culturally in interesting kind of ways. It’s,for example, worse in Germany than it is in the United States,and it’s better in France than it is in the United States. Nowwhen you think about that why would that be? Germany has likethis great parental leave benefits. France has great daycare forvery young children, and the way we understand that finding, andthis is getting a little more to the point about familystructure, policies that help promote family structure,sometimes are sort of helpful, and sometimes are less helpful.In the case of Germany there’s the very strong norm that womentake the parental leave, and it pulls them out of the paid laborforce for so long that it starts to negatively affect theirwages where, for example, France has early daycare available forchildren, and that keeps women in the paid labor force, and thatdecreases the wage penalty that mothers experience. Of course,in the United States we have neither of those things so we’reright in the middle.

AUDIENCE MEMBER: I’m very aware that businesses seem less andless willing to train people over the decades, and in my agegroup that’s a way that a lot of women came into computerscience. Today I’m still in the software industry, and no oneeven seems to have a concept of training. I feel it acts againstwomen, and I wonder in your attempt to get organizational changedo you see resistance to that or are people open to it?

SHELLEY CORRELL: Yeah, you’re right. Our businesses in thiscountry, and our universities as well seem to want the peoplethey’re interviewing to already have all the skills that they

want them to have, so I do see that sort of resistance totraining people. What organizations to me seem most receptive tois, and I do a lot of this training myself is going in, and sortof doing trainings these sort of “survival” skill trainings forwomen teach women how to negotiate, teach women how to do thisor that as if women are in some ways the problem, but trainingthat might be more skills based training like can we get peopleup to speed in the way that say the Carnegie Mellon was willingto do with computer science that’s a much harder sell. I mean, Ithink there is sort of a strong sense that people should be sortof ready to go with the criteria that we have in mind.Interestingly, I do work with some of the tech firms in theSilicon Valley and I won’t name the firm, but one of the peoplewas sort of telling me about that one thing is that the men thatcame in, the entry level men just were much more interested insort of the kind of hacking culture that is computer science,and I said, well, does this even matter in terms of how peopleend up doing in their job? How much does this sort of like sortof almost nerdy programming matter in terms of people’strajectory in your firm? They said not really at all. After twoor three years nobody is doing this anyway, and I thought, well,what a shame that women are coming in, and sort of feelingreally kind of chilled out on the place when the kind of thingthat’s being sort of selected on really doesn’t even seem tohave long-term consequences for the firm, so I think it’s a realproblem, and I feel like a lot of what I do these days is try tothink of new ways to frame things to open doors to those kind ofconversations because I think it’s really important.

AUDIENCE MEMBER: So within your research or other research,obviously, an emerging issue now is gender identity not justbiological gender, but a person’s gender identity. Where doesthat play into these gender biases when we have biological maleswho are psychological females, and all those other more complex

issues of gender that HR departments are struggling with, and,obviously, in academia is a major problem?

SHELLEY CORRELL: That’s an area I have students that are juststarting to work on this. I always feel like the academicresearch has to catch up to sort of the new things that we’redealing with in the actual workplace, and we don’t have a lot ofresearch on this. There is a book that was written by thesociologist Kristen Schilt at the University of Chicago thatlooked at the experiences of trans men in the workplace, and howsort of stereotypes about gender influence them as theytransition. It’s called Just One of the Guys? That title comesabout because once they had transitioned they’re pretty muchaccepted as men by their co-workers, so that’s one of the veryfew studies that we have about that, and is certainly an areawhere we need more research, so I wish I had more to tell you,but there’s just not that much right now.

AUDIENCE MMEBER: Thanks so much it was great, enjoyed your … I’msorry I’m supposed to stand.

SHELLEY CORRELL: It’s that and the light, I can hardly see you.

AUDIENCE MEMBER: Your research because it resonates so much. I’msure that it was included in AAUW’s Why So Few? report, and isvery useful in the workplace. One of the things that wasincluded in that report was the Harvard Implicit bias site, andI didn’t know whether you recommended that, whether you’ve usedit, whether enough people because it is anonymous have beenwilling to talk about it after the fact and so on?

SHELLEY CORRELL: Yeah, that’s been a very productive area ofresearch. I understand I think somebody maybe last year came andspoke about implicit bias in this particular group, but implicitbias for those of you who don’t know is a way of detecting howquickly we make cultural associations between categories of

people, and the traits that we think that we have, so can youpush a button more quickly when you see the word science andthen a man’s name then when you see the word science, and yousee a woman’s name or something like that, and it just shows howquickly we make cultural associations that make “sense” to us inour society, and how much more difficult it is when those don’t.I find this body of research to be very convincing for showingpeople the kinds of biases that we all hold. If you’ve ever beenin one of these participations they kind of go through ademonstration with you, and you sort of see how even verbally,and even though you’re a person whose sort of actively workingon breaking down barriers you yourself are having a hard timeassociating say women with science as quickly as you couldassociate men with science, or something like that, so I thinkit’s very useful for getting people to see the way that genderbiases continue to matter, and that we all possess them, so Ithink it’s very useful in that way. I’ve not personally done anyof the research myself, but I will say another reason it’suseful is that there are some things for which implicit measuresof bias are more predictive of our behavior than explicitmeasures of bias, so if you were to ask somebody questions aboutdo you think women should be computer scientists somebody thatwould say no to that is exhibiting a rather high level of a veryconscious bias, but these sort of implicit biases, actually, insome ways can be more predictive. They actually, especially,predict the negative reactions to women who self-promote andthings like that the violations of those prescriptivestereotypes are heavily predicted by implicit bias.

AUDIENCE MEMBER: Thank you, I found that really fascinating whenyou were talking about the study … Sorry, I’m right here. Ifound it really fascinating when you were talking about thestudy that used the two different resumes, right? So there wasthe male resume, and there was the female resume, and then we

kind of saw the results of that, and I was wondering if you wereaware of any research that does that on different levels ofdifference, for instance, with names that are associated withparticular cultural groups, for instance, Latino names, Latinanames and how that might also make a difference?

SHELLEY CORRELL: It really does, it really does make adifference. There was a study that was done out of theUniversity of Chicago economics department back in I think 2002that looked at the extent to which having a common AfricanAmerican name affected whether or not somebody would be hiredinto low wage work, so that was the kind of job they werelooking at. Again, it was the same kind of design where you hadthe very same resume, but it would have say Tyrone on it insteadof Bob, or something like that, and it found a very strong biasagainst African Americans in the hiring process so the resultslooked very similar to this here, so we do have several studiesout there. Not as many as we do with gender, but an increasingnumber that at least look at the experiences of AfricanAmericans. Recently, there was another one of these kind ofstudies that was done to look at the extent to which gay menwere discriminated against. This was done by a PhD student insociology at Harvard, and basically he sent resumes out to jobsin I think five different cities across the United States. Whathe would do is that on some of the resumes you learn that theperson had had a very important position in a group when he wasan undergrad, a very important position in some sort of a gayand lesbian group, and the other person was involved insomething else, and found a really pretty substantial biasagainst gay men, although, it varied regionally as we mightexpect, so the bias against gay men was much smaller in statesthat tend to be more progressive on these issues, and worse insay the Deep South, so there have been studies that look atother forms of difference as well. What I think we have lacking

here is something that’s truly more intersectional that is thatreally brings together in the same study a concentrated focus onthe combined effect of different kinds of identities. Oneexample that I’ll give, though, that is quite striking is on themotherhood stuff, and the author of this study was interested inthe way that stereotypes about black mothers and white mothersmight play out differently when people are making evaluations,and what she did in the study it was timed to coincide rightwith Mother’s Day, so what she did is she had people, a randomsample of the United States population read about a couple thathad children, and the mother in the couple was either white orshe was black, and she was either a stay-at-home mother, or shewas employed in the paid labor force. The question participantswere asked is how much money should be spent on her Mother’s Daypresent? All these couples had the same income level, so this isa great example I think of something that’s a little moreintersectional what they found is that white stay-at-homemothers were given larger Mother’s Day presents than the onesthat were in the paid labor force, so if you were a white momyou want a big Mother’s Day present be a stay-at-home mom, butfor black mothers they got bigger Mother’s Day presents whenthey were in the paid labor force than when they were staying athome, so we sort of see the way that cultural conceptions aboutparenting are not only gender they’re also simultaneouslyraised, but these are the kind of studies I would love to seemore of.

AUDIENCE MEMBER: Thank you, I was curious to understand whatrole as you looked at this introversion and extroversion playsin that bias if any, meaning if you’re an extroverted female isthere any more or less bias versus an introverted female ormale?

SHELLEY CORRELL: Yeah, it’s like I guess the prediction would bejust in general to the extent that extroversion starts to look

anything like being assertive or what have you that that wouldlead to a violation of prescriptive stereotypes, but when peopleare extroverted they’re not always extroverted in just anassertive way they might be bubbly and outgoing, or somethinglike that as well. In these particular studies it ends up notaffecting the result. People are randomized into conditions, sothere’s equal numbers of introverts and extroverts across thecondition so it doesn’t matter in that particular way. In theone study I showed you those were actually scripts they weren’tacting, so that was kind of muted out, but in the other studiesit’s common to collect these individual personality measures,and to see if they affect our results, and with the sort ofthese sanctioning of people it doesn’t seem to matter so much.

WOMAN: We have time for one more.

AUDEINCE MEMBER: Do you see generational differences in thesciences?

SHELLEY CORRELL: That’s a great question. The content of ourstereotypes have changed over time. For example, there used tobe a pretty strong stereotype that men are more intelligent thanwomen that’s no longer the case. There’s no difference in howpeople assess just the intelligence of men and women, so thecontent of that stereotype has shifted somewhat, so we can seethat with other kinds of things. The stereotype differencesaren’t as great as they used to be on most dimensions, although,those prescriptive stereotypes that women should be nice andconcerned about others haven’t budged an inch, so we’re reallykind of stuck on those for sure, but amongst the sort of doesgender, I mean, does age of say the participants matter, yeah,you do see it. I think things are getting a bit better in thisregard that we do see sort of smaller biases in general againstyounger people. It’s not a tremendous difference, but we do seesome of that. If that sticks and doesn’t wash out as they

themselves get older that would be encouraging news.

WOMAN: I think we have time for one more, so is there a handover here?

SHELLEY CORRELL: There’s a hand over here I see.

WOMAN: Over here, oh, over there.

SHELLEY CORRELL: We may not have time she’s so far.

WOMAN: I can project [mumbles]

SHELLEY CORRELL: And we still like you.

AUDIENCE MEMBER: So as you were talking about generationaldifferences because I’m looking at the three male leaders in myoffice, and we only have three leaders and they’re all male, allof their mentors were women. Lucy and I met with my boss and hehad to leave because he had to get the kids from daycare becausehis wife had to work, so as you see that shift and actuallyevery man in my office when their kids are sick they stay homefrom work because their wives have jobs is the response we getreal quick. My wife has a job I have to go home. Do you see thatchanging, you know, there was the Eagly study that was done outof Harvard showing that there is this shift in the role men aretaking at home in terms of housework, cleaning, et cetera, doyou see this shift in how those men as they get into positionsof power how they drive that down?

SHELLEY CORRELL: Yeah, I mean, we have seen a shift in men doingmore housework, and in particular doing more child care that’swhere we’ve seen sort of the biggest increase there, so that’sanother sort of encouraging trend, and in some ways if you thinkabout women who are partnered with men, and they have childrento the extent that men are doing some of that it also means thatwomen aren’t having to do as much which is good for avoiding

some of the bias that mothers face, so I do think this is atrend that we’re seeing, but at the very top, I mean, here itdepends on what kind of job we’re talking about. At the very topwith the sort of the more extreme careers people who work over50 hours a week, for example, we don’t see as much help from menin that sector, so that’s an area where we still I think reallyneed some improvement, so women who are married to men who workmore than 50 hours a week, for example, are very likely to cutdown on their hours, or opt out of paid labor the other wayaround it doesn’t happen. Men’s hours are not really responsiveto how many hours their wives are working. There’s still a lotof improvement that I think needs to be made, but it isencouraging. Somebody who teaches college students at an eliteuniversity I’m very encouraged at least by what they say at thispoint in time about how they want their lives to be in terms ofbeing fully involved with their children, and having a partnerwhose fully involved in the paid labor force.

WOMAN: Thank you, please join me in thanking. [applause]

Vimeo ID 45873136