social glue: everything i learned about coding from diego, a harris's hawk

Download Social Glue: Everything I learned about coding from Diego, a Harris's Hawk

Post on 16-Apr-2017




1 download

Embed Size (px)


Social Glue

Social GlueRachel ChalmersIgnition Partners


I recently got to fulfill a childhood dream and the reason I was able to do it is pretty interesting. If you grew up dreaming of being a falconer, and who among us did not, you might know that the raptors used in traditional European falconry are all clinically insane, or at least so terrified of humans that they take a long time to acclimate and will usually only tolerate the presence of one or two specific handlers.But this beauty here is not a traditional European falcon. Its a Harriss hawk, a native of the Americas from the Sonora desert down to Argentina and Chile. Unlike almost every other raptor, Harriss hawks are cooperative hunters. They form packs of two to six, and theyll scout ahead for one another, or theyll split up so that some are flushing out prey and the others come in for the kill.


Harriss hawk has changed falconryHighly intelligent, social raptorsTraditional falconry methods dont applyPack hunters, like wolves, humans, chimpanzees, dolphins, orcas, lions, dwarf and banded mongooses, hyenas, butcherbirds and three out of four kookaburrasCooperative breedingAllomaternal behaviorInclusive fitness

Now if youve ever read any of Jared Diamonds books that touch on the domestication of animals, youll recognize this combination of a calm temperament and a pack mentality as a set of traits that really lends itself towards animals forming cooperative relationships with human beings. And sure enough, the Harriss hawk has been used more and more in falconry since the 1980s, so much so that the definitive book on the topic is called the Harriss Hawk Revolution. Its because these hawks are are so social and friendly that an amateur like me can go out I went with West Coast Falconry, near Sacramento and have a hawk fly onto my glove. And while theyre American natives, theyve become the most widely used hawks in the West. Theyre used for pigeon control in Trafalgar Square.

Two more cool stories about the sociability of the Harriss hawk, as told to me by Master Falconer Jana Barkley. One wild pack was observed to have an adult male member with a deformed talon, and this guy became a stay-at-home dad. Hed care for the fledglings while the other birds brought back meat from the hunt. The other story, that really blew my mind. Not only can you hunt Harriss hawks that are already bonded to each other; groups of falconers can bring their own Harriss hawks that have never met before and form spontaneous packs. The hawks will figure out their own pack hierarchy and hunt together.


Collective intelligenceA female-male cast of Harris's hawks gives the best of both worlds. It is a pleasure to watch the two hawks simultaneously compete to get to the quarry first, then cooperate in catching it. The flying skills of the female and male are complementary -- the aerobatic male frequently forces the quarry to leave cover just as the female closes in at maximum speed to deliver the final powerful stroke. Toby Bradshaw, falconer

All in all, theyre the kind of keystone predator that, like us, benefits from a range of cooperative behaviors, like kindergartens and specialization. I especially like this quote for the way it illustrates that the hawks are putting their differences, not just their raw numbers, to good use. 4

Here they are, doing their thing. Obviously Id like us to take some lessons from the Harriss hawk like hey, maybe we should be highly intelligent and social too, but I dont want to be too motherhood and apple pie about it. Problem-solving in real time which is fundamentally what both hunting and programming are is difficult any way you slice it. Lets acknowledge the difficulty of whats been undertaken here, and then look at a couple of approaches that I think look promising in how to address it. 5

Raw numbers are a mixed blessing in any case: I mentioned that the optimal cast size the cast being the hunting group is 2 to 6 members, which maps pretty precisely onto everyones rule-of-thumb for good software development or indeed any other team sizes. 6

Weve known for forty years that the communications overhead involved in adding people to a project increases exponentially as the number of people involved increases arithmetically. In other words, the bigger the team, the less effective it is in working together at a common task. Thats why Fred Brooks called his classic book on the subject The Mythical Man-Month. Brooks worked at IBM where he managed the development of the OS/360 operating system. The project ran late, and he added more programmers, and in the end he concluded that his doing so had made the software ship even later. Brooks argument is that theres no such thing as a man-month. You cant speed up software projects by adding people any more than you can speed up a pregnancy by having nine women carry the baby for one month each. 7

You can actually calculate the communications overhead of adding people to projects as n squared minus n over 2, where n is the number of people and the answer is the number of links. 8

And when you plug that back into a guesstimate for optimal team size, to figure out where the productivity of the group isnt being overwhelmed by the noisiness of the system, lo and behold, you get teams in the exact range of casts of Harriss hawks: 2 to 6. It seems like this might be an actual hard physical limit of the universe or maybe just of carbon-based avian and mammalian brains, but a hard limit nonetheless. So heres the question I have for you, my fellow attendees at Gluecon: can we break it? 9

The Mythical Man-MonthAdding manpower to a late software project makes it laterComplex programming projects cannot be perfectly partitioned into discrete tasks

Because thats not the only assertion that Fred Brooks made in the Mythical Man-Month. The other thing he claimed is that complex programming projects dont lend themselves to being tackled in discrete parts. 10

I didnt want to take this at face value without checking with my guru, Edsger Dijkstra, who taught me everything I know about computer science. Dijkstra was a Dutch supergenius who in the 1970s and 1980s wrote a ton of short essays about all kinds of topics and circulated them with his colleagues and students you might recognize the phrase Goto Considered Harmful - and these have all been collected and published online by the University of Texas. Theyre amazing, I keep coming back to them and learning more every time. Dijkstra believed that programming was a branch of mathematics, that learning the bells and whistles of different programming languages was a distraction, and that the core job of a programmer was to sit in front of a blank sheet of paper and ponder until she reasoned herself towards an algorithm that would solve her problem. 11

Edsger DijkstraOn the teaching of programming, i.e. on the teaching of thinkingUnless the interfaces are indeed sufficiently thin, communications problems become prohibitively severe.

So I reread an earlyish Dijkstra essay, EWD473, On the teaching of programming, ie on the teaching of thinking, where he tackles this question of modularization that Brooks has brought up and acknowledges that yeah, the communications overhead is a thing unless your interfaces are sufficiently thin. And I thought, huh, what does he mean by that? 12

Breaking up the problemHere, however, we see in all its clarity a quite different and more fundamental purpose of modularization, viz. to reduce the total reasoning (irrespective of whether thereafter it will be done by different groups in parallel, or by a single one in succession.)

Heres what he means. Here, however, we see in all its clarity a quite different and more fundamental purpose of modularization, viz. to reduce the total reasoning (irrespective of whether thereafter it will be done by different groups in parallel, or by a single one in succession.) Okay, so English was his second language, but what hes saying here is really awesome. You can use that pondering step to think about how to break the problem up, and then, if you have good interfaces, you can parallelize your two to six person teams to work on really, really huge problems. Such as for example Web-scale IT. 13

Breaking up the problemSolving large problems is very hardBreaking down the problem is itself a problemBut it can be doneGood interfaces make good solutionsInterfaces are cultural as well as technical

I think what were seeing right now in Web-scale software engineering are a couple of very promising approaches for routing around the hard upper limit of team sizes. One approach is technical lets call it modularization, although Im thinking specifically of the proliferation of services exposed through clean APIs. And, more recently, of the proliferation of containers, which were going to hear plenty more about from our next speaker, since hes the person responsible. The other approach is cultural. Lets call it devops, although again, thats only an approximation. The argument I want to make to you today is that these two approaches are inextricably linked. Its the combination of the two that I call social glue. 14

I was pretty sure you were going to want some practical examples, and ones that werent from the 1970s. Although Im sure Flickr seems that long ago to some of you. Flickr started out as a company called Ludicorp, which built a virtual world called Game Neverending. It was the Second Life of its day. It failed, and was resurrected as Glitch, and failed again. But Flickr, the Instagram of its day, was originally built as a photo-sharing application for use within Game Neverending, and in case youve ever wondered, thats why Flickr filenames use the extension .gne. I still really love Flickr, but to me the most influe

View more >