I Have No Idea What I’m Doing

Writing this talk

The most relevant example is the process of trying to write this talk. Most of the talks I give are teaching in one form or another, so in a typical talk I take someone else’s idea and explain it by deconstructing it into pieces that are easier to understand. But this talk’s different, because I didn’t know in advance what I wanted to say; I didn’t really have any ideas, just an oblique title written on an index card.

I sat and stared at it for a long time and didn’t know what to do. Eventually I started making notes. I diagrammed some vague ideas on paper, then I opened OmniOutliner and started making an outline. I wrote a list of big-picture ideas and broke each of them down into smaller ideas, filling out each small idea with as much information as I could think of. In my desperation I kept adding more material — ideas, drawings, pictures, quotations, pieces of research — until the outline became vast and unmanageable.

I walked away from the computer and gave up. I didn’t have any idea how to write this talk or what it should be about. That was disappointing.

Writing a book

Another example: last year I was supposed to write a book that used Ruby to explain computation theory. I was initially very excited about this — the contract was signed, and I was really looking forward to being an O’Reilly author — but eventually the moment came when I was sitting alone in front of a blank text editor. I panicked a bit when I realised I’d not written many things longer than a tweet before, so I certainly didn’t know how to write an entire book.

I wrote pages and pages of notes to try to get my thoughts into some kind of order, which didn’t really work. I bought a stack of index cards, filled them with ideas, and spent hours arranging and rearranging them on the floor to see if they’d fit together; that didn’t really work either.

I did lots of hoovering, I did lots of washing up — my flat has never been so clean — I stared out the window, and I didn’t know what to do. That was a disappointment as well.

Writing a Scheme interpreter

And then there’s the time I tried to write a Scheme interpreter in Ruby.

Some friends and I decided to start a book club for The Little Schemer; it’s a classic of computer science but none of us had read it, so we started meeting up regularly to work through it together.

The Little Schemer has an interesting format. Each page is divided into two columns: the left column asks questions about Scheme expressions, and the right column answers them. The entire book is a dialogue between the two sides of the page.

Since we were all Ruby developers, we thought it might be fun to turn these questions and answers into RSpec examples. The idea was to work through the book and make those examples pass as our understanding of Scheme improved.

I converted the first few chapters into RSpec examples in advance of our first meeting, so that we could immediately get started on making them pass. As I did the conversion I quickly tried to make each example pass on my own, just to check that I hadn’t made any careless mistakes in transcribing the information from the book.

So as I wrote these examples in spec/ I was also putting implementation code into lib/, and by making the examples pass I was gradually implementing a Scheme interpreter, because the book essentially explains how to evaluate Scheme.

Here’s a graph of how many lines of code I wrote over time:

The red line is how many lines of examples I had in spec/, and the green line is how many lines of implementation code I had in lib/. The distressing trend is that lib/ kept getting larger: every time I added examples of new features, my interpreter grew to support them, and it soon became really big and messy.

In the end I did make all the early examples pass, but I didn’t feel as though I’d properly understood anything. I hadn’t learned anything deep about Scheme or obtained any insight into the problem I was trying to solve. My interpreter would soon be so complex that I wouldn’t be able to make any progress, and I still wouldn’t have grasped anything interesting about Scheme. Again, that was disappointing.

The impostor syndrome

I’m talking about you specifically here; you know what I’m talking about. Sometimes you think things like “I’m just making stuff up as I go along”, or “I don’t belong in this group of people”, or “I’m going to be found out”.

Those feelings are a symptom of the impostor syndrome. Wikipedia says:

The impostor syndrome […] is a psychological phenomenon in which people are unable to internalize their accomplishments.

That means that when you succeed, you’re not necessarily predisposed to incorporate that information into your self-image; you keep the success at a distance, and your self-image remains unchanged.

Despite external evidence of their competence, those with the syndrome remain convinced that they are frauds and do not deserve the success they have achieved. Proof of success is dismissed as luck, timing, or as a result of deceiving others into thinking they are more intelligent and competent than they believe themselves to be.

I’m not an expert on the impostor syndrome. If you’d like to learn more about it, I recommend Julie Pagano’s talk from Nickel City Ruby (and PyCon), as well as Tim Chevalier’s four-part blog post and Ruby Rogues episode.

The Dunning–Kruger effect

The impostor syndrome isn’t the only way to misjudge your own competence. You might also think things like “there’s so much that I don’t know”, or “any idiot can write a Ruby program”, or “everyone here’s a better programmer than I am”. Those thoughts can be a result of the Dunning–Kruger effect, which was identified in 1999 in a paper called Unskilled and unaware of it: how difficulties in recognising one’s own incompetence lead to inflated self-assessments.

The paper’s quite well-known now; it was relatively widely publicised at the time because it won the Ig Nobel prize for useless psychological research. It summarises and analyses experiments conducted by David Dunning and Justin Kruger in which they asked university students to evaluate their own competence.

The students were given tests which assessed their sense of humour, logical reasoning skills and English grammar skills, and then asked to guess how they’d performed relative to the other participants — they had to decide whether they’d performed better than ten percent of other people who took the same test, or fifty percent, or ninety-nine percent, or whatever.

(The first thing the paper reports, incidentally, is that “gender failed to qualify any results in this or any of the studies reported in this article, and thus receives no further mention”. So maybe gender essentialism is nonsense, and gender has no bearing on competence. Who knew?)

The paper presents this graph of the humour test results:

The lighter line shows the actual results, and the darker line shows the participants’ perceived performance. You can see that most people’s perceived performance is way above their actual performance: the people in the lower quartile thought they’d performed better than perhaps sixty percent of other participants, whereas in reality they’d only done better than between zero and twenty-five percent of them.

On the other hand, in the upper quartile you can see that the highest-performing participants have gone in the opposite direction and underestimated their own performance. (The paper’s authors argue convincingly that this isn’t a regression effect.)

The grammar test took things a little further. After each participant had answered the test questions and estimated how well they’d performed, they were shown some other students’ answers and given the opportunity to re-estimate their own relative performance. The intention of this was to find out how someone’s assessment of their own competence would change after they’d seen how well (or how badly) others had done.

The study found that people who performed poorly on the test not only overestimated their own competence initially, but also maintained that overestimate even when they’d seen other people’s answers:

Despite seeing the superior performances of their peers, bottom-quartile participants continued to hold the mistaken impression that they had performed just fine.

Conversely, better-performing individuals did improve the accuracy of their self-evaluations, because seeing other people’s papers helped them to realise that others hadn’t done as well as them:

Simply put, because top-quartile participants performed so adeptly, they assumed the same was true of their peers.

The most interesting summary of these results is this sentence from the paper:

The miscalibration of the incompetent stems from an error about the self, whereas the miscalibration of the highly competent stems from an error about others.

That means that if you start out with low competence in a particular domain you’re likely to be mistaken about your own ability, but as you improve you become more likely to be mistaken about other people’s abilities instead. Instead of assuming that you’re comparatively good, you begin assuming that everyone else is good as well, because as you become more capable you start to think “well, surely everyone else knows this too”.

In the paper’s conclusion, its authors say:

One way to make people recognise their incompetence is to make them competent.

By educating someone who doesn’t know what they’re doing, you give them not only the specific skill in question but also the metacognitive skill of being able to more readily spot their own inadequacies; you make them better at what they do, but an unintended side-effect is that they’ll spend more time thinking that they’re not especially good, because they’ll assume that everyone else is just as good as them.

Douglas Adams

The first person I wanted to mention is Douglas Adams, who wrote a radio series called The Hitchhiker’s Guide to the Galaxy which subsequently became a book, TV series and film.

He wrote the radio series by the skin of his teeth: budgets were small and deadlines were tight, so he was writing each episode immediately before it was recorded, and each episode was recorded immediately before it got broadcast.

In the second episode of the series, its protagonists are thrown out of an airlock without spacesuits on. This situation is quite perilous and dramatic, but unfortunately Douglas Adams didn’t have a plan for how he was going to get the characters out of it.

He said:

I carelessly thought that it might be fun to have Arthur Dent and Ford Prefect thrown out of the airlock of a Vogon ship without spacesuits, just to see what would happen. Unfortunately, of course, if anything was going to happen, I was going to have to think of it. I got very stuck.

How irresponsible. Who writes like that?

Animals

Animals have no idea what they’re doing either. Here’s a picture of an Indo-Chinese flying lizard:

It’s difficult to see because it’s on a tree and it has extremely detailed camouflage. You can just make out its head, front legs and tail, but it’s otherwise almost completely invisible.

These are peppered moth caterpillars:

The left one is on a birch tree, and the right one is on a willow tree. Again, these have remarkable attention to detail in their camouflage; the right one has little markings that look just like the markings on a willow branch. They both strongly resemble the trees that they live on.

Here’s a stick insect from Australia that looks exactly like a eucalyptus twig:

All of these animals look remarkably like something in their environment, but none of them has any meaningful understanding of why they look that way. They don’t know what they’re imitating. The stick insect doesn’t know anything about eucalyptus; that’s just what it looks like.

Here’s a giraffe:

A giraffe has a very long neck, which allows it to browse the tops of trees free from competition from other leaf-eaters. But no giraffe appreciates that advantage, or even understands it; they just get on with eating leaves.

It’s not the just the bodies of animals that fit snugly into their environment, it’s their behaviour too. This is a North American beaver:

Beavers build dams.

Beaver dams are made from wood, mud and stones. Beavers cut down trees with their teeth and use the timber to divert a stream and build a dam across it, creating a pond. The pond protects the beaver against predators, provides them with food, and prevents the underwater entrance to their beaver lodge from blocking up with ice in the winter.

Dams are really impressive structures. They’re a couple of metres high and they can be tens or hundreds of metres long. Building one is quite difficult — it involves understanding the weather and the seasons and the way that water flows — but of course beavers don’t know anything about meteorology or hydrodynamics or planetary axial tilt.

If you only take one thing away from this talk, make it this: beavers are idiots. They have no idea why they’re building these huge structures; they just blindly do it.

Shoshin

One possible solution is shoshin, a concept from Zen Buddhism.

Shoshin means “beginner’s mind”. The idea is to approach a problem as if you knew nothing about it, like a beginner learning about that problem for the first time. Shoshin is about abandoning your preconceptions, being receptive and eager and open to possibilities without making premature decisions about what’s possible.

A Zen teacher said: “in the beginner’s mind there are many possibilities, in the expert’s mind there are few”. Sometimes being an expert can work against you, because it leads you to decide on a solution before you’ve had a chance to properly think about the problem. Beginner’s mind allows you to explore a problem more authentically and pay more attention to its characteristics, rather than constantly jump to conclusions based on your experience. Experience is valuable, but not as valuable as the ability to pay proper attention to the problem in front of you.

Judo

Another useful strategy is judo. (I don’t mean the Basecamp lingo.)

Judo is how Douglas Adams solved his problem with Ford Prefect and Arthur Dent being thrown into deep space:

The problem was the sheer improbability of every solution I came up with. [The answer] came about through watching a TV programme about Judo.

If you have a problem, said the instructor on the programme, such as for instance a nineteen stone [man] in pyjamas trying to beat you into a pulp, the trick is to use this problem to solve itself. If you can trip or throw or deflect [him] as he hurtles towards you, then the fact that he weighs nineteen stone quickly becomes his worry instead of yours.

So — I thought — if my problem is one of improbability, let’s use Improbability to solve the problem, so just for the heck of it I invented the Infinite Improbability Drive, and gave myself a whole new thing to write about.

By putting himself in a difficult situation without thinking about how he was going to get out of it, Douglas Adams forced himself to think creatively and use the problem to solve itself. It was only by realising that the real problem was the improbability of all of the easy solutions that he was able to come up with the idea of the Infinite Improbability Drive, which was funny and creative and turned out to be a good source of plots later in the series.

Rolling up

There’s a video game called Katamari Damacy in which you play a character, the Prince, whose job is to roll a magical ball called a Katamari around a three-dimensional world.

The Katamari starts out small and grows by picking up objects as you roll it. When it’s small the only things that stick to it are tin cans and stationery and bits of rubbish, but larger objects like animals and trees can become stuck to it as it grows. Over time it gets big enough that cars and buildings and even continents can stick to it.

I find this game fun because the Katamari grows by rolling up the environment — it’s made of drawing pins and cows and clouds. Pieces of the world that start out as obstacles can become beneficial once the Katamari gets large enough to roll them up, and the more of the environment the Katamari absorbs, the more you’re able to do with it.

The way a Katamari grows is reminiscent of how biological evolution works. Jean-Baptiste Lamarck, an eighteenth-century French naturalist, made lots of important contributions to evolutionary theory but unfortunately popularised one idea that turned out to be wrong, and now that’s all anybody remembers him for. His theory, Lamarckian inheritance, was the idea that an organism can acquire characteristics during its lifetime and then pass them on to its offspring. (This is sometimes called inheritance of acquired characteristics.)

Lamarckian inheritance can be used to explain the evolution of giraffes’ long necks. The story goes that ancestors of modern giraffes, with their normal-length necks, were constantly reaching upwards to try and eat leaves from trees, exercising their neck muscles and causing them to strengthen and elongate over the course of a lifetime.

Then, when two of these ancestral giraffes mated to produce a baby giraffe, the baby would be a kind of copy of the parents (the mechanisms of inheritance weren’t well understood in Lamarck’s time), so the baby would be born with a slightly elongated neck just like its parents. As the baby grew into an adult it would repeat the same pattern, reaching for leaves and lengthening its neck further, so that its own offspring would have longer necks still, and over many generations its descendants would eventually develop extremely long necks.

Unfortunately for Lamarck, this is nonsense. The year that he published this idea, Charles Darwin was born, and it was Darwin who came up with the correct explanation.

Darwin realised that adaptations like long necks are the result of differential reproductive success of individual organisms in the presence of random variation: firstly, the random recombination of genomes that happens when two animals mate to produce offspring, and secondly the occasional introduction of random mutations into that process.

Mutations are usually disadvantageous and produce uncompetitive offspring, but occasionally, purely by chance, a mutation occurs that makes an animal slightly better adapted to its environment, giving it a slightly higher chance of surviving and a slightly higher chance of reproducing. That reproductive advantage allows the mutation to become more widespread in the gene pool, and as that process is repeated with each successive generation, more and more information from the environment gets rolled up into the genome of a species.

Over evolutionary time, organisms work like sponges. They absorb information from their environment, they become better and better adapted to that environment, and their genome begins to encode more and more information about it. That’s how you end with insects that look a lot like willow branches or eucalyptus twigs, and beavers that know how to modify wetlands for their own benefit.

In the case of the peppered moth caterpillars, birds are the conduit for this information. When a bird looks at a tree, it’s less likely to spot and eat a caterpillar that looks a bit more like a branch than the other caterpillars on the tree. The brains of birds have transmitted this information from the caterpillar’s physical environment into its genome over hundreds of thousands or millions of years.

Material exploration

Rolling up is also how material exploration works. Material exploration is an idea from the world of design — designers are taught about it, but programmers aren’t, and I don’t know why.

Material exploration is about finding the grain of a material by trying things out and seeing what works. The idea is to allow the structure and properties of your material inform your creative process, rather than simply imposing your will and preconceptions onto it.

A “material” in this context could be something tangible like wood. Wood has got a grain — the fibres of cellulose point in one particular direction because of the way that wood grows — which gives it an internal structure that you can work with. Depending on whether you cut a piece of wood along the grain or against the grain, you may end up with smaller pieces that have different structural, textural and visual properties. The cut pieces might be flexible in one direction but not another; they might be very brittle, or very strong, or have an interesting pattern on their surface. By working directly with wood and trying different things with it, you can make discoveries about its behaviour that you might not have otherwise predicted.

Alternatively, your material could be something intangible. Timo Arnall and Jack Schulze from the London design consultancy BERG worked on a project a few years ago to investigate the shape of the electromagnetic field emitted by RFID readers. They pointed a locked-off camera at an RFID reader and used a probe with an RFID tag and an LED to detect the edge of the field; every time the reader recognised the presence of the tag, the LED on the probe would flash.

By taking a series of photographs of the LED flashes they were able to reconstruct an image of the three-dimensional field around the tag reader:

Through this process they were able to visualise how variations in the size and shape of the RFID antenna affect the shape of the field. The rectangular antenna in an Oyster card, for example, produced a very different shape:

This is very useful information if you intend to build something that uses RFID. Having this sort of insight into the material you’re working with — what shape the field is, how close the tag and the reader have to be, what effect the tag’s orientation has — allows your designs to be informed in a way that wouldn’t have happened if you’d sat down and thought about it instead of allowing the material to guide you.

Software is itself an intangible material. This man is a lead programmer at a games company called Naughty Dog, and he’s playing a debug build of a video game called Uncharted 2 on the PlayStation 3:

Superimposed on his screen is live debug information. This footage is taken from a video in which Naughty Dog talk about making the best use of the PlayStation’s Cell processor architecture, and all this information on the screen is presumably visualising data about what the processor, graphics pipeline and GPU are doing.

You can’t do material exploration without having proper visibility on how the material is responding to you, so doing something like this — having a visualisation of what’s going on — gives you the opportunity to feel your way around the problem and get a sense of whether your work is going in the right direction.

Test-driven development

Rolling up is also a big part of test-driven development (and I suppose agile development generally). TDD encourages you to work iteratively by making small changes, and to impose selective pressure on your implementation by writing tests.

When you’re doing TDD, you write tests to characterise new features or to reproduce existing bugs. Acceptance tests roll up information from stakeholders about what the software is supposed to do; regression tests reproduce problems by rolling up information from the world about weird edge cases, acting as a sponge that soaks up knowledge from bug reports and stores it somewhere useful.

This is one reason why rewriting software from scratch can be so dangerous. Old software has all of this knowledge already soaked into it; it bristles with fussy little fixes and weird little features that each represent a piece of information from the environment in which it’s being used. If you throw old software away and replace it with shiny new software, you run the risk of jettisoning all of that knowledge that’s been painstakingly rolled up into it. Putting that knowledge into a test suite helps to lock it up in a place where it can live longer than the implementation it’s testing.

TDD encourages beginner’s mind as well, because when you’re working test-first you can write the tests and then make them pass without thinking too far ahead and without getting too invested in any particular implementation strategy. TDD practitioners sometimes speak about first writing the test while thinking hard about the problem, then switching into the different mode of focusing on simply trying to make the test pass. That mode switch makes it easier to have beginner’s mind with respect to the larger problem you’re trying to solve, because you can focus on the small task of getting a test to pass and not become distracted by your expert preconceptions about the big picture.

That process encourages material exploration as well. It allows you to be guided by the grain of the problem, rather than just charging ahead and hacking out the solution you were already thinking of. It gives you a chance to take things more slowly and to feel your way through the obstacles that have been placed in your path by the test, helping you to channel your energy in what is more likely to be a productive and creative way.

And there’s a little bit of judo in TDD, I think. If you use tests to characterise a problem, that makes it easier to use the problem against itself when you’re writing the implementation, letting each failing test guide you incrementally towards a solution. Once you’ve put the problem’s constraints in place, they might help you to see how you can solve it in an elegant way, leading you towards a more natural solution which mirrors the structure of the problem.

Refactoring toward deeper insight

“Refactoring toward deeper insight” is a phrase from Eric Evans’ book Domain-Driven Design, and it’s how I fixed my Scheme interpreter.

To recap, this was the situation I found myself in:

I got a bit dispirited at this point, so I put the project away for a bit and came back a few weeks later with a clearer head. This time I tried to be a little more patient and calm and pay more attention to what was going on in the code; previously it had been a mad rush to try to get the examples written in time for the book club.

When I came back to it I did some more thoughtful, more patient refactoring, and an interesting thing happened:

The point where the green line drops off is where I did some refactoring and discovered some hidden structure in my implementation. The refactoring wasn’t just moving furniture around; I was doing mindful stuff like eliminating duplication, introducing new abstractions, looking for interesting patterns, and to a certain extent thinking about what the book was trying to say.

As I thought more about it, and tried to arrange the code in a way that better reflected what the book was trying to say, suddenly different pieces of my implementation began clicking together. Whole classes collapsed down to single concepts, and I had an amazing few days where I was able to delete a lot of code as more and more concepts clicked into place. That was the point when I started getting some insight.

Now that I’ve gone through that process, I feel like I finally understand what’s amazing about Scheme. Writing the interpreter had felt like a real pain at first, and it wasn’t until I explored the code by refactoring it — and saw how it was supposed to work — that I felt like I was getting something out of the exercise.

Play

The last solution I want to talk about is play.

Play was how I dealt with the problem of writing my book. Given that I wasn’t getting any writing done anyway, I gave myself permission to mess about with fun little programs instead. I already had a list of interesting computer science ideas in my head, so I spent a couple of weeks writing lots of small programs, playing with each of those ideas in code form. I wanted to see if it was possible to make little Ruby programs that were fun to read, fun to write, and that showed the underlying concepts clearly.

Some of those programs didn’t go anywhere — they felt a bit boring, I wasn’t really enjoying them, and they naturally fell by the wayside — but some of them were fun, and I kept playing with the fun ones. The fun programs grew new features as I put more ideas into them, and they gradually matured into usable pieces of example code. Those pieces of example code began to grow prose around them, and that prose eventually turned into chapters of the book.

My own curiosity and playfulness was the source of noise in this system, and my sense of what was interesting and fun exerted selective pressure on the programs I wrote, allowing the best ones to survive and blossom into entire chapters.

Incredibly, that kind of worked. I did eventually end up writing a book, and now I’ve got a box of unsold books in my flat:

You don’t really make any money by writing a book, but O’Reilly did send me a nice hat:

My other reward was getting a really good review of the book on O’Reilly’s site, which I’m very proud of:

it is quite useful . technical but not so technical that it makes me want to stop reading it.

That makes it all worthwhile.