Analogue Exams in a Digital World: Why We Urgently Need to Rethink How We Test Augmented Minds

Would you remove the bike from a cyclist just before their race? Or golf clubs from a tour pro before they stepped onto the first tee? Didn't think so.

Picture this. You’ve been training for two years for the Tour de France. Every day in the saddle, putting in the kilometres, suffering through saddle soreness, cramped muscles and exhaustion. But every time you’re out on your bike you feel increasingly connected to it - in fact at times it’s hard to know where you end and the bike begins. 

Or you’re a pro golfer. Practising every day for two years in preparation for the British Open, hours at a time on the practice ground and the course, swinging the club until you have blisters on your blisters. And you reach a point where the club feels like an extension of your arm. 

Then you know you’re ready. To take your place on the starting line, or the first tee. To compete against others, but also against yourself. 

Then something happens. Something quite bizarre and ridiculous. An official comes to you, holds out his hand and says ‘I’ll take that, thank you very much.’ And removes your bike. Or your golf clubs. 

But. You still need to complete the race, or the round. A 250km stage of the Tour on foot, or throwing a golf ball around eighteen holes.

Naturally, you do terribly. And not only that, but with every step, or every badly thrown ball, you become increasingly angry and frustrated. You’ve spent two years preparing, becoming one with the tools of your trade. And now you have to compete without them. 

Sounds crazy, right? 

But in a way, this is what we do every exam season with our students. And have been for at least twenty years. And it’s only getting worse. 

The Mind is More than the Mind

If we have a tricky maths problem to solve, what do we do? We work it through on paper. Trying out ideas, scratching some out, moving ahead slowly until we crack it. What we don’t do is figure out the whole thing in our head (unless we’re some maths genius, which isn’t 99.999% of us). 

What’s going on here? Is the paper simply a way to record what our mind is doing? Or is it something more?

What if the pen and paper were part of the mental activity: that we actually think on paper? This is the foundational idea of Andy Clark, who in his book Supersizing the Mind suggests that “The loop through pen and paper is part of the physical machinery responsible for the shape of the flow of thoughts and ideas that we take”. To Clark, “the outward loop [is] a functional part of an extended cognitive machine.” These loops “supersize the mind”.¹

It makes sense. The maths problem we worked through didn’t only need pen and paper in support. The mental process could not be done without them. It was the physical movement of pen over paper that formed part of the cognitive process. Remove the pen, and you remove part of the process. 

Most of us can’t go through an extended thought process in any other way. We have to get stuff down on paper. Therefore, the pen and paper become part what Clark calls an “extended cognitive circuit”. The pen and paper extend the mind. They are not separate from the process from problem to solution. They are not simply tools, but rather augmentations of our mind, without which our minds cannot be what they are. 

Like the cyclist not knowing where they end and the bike begins, so there is no division between mind and pen. The pen is the mind, physically extended outside the walls of the body.

The Pen is Lowlier than the iPhone

It therefore follows that when it comes to pen and paper exams all is good, because we allow students these augmentations in order for them to answer questions. In fact, we welcome the ’show your workings’ approach in most subjects like maths, so we can see how a student worked through to an answer.

But probably since the mid 1990s, and certainly since the early 2000s, our analogue augmentations are no longer all we have. Throughout my schooling in the 1980s I can’t think of many digital augmentations, save for the trusty Texas Instruments scientific calculator none of us knew how to use. It was pen and paper all the way. I did my workings on paper, wrote essays on paper, recorded all my class notes on paper. We had a few BBC-B computers in the IT room but it was a rare and special occasion when we were allowed to use them. I think we did very basic programming, but never used them for any non-computing work. 

You could therefore say that I was tested in the right way. The tools of my extended cognitive machine were the pen, ruler, compass and set square, and not much else. We weren’t even allowed to use that ridiculous calculator in most of our maths exams. 

But, to return to the analogy that began this post, that’s no longer the case. The simple pushbike has been replaced not only by a lighter and more efficient racing bike, but by a superbike with a five hundred horsepower, computer controlled engine. Or steel and wooden golf clubs replaced by carbon fibre clubs capable of hitting the ball one hundred yards further than ever before.

This is what our students cognitively ride or swing every day whenever they use their smartphone, iPad, laptop - and now AI. These super fast, super intelligent machines and systems are now part of their external loop. They didn’t ask for them, but have had them since they were young. And these tools are developing in raw power all the time. 

Seen from an intellectual augmentation point of view, the pen is an infinitely inferior tool to the smartphone. It extends our intelligence only inasmuch as enables us to spatially organise our thoughts. We could argue that it is in this spatial organisation that our minds are augmented and we are able to think harder, faster and in more depth than if we had to only use our minds. But the loop only extends as far as the pen in our hand. It goes no further. 

But with the smartphone, that loop now reaches across the globe, connecting our minds to everything, everywhere, all at once. Access to the internet, social media, video, podcasts, and new ways to record and share ideas means that it is difficult to see it as a loop any more. We are drawing on an infinite number of nodes, sending requests out into the world and receiving information back at the speed of light. The loop becomes more of an extended neural network, each connection we have out there in the world like a synapse, each new piece of information we send or receive like a neural impulse. 

And with AI this exponentially increases, as we are now connected to forms of artificial super intelligence that will likely soon be powered by quantum computers. We will find ourselves increasingly enmeshed with AI, our minds and these artificial minds joining together to synergise. Networks within networks, nodes within nodes, without beginning or end. 

This is our reality. What we need to understand is that this isn’t only about us using the technology in our lives as tools. The tools are us. There is no longer a division. 

Thinking Different

I’ve mentioned in previous articles this idea that we have moved from the Anthropocene period into what James Lovelock has coined the Novacene: a period defined by its dependence on the AI machines in our lives to take on an increasingly fundamental role in our existence and survival. And that we are now Homo technos as much as we are Homo sapiens. That we are all cyborg, part human, part machine. The thinking of Andy Clark fits neatly into this hypothesis. If we agree that the tools we use that rest outside our minds and bodies are indispensable to how our minds operate, then the more these tools advance, the more our minds (and bodies) should in theory concomitantly advance. 

The idea that an over reliance on technology will make us stupid is therefore a fallacy. It’s not that our minds will stop working - rather, they will work increasingly differently from before. Yes, some of the functions of our thinking minds may diminish (indeed they already have, as our ability to read maps has drastically reduced since the advent of GPS), but they will be replaced by growth in other areas (remembering that nature abhors a vacuum). We shouldn’t therefore worry about AI making us all dumb. We’ll just start to ‘think different’ (to borrow Apple’s much-used slogan). 

Lovelock said it way better than I ever could:

The future world I now envisage is one where the code of life is no longer written solely in RNA (ribonucleic acid) and DNA, but also in other codes, including those based on digital electronics and instructions that we have not yet invented. In this future period, the great Earth system that I call Gaia might then be run jointly by what we see as life and by a new life, the descendants of our inventions. ²

Which brings me to exams. 

Testing Half a Brain

Back to the original analogy. The super fast bike. The carbon golf clubs. Removed for competition? Madness. If we did that, no one could show what they were actually capable of. Because you’re not only testing the person. You’re testing the human/machine interface. 

Smartphones, laptops, access to the internet, digital communication, and now AI. Removed when we examine. Again, madness. Because they’re part of our students’ mind-loops. Without them, they only have access to part of their intellectual ability. Following Clark’s argument, the locus of intelligence does not only rest inside our skulls. It’s out there in the physical/digital world as well. We are therefore literally testing only half our students’ minds at best when we examine them in the traditional way. 

So why are we still doing it? Surely we’ve worked out by now that it makes no sense? Perhaps it’s because of this outdated notion of the mind resting solely inside the skull (what Clark refers to as the ‘BRAINBOUND’ model). As soon as we step away from that notion, towards what he calls the ‘EXTENDED’ model of mind, then we begin to understand that we are not in fact testing a student’s intellectual abilities when we examine them. We are in fact testing their ability to ‘play the game’.

Game of Exams

When I was in the classroom I always referred to exams as a game students needed to learn the rules of. In fact, I wrote a whole series of revision guides called the ‘Examiner’s Head’ series, where I (at times) humorously took students through how to think like an examiner - how to get inside the examiner’s head and understand the rules of the exam game. They’re still available on Amazon and continue to sell quite well. (I later changed the series name to Learning Engineers and sales fell as a result - I really must change them back…)

Those who do best in exams tend to be the ones who not only have the academic chops, but can also do what needs to be done to tick the required boxes to get the higher grades. Writing under time pressure requires a certain approach, one that privileges doing only what needs to be done and nothing more. Four mark question? Make four points as quickly as you can and move on. 

And because the exams privilege this approach, so does our teaching. Which is why, the closer you get to exams, the less creative you’re able to be in the classroom. Any push towards project-based learning is usually met with enthusiasm by primary and middle school teachers but dread by upper secondary teachers. They just cannot see how they can get through the syllabus without reverting to the standard delivery model. 

This is why I can’t see our educational model changing before how we test progress and ability changes. Which means there’ll be this growing mismatch, this gulf between how our students’ minds are developing, increasingly augmented as they are by greater forms of digital intelligence, and how we believe their minds should still be tested, divorced from augmentation, alone in a stuffy exam hall, surrounded by sniffing teens, only able to demonstrate a fraction of their potential. 

Surely, if we ever wish for them to win their mental Tour de France, or British Open, we need to give them their bike and clubs back?

1. Clark, Andy, Supersizing the Mind: Embodiment, Action, and Cognitive Extension, Oxford University Press, 2008

2. Lovelock, James, Novacene: The Coming Age of Hyperintelligence, Penguin 2020