Why you’re probably wrong

If you’re answering a multiple-choice question and you have second doubts about your answer, you should always switch. In fact, there is 70 years of research backing up this strategy: the majority of answer changes are from incorrect to correct, and most people who change their answers usually improve their test scores – regardless of test content or environment. Like the case of the Monty Hall problem, you should always switch your answer if given the chance (although in the case of the Monty Hall problem, one of the answers you didn’t choose is revealed to be incorrect before you’re allowed to switch). The only reason that we don’t think this is the case, despite it being demonstrably true, is because it’s counterintuitive.

In the case of MCQs, when we make a change our answer and then are wrong, it seems like an easily-avoidable error, causing frustration and making this type of error more available in memory, making its instances seem more frequent than times when changing your answer results in being correct – known as the first instinct fallacy. In other words, experience indicates that answer-changing is a poor strategy, but experience is misleading. This happens all the time – for example, when choosing a different checkout at a supermarket, whenever your original queue speeds up relative to your new queue, you feel frustrated at your decision and are therefore more likely to remember not to change queues in future, as this is more likely to slow you down.

Escalating commitment (also known as commitment bias) holds that when you choose a course of action, for example an answer to a multiple-choice question, prior investment in a decision (including thinking about that answer) makes us far more likely to increase investment in that decision. James Surowiecki argues that groups are far better at making decisions than individuals, as the aggregation of that group’s individual answers tend to be quite accurate. For this reason, on Who Wants to be a Millionaire,ask the audience’ produces the correct answer 91% of the time, whereas it’s 65% for ‘phone a friend’.  (N.B. this doesn’t always work.)

As an aside, when combined with a risk-reward system (such as in The Weakest Link), loss aversion is a massive problem: whenever there is a potential of ending up with less as the result of an action, even when taking the risk means potentially ending up with much, much more, our fear of having nothing overrides the logical choice (go for broke).

What this all really illustrates is that by simply telling students that they should switch their answers if they’re in doubt, we’re also saying that, just because they had one thought first and then changed their mind, doesn’t mean that those latter thoughts are more likely to be wrong – in fact it’s the opposite; in other words, you shouldn’t be afraid of changing lanes in traffic, because if it looks faster, it probably will be – despite what experience tells you.

So, to give educators a more accurate picture of what students know and don’t know, as well as to maximise learning (because realising when you should change your answer, by virtue of being aware of what you know and don’t know, is an example of metacognition, which aids learning*), it might be worth telling students, if they’re having second thoughts about an answer, to switch – or to make them really frustrated while they’re trying to revise.

*For more on the interaction between metacognition and the first instinct fallacy, see Higham & Gerrard (2005).

EDIT: fixed link.

2 Comments

Filed under Uncategorized

Tricking children into learning: …why?

In my primary school in the mid-‘90s, this was the closest we ever came to interactive learning: Hooray for Henrietta, and Henrietta’s Book of Spells.

That’s right. For about 10 minutes a week over a couple of months, each of us was allowed access to either the BBC Micro in the corner, or the stunning beige tower running Windows 3.1 – and this was what our experiences with I.T. amounted to. No internet, no programming – although I was allowed to type once – just this abomination.

The adverts say it all – “give ‘em something to laugh about and they’ll forget it’s homework,” “…could do with a bit of torturing, erm, learning,” and “HOORAY FOR HENRIETTA achieves a perfect balance of learning and having fun” – balance? Are learning and fun mutually exclusive?

That children find learning inherently boring is a strange assumption for the producers of an educational tool to make. There’s this widespread belief that we have to trick children into learning, by disguising it as entertainment – however thinly-veiled.

B. F. Skinner was one of the first psychologists to suggest ways that technology could aid education: Skinner thought that the classroom was flawed because of children’s varied learning rates and delayed reinforcement/feedback, because of the lack of individual attention. As early as 1954, Skinner carried out successful studies based on his Programmed Learning Theory involving his “Teaching Machine”, which was designed to provide personalised teaching to match the rate of the student. It being 1954, his teaching machine consisted of a program that combined teaching and testing in the simplest way, where you either fill in the blanks or complete a workbook – and on giving a correct answer, reinforcement would be given and it would move on to the next question.

As you can see, Hooray for Henrietta basically follows the same lines – there’s a maths question at the bottom, you fill in the answer, and you get rewarded by moving forward closer to saving that thing in the corner. But that’s all it does – you answer correctly and get to move forward, and if you win they’ll dance – considering that this was nearly 40 years on from Skinner’s teaching machine, you would have thought that things would have moved on a bit.

It turns out that pretty much all educational games just consist of the most unbelievably basic, get-it-right-and-maybe-they’ll-do-a-funny-dance, strategy. The graphics have improved, but you won’t find many games explicitly attempting to teach maths that are any more involving than this. We have the technology to make interactive, involving games that actually teach more than times tables – and yet the furthest anybody has gone with it is things like Brain Academy, which has been demonstrated not to improve general cognitive function, and which still relies on asking you things like addition/subtraction questions. But even the proficiency gained at the other various tasks don’t transfer – your only hope is that you get better at playing that game. So, how can we use gaming, and the motivation it gives people, to teach?

A number of blogs last week explored the idea of using technology in education, and I’ve duly nicked their references: scofedhannah talked about using perceptual learning, which is basically learning through repeated pattern recognition – this makes sense, as all the brain really does is recognise patterns in its environment – which makes it really easy to spot irregularities. The potential applications for any rule-based subject (e.g. maths) are self-evident. Massey & Kellman (2010) tested a program that asked students in the equivalent of years 4-7 different kinds of fraction problems, that emphasised transfer and generalisation, so they could use what they learned outside of this program, and found that these students consistently performed better than students a year or more above them. What’s more, they retained this advantage 5 months on. Psub1a mentioned a number of studies that point to the usefulness of using story-telling as a way of making things more memorable and entertaining, provided they correspond with the context.

Guillén-Nieto and Aleson-Carbonell (2012) found that a game designed to teach intercultural business communication between Spaniards and Brits where English is the common language, had a huge learning effect on intercultural communicative competence, as well as effects on intercultural awareness and knowledge. They concluded that, while the immersion and interactive learning environment of the game contributed to its effectiveness, it was so effective because it found a good balance between instructional content, game dimensions, game cycle, debriefing, perceived educational value, and transfer of learnt skills and intrinsic motivation.

Amory et al. (1999) found that students prefer adventure and strategy games to shoot-em-up and simulation games, and luckily things like logic, memory, and problem-solving are integral to adventure games, as well as being integral to the learning process. They then came up with a model that links these game elements to learning – so we could feasibly develop an educational game based around this model, that takes all of the parts of a game that students find interesting and stimulating, and which also happen to help learning, and give students the intrinsic motivation to solve those kinds of problems with the virtual environment the game provides.

So it looks like the best way to make use of the potential of games is to emphasise feature such as logic, memory, and problem-solving; incorporate a storyline (not just ‘save the princess’); make sure that the skills obtained are transferable; make the player feel as if they’re learning; and for God’s sake, use a bit of imagination.

2 Comments

Filed under Uncategorized

Demographics do not define destiny

In one of his TED talks, Ken Robinson briefly mentioned the Knowledge is Power Program (KIPP), a system of free ‘college-preparatory’ public schools that serves students from low-income communities, who are typically at risk of under-achieving at school (see Ch. 3): their stated mission is to ensure that every student graduates from university. Their model serves over 32,000 students from communities where 10% of students graduate from university. 85% of KIPP students graduate.

An analysis by Angrist et al. (2010) found that typically under-served students significantly benefited from KIPP, arguing that this is due to their “No Excuses” approach (Carter, 2000; Thernstrom & Thernstrom, 2003). But is this really the secret to their success? According to the teachers, the success of KIPP in giving students who, as far as anyone can tell, would otherwise under-perform, the opportunities and drive towards learning and self-improvement, lies within high expectations placed upon its students and consistent ‘behavior [sic] management’. Whilst Ross et al. mentioned that we can only speculate about which elements of KIPP were effective, their conclusion was that KIPP schools appear to achieve such high standards for a number of reasons, including their emphasis on expectations of high standards from students (Wentzel, 1997, suggests that perceived caring predicts motivational outcomes, which will inevitably affect any student’s attainment), consistent behaviour management, teachers’ commitment to the program, extended learning hours (which encouraged teachers to vary their strategies to avoid boring their students)… students’ complaints were mostly directed towards disciplinary measures.

Ross et al. (2007) found that, in a survey of students, staff and parents at one KIPP school, perceptions of KIPP climate attributes such as high expectations (“preparing students to be competitive for the schools that they will attend after 8th grade”), involvement, and collaboration were significantly higher than national norms – notably, 100% of respondents agreed that “teachers have high expectations for all students”. These results could be a result of selection bias – a criticism of KIPP is that their admission process not only requires parents to commit to a certain amount of involvement, thereby excluding children from dysfunctional families, but also self-screens for motivated and compliant students (cf. Bulkley & Fisler, 2002; RAND, 2001); alarmingly high attrition rates for some low-performing students, as well as teacher turnover rates, are also evident in some KIPP schools. (KIPP schools are run independently of one another, however, and this study only looked at San Francisco Bay Area KIPP schools.) It is also clear, however, that for those students who did not leave, the benefits are substantial.

We have evidence that the negative effect of poverty on chances of leaving school with qualifications applies here in the UK, too (Ermisch & Francesconi, 2001), so the benefits of such a system in terms of low-income children’s qualifications are apparent – but more importantly, is academic achievement the best score of success? Clearly academic achievement is preferable to underachievement, but standardised test scores aren’t, and shouldn’t be, the arbiter of a successful school. Maybe a measurement such as PISA (see p. 119) would be a more appropriate measure of a school’s success.

Overall, the KIPP program has been shown to produce significant benefits in students who would otherwise suffer in school. Considering the recent news about underachieving students in the UK, and the evidence that such a system would improve educational mobility in poor households, which we could really do with (p. 133), we might benefit from some of the lessons learnt from KIPP.

1 Comment

Filed under Uncategorized

Hello world!

Hi everyone, and welcome to my Science of Education blog for 2012.

This isn’t my first blog – my other blog, matsproject.wordpress.com,  follows the development of my research project on metacognition. Some of the posts have had to be password-protected, and not all of it will make sense unless you’re in my project group, but anyone is welcome to whatever information is available there.

Enjoy.

Steve

Leave a comment

Filed under Uncategorized