National 5 Computing Science

1. Predict before you run

The research. PRIMM stands for Predict, Run, Investigate, Modify, Make. Pupils first predict what a piece of code will do, then run it, then work with it. In a study across 13 schools with 493 pupils aged 11 to 14, classes taught with PRIMM did better on a programming test than classes taught as usual, and teachers said it helped every pupil in a mixed-ability class make progress.

What we do. The program-stepping simulations stop before any line that displays something and ask what it will show. The Python lessons ask for a prediction the first time an example is run, then compare it with what really happens.

Sentance, S., Waite, J. and Kallia, M. (2019). Teaching computer programming with PRIMM: a sociocultural perspective. Computer Science Education, 29(2-3), 136-176. doi.org/10.1080/08993408.2019.1608781

2. Read and trace code before you write it

The research. A study of over 500 students in seven countries found that many beginners could not reliably say what a short piece of code would do. The authors argued that this weak grasp of reading code holds back problem solving. Reading and tracing come first.

What we do. Simulations step through programs one line at a time, showing every variable as it changes. Many quiz questions ask what a program will display or store, which is also how the exam tests it.

Lister, R. et al. (2004). A multi-national study of reading and tracing skills in novice programmers. ACM SIGCSE Bulletin, 36(4), 119-150. Open-access copy

3. Put code in order before writing it from scratch

The research. In a Parsons problem the learner puts mixed-up lines of code into the right order. Studies found that students solved Parsons problems much faster than fixing or writing the same code, with the same learning gains.

What we do. Quizzes include "put the lines in order" and "fix the code" questions for the longer algorithms, such as input validation and running totals. The coding lessons then ask pupils to write the code themselves.

Ericson, B., Margulieux, L. and Rick, J. (2017). Solving Parsons problems versus fixing and writing code. Proceedings of the 17th Koli Calling International Conference on Computing Education Research, 20-29. Find it

4. Interactive, not just animated

The research. A review of 24 experiments on algorithm animations found that what learners do with a visualisation matters more than what it shows. A working group of researchers went on to argue that visualisations add little unless they get learners actively doing something: predicting, answering, changing the input.

What we do. Every simulation makes you act: sort, build, step, change the data, answer. Each ends with a Check yourself task, and it only counts as complete when you get it right.

Hundhausen, C., Douglas, S. and Stasko, J. (2002). A meta-study of algorithm visualization effectiveness. Journal of Visual Languages and Computing, 13(3), 259-290. doi.org/10.1006/jvlc.2002.0237
Naps, T. et al. (2003). Exploring the role of visualization and engagement in computer science education. ACM SIGCSE Bulletin, 35(2), 131-152. Find it

5. A clear picture of what the computer does

The research. Beginners struggle when they have no clear model of what the computer is doing as a program runs. Researchers call that model a "notional machine" and recommend teaching it directly.

What we do. Bit the seagull, who understands only "chip" and "no chip", is a running picture of a computer that only knows 1 and 0. The simulations show the machine at work: variables in boxes, the processor fetching and doing, a database joining its tables.

Sorva, J. (2013). Notional machines and introductory programming education. ACM Transactions on Computing Education, 13(2), article 8. Find it

6. Spread your practice out

The research. A review of 317 experiments found that practice spread out over time is remembered far better than the same practice crammed together, and that the best gap grows with how long you need to remember it.

What we do. Your dashboard has a Keep it fresh list. A topic you have passed comes back a week later, then after two weeks, then every four weeks. Each retake has new questions, and your best score still counts.

Cepeda, N., Pashler, H., Vul, E., Wixted, J. and Rohrer, D. (2006). Distributed practice in verbal recall tasks: a review and quantitative synthesis. Psychological Bulletin, 132(3), 354-380. doi.org/10.1037/0033-2909.132.3.354

7. Test yourself, don't just re-read

The research. Taking a test is not only a way of measuring learning: recalling an answer makes the memory stronger. Students who practised by testing themselves remembered more a week later than students who spent the same time re-reading.

What we do. Practice quizzes for every topic, with questions picked at random so they are different each time, and mixed review quizzes for each unit.

Roediger, H. and Karpicke, J. (2006). Test-enhanced learning: taking memory tests improves long-term retention. Psychological Science, 17(3), 249-255. Find it

8. Learn most from the mistakes you were sure about

The research. When people get an answer wrong that they were confident about, and are then told the right answer, they are more likely to remember the correction than for answers they were unsure of. This is called the hypercorrection effect.

What we do. Under every quiz question you can say how sure you are. It never changes your marks. When you were certain but wrong, the feedback says so, and your dashboard lists those topics first, because they are where your understanding needs fixing.

Butterfield, B. and Metcalfe, J. (2001). Errors committed with high confidence are hypercorrected. Journal of Experimental Psychology: Learning, Memory, and Cognition, 27(6), 1491-1494. doi.org/10.1037/0278-7393.27.6.1491

9. Feedback that explains, and wrong answers that teach

The research. Getting things wrong is useful when you find out why. The confidence research above depends on that: the correction is what gets remembered.

What we do. Every wrong option in every question has its own feedback saying why it is wrong. The wrong options are built from the mistakes pupils really make, and the feedback says what would and would not gain the mark in the exam.

10. Badges, with care

The research. A review of the evidence on gamification found that points and badges can help, but the effect depends heavily on the setting and the people using them.

What we do. Badges are earned only by finishing things that matter: getting all the coding exercises working, passing every topic in a unit, passing the review quizzes. There are no points for clicking around.

Hamari, J., Koivisto, J. and Sarsa, H. (2014). Does gamification work? A literature review of empirical studies on gamification. Proceedings of the 47th Hawaii International Conference on System Sciences, 3025-3034. doi.org/10.1109/HICSS.2014.377

Where our approach comes from: Scottish computing education

The research. The reference language used for code in National 5 exams was developed by Quintin Cutts (University of Glasgow), Greg Michaelson (Heriot-Watt University) and Richard Connor (University of Strathclyde), after SQA asked for one in 2010. It gives exam questions code that is exact, so that a pupil's job is to read it and explain what it does. Cutts and colleagues explained why it matters: they argued that a loose mix of code and English is a poor basis for questions about understanding programs. Quintin Cutts also led PLAN C, a Scottish Government-funded programme in which computing teachers across Scotland learned together through local groups led by teachers. Its evaluation reported that teachers grew in confidence and changed how they thought about learning. One of its approaches is tracing: marking up code to show what the computer is doing as it runs.

What we owe them. This site inherits that thinking. Code is shown in the reference language for reading and tracing, while pupils write their own code in Python. The simulations walk through programs one line at a time, and the code stepper in our free teacher tools also shows the working for each line and fills in a trace table. We are not connected with, or endorsed by, these researchers or the PLAN C project. We are simply grateful for the foundations they laid in Scotland.

What else we took from it. The same group's taxonomy of question types - what level a question is asked at (the scenario in plain words, the technical vocabulary, or the code) and what level the answer is at - is used to check that our question bank asks for the full range of moves, not just "read this code". Their course was taught with Peer Instruction, where a class votes, argues it out in pairs and votes again: that routine is one of the options in our free live quiz for teachers.

Cutts, Q., Esper, S., Fecho, M., Foster, S. R. and Simon, B. (2012). The abstraction transition taxonomy: developing desired learning outcomes through the lens of situated cognition. Proceedings of the Ninth Annual International Conference on International Computing Education Research, 63-70. doi.org/10.1145/2361276.2361290
Cutts, Q., Connor, R., Michaelson, G. and Donaldson, P. (2014). Code or (not code): separating formal and natural language in CS education. Proceedings of the 9th Workshop in Primary and Secondary Computing Education, 20-28. doi.org/10.1145/2670757.2670780
Cutts, Q., Robertson, J., Donaldson, P. and O'Donnell, L. (2017). An evaluation of a professional learning network for computer science teachers. Computer Science Education, 27(1), 30-53. doi.org/10.1080/08993408.2017.1315958