The Anki algorithm needs more research and development

Please developer, you are the one who is able to do this.

If you dig into it the picture that emerges is deeply unsatisfying. Anki uses the Supermemo 2 (SM-2) algorithm, released in 1987, with some minor modifications. Piotr Woźniak developed the early Supermemo algorithms as a student during the 1980s (while trying to learn mainly English) and describes the procedure in his 1990 master’s thesis “Optimization of learning”. As I understand it, SM-2 was based solely on Woźniak’s own learning, i.e. on a range of informal experiments of sample size n = 1.

His 1990 master’s thesis has to date received 45 citations on Google Scholar and the most influential follow-up by him I could find, the 1994 article “Optimization of repetition spacing in the practice of learning”, has 97 citations. While a range of studies replicate the basic finding that spaced repetition helps retaining knowledge, I cannot find any that attempt to independently replicate, validate, or test the effectiveness of the specific choices made in the Supermemo algorithm and SM-2 in particular. Any links to studies that do this would be appreciated.

Woźniak et al. have of course made such tests and improvements themselves: The latest iteration of the Supermemo algorithm is called SM-18. Woźniak (1990), for example, writes already about SM-5 that “the acquisition rate was at least twice as high as that indicated by combined application of the SM-2 and SM-4 algorithms” (still n = 1). Such potential improvements have not been integrated into Anki, much because descriptions of later versions of the algorithm are not openly available.

Now, I have long intended to write an Anki add-on that enables us to experiment with the algorithm more systematically. Sadly, I will probably never have time for this, so I hope someone else takes this up. Basically, it would randomly vary some of the algorithms parameters for users, collect the results (retention rates along with some metadata, e.g. the sting length of notes might already be useful enough), and make them available for statistical analysis. This could immediately be used to suggest better defaults for the current SM-2 algorithm, and in the longer term one could use it to try out more substantial modifications of the algorithm. As others have suggested I also believe that there are large gains to be made from making the algorithm even more adaptive and personalized!

TL;DR: To sum up, admittedly somewhat unfavorably, Anki is stuck with an algorithm developed in the 1980s by some student trying to learn English who did some informal, incompletely documented, experiments on himself (n = 1). This research has largely been ignored by the scientific community and even its originator had already long abandoned the algorithm once it was integrated into Anki. We can surely do better!

Source: Reddit

5 Likes

Need to mention that then to various other apps such as the new Neuracache that also uses SM2…

(Please developer, you are the one who is able to do this!)

Inspired by the post Why isn’t the spaced-repetition algorithm more researched? I want to share some background on the Anki algorithm with you and my suggestions for how we can improve it. Any input is highly appreciated. The spaced-repetititon algorithm is the core that makes or breaks Anki and it needs to receive much more research and development.

If you dig into it the picture that emerges is deeply unsatisfying. Anki uses the Supermemo 2 (SM-2) algorithm, released in 1987, with some minor modifications. Piotr Woźniak developed the early Supermemo algorithms as a student during the 1980s (while trying to learn mainly English) and describes the procedure in his 1990 master’s thesis “Optimization of learning”. As I understand it, SM-2 was based solely on Woźniak’s own learning, i.e. on a range of informal experiments of sample size n = 1.

His 1990 master’s thesis has to date received 45 citations on Google Scholar and the most influential follow-up by him I could find, the 1994 article “Optimization of repetition spacing in the practice of learning”, has 97 citations. While a range of studies replicate the basic finding that spaced repetition helps retaining knowledge, I cannot find any that attempt to independently replicate, validate, or test the effectiveness of the specific choices made in the Supermemo algorithm and SM-2 in particular. Any links to studies that do this would be appreciated.

Woźniak et al. have of course made such tests and improvements themselves: The latest iteration of the Supermemo algorithm is called SM-18. Woźniak (1990), for example, writes already about SM-5 that “the acquisition rate was at least twice as high as that indicated by combined application of the SM-2 and SM-4 algorithms” (still n = 1). Such potential improvements have not been integrated into Anki, much because descriptions of later versions of the algorithm are not openly available.

Now, I have long intended to write an Anki add-on that enables us to experiment with the algorithm more systematically. Sadly, I will probably never have time for this, so I hope someone else takes this up. Basically, it would randomly vary some of the algorithms parameters for users, collect the results (retention rates along with some metadata, e.g. the sting length of notes might already be useful enough), and make them available for statistical analysis. This could immediately be used to suggest better defaults for the current SM-2 algorithm, and in the longer term one could use it to try out more substantial modifications of the algorithm. As others have suggested I also believe that there are large gains to be made from making the algorithm even more adaptive and personalized!

TL;DR: To sum up, admittedly somewhat unfavorably, Anki is stuck with an algorithm developed in the 1980s by some student trying to learn English who did some informal, incompletely documented, experiments on himself (n = 1). This research has largely been ignored by the scientific community and even its originator had already long abandoned the algorithm once it was integrated into Anki. We can surely do better!

Source: Reddit - The Anki algorithm needs more research and development