A situation in which descending retrievability may not be the most efficient review sort order?

Anki Learning Effectively
1

Hi, I’m currently dealing with a huge backlog in a deck that I had to stop reviewing for several months. Initially, I chose to sort the cards by descending retrievability, the order which is usually recommended. I read that it’s recommended because data showed that it gives the best results in terms of average retention.

However, I’m not sure whether these data take into account that cards may be interconnected. If this is not the case, that could be an important limitation.

After starting the backlog, I decided to switch to sorting cards by the order in which they were added, and this approach looks to me more time-efficient. When using descending retrievability, the cards are more scattered, and so without much logical sequence or context. Although studying without context has its own advantages, in this case it seems an issue, because many of the cards I reviewed made little sense after I had completely forgotten the surrounding notions that originally supported their recall.

By reviewing cards in the order they were added, I encounter fewer of these problems. In my deck, earlier cards often provide the context needed to understand later ones, making the review process smoother. I don’t know how this impacts average retention, but this doesn’t affect my considerations.

The optimal sorting order likely depends on several factors. But in cases like mine, sorting by “order added” may offer important advantages that outweigh the disadvantages.

I would like to hear your thoughts on this. Or also, if you know about data/simulations/analysis that address this topic I would appreciate.

2

I agree with this. Anki’s algorithm is perfect, meaning the cards should be perfect too. These are cards that have no connections to other cards. But in reality, this is often not the case.
So if we add cards as the subject becomes more complex, then reviewing forgotten cards becomes more effective as the subject becomes more complex. It’s more likely that even if the early cards are forgotten, they’re not that difficult, and you’re less likely to forget them immediately after reviewing them, meaning you won’t be asked to study them again.

Let’s say you’re given a deck where all the cards are shuffled. Even worse, let’s take the deck and start studying from the back. :slight_smile: Since all the difficult cards are there, you’ll forget them more often, but even so, you can learn them and eventually get to the first, easiest ones. After studying them, you’ll understand why you forgot those difficult ones.

So you’ll still learn the cards, but at what cost?

So even if you shuffle the cards, the algorithm will figure out the difficulty of all the cards after many showings, but this could take up to a year (if you’re working on 10 cards a day, and you have over 3,000). Then, by sorting such a deck correctly and passing it on to someone else, you can see how much easier it is for them to learn. :slight_smile:
My opinion is that it will be easier for them, since everything is learned from the simple, that is, the understandable, to the complex, that is, the obscure.

1 Like
3

Thank you for developing further the reasoning I had started.

Sorting cards according to the order in which they were created seems particularly useful when one has to begin from scratch, whether because 100% of the material has been forgotten or because the topic is being studied for the first time.

This intuitively makes sense. Most people would probably agree that studying a subject for the first time in a completely random order is far from ideal. Descending retrievability is certainly preferable to a purely random order, since it still prioritizes easier cards, which is beneficial. However, in many cases it may still be more scattered than the original logical progression in which the cards were created.

In my specific situation, the backlog is so large that reviewing feels almost equivalent to studying the material anew — even for cards that currently have the highest retrievability. If that is the case, then the advantage of sorting by descending retrievability may be significantly reduced, since i am unlikely to remember the card in any case.

1 Like
4

The reason that sort order was the most “efficient” is all about retention – so if you aren’t concerned with that, there’d be no reason to follow that finding. You can study your backlog in whatever order works best for you.

5

Thank you for replying. What I truly care about — and I think this is true for many Anki users — is learning what I need to learn in the shortest time possible. Average retention is probably a good proxy for that. Another important parameter, however, might be the average time required to review cards.

In my case, which may also apply to other users, studying cards in a scattered order was costing me a lot of time because I often had to reconstruct the surrounding context to make sense of that card. I suspect that this fact is quite difficult to measure. This may also indirectly affect average retention: the longer it takes to review, the more days the backlog persists, which is of course not desirable.

6

That’s efficiency! But it means you do care about your retention outcomes, because that’s how you measure that.

Inherent in the rationale for why Descending R is the most efficient, is that you will be sacrificing some cards that already have lower R in order to rescue cards that still have higher R. That’s not a bargain that everyone is willing/able to make – and it sounds like you’re not.

That’s usually a sign that you have poorly made cards. Each card should be able to stand on its own without the surrounding cards.

7

Inherent in the rationale for why Descending R is the most efficient, is that you will be sacrificing some cards that already have lower R in order to rescue cards that still have higher R

That’s great, but I fear this assumes that the R of each card is independent from the rest of the deck. In reality, even if cards are designed to stand alone, knowledge itself is a network. The probability of recalling one fact often depends on remembering related facts stored in other cards.

To take a simple example: if you have a card asking for the end date of the First World War, you are likely to recall it more easily if you have just reviewed the beginning date. Of course, one could argue that the reverse is also true in this specific case. But more generally, when a professor or a textbook presents information in a certain order, it is usually because earlier notions provide the background knowledge needed to understand and remember later ones.

My hypothesis is that when much of that background knowledge has been forgotten, as in the case of a large backlog, restarting reviews in the original logical order might lead to higher average retention than descending R. This is because reviewing earlier cards may increase the retrievability of related cards that come later.

8

This isn’t just a hypothesis. It’s been a fact for thousands of years. We remember things in order, not backwards. Everyone learns the alphabet in order, but try reciting it just as quickly in reverse order? Every image, emotion, thought, generates another thought, emotion, and so on. That’s how neurons work. Moreover, often more than one signal is needed to trigger neurons; the switching potential must be increased, and several repetitions are needed, or better yet, signals must come from several directions. For people who are learning, this means learning not just the word, but everything together: seeing, hearing, more emotions, and so on. This doesn’t mean you should always do this; there is a method for concentrating on just one thing, but that only works when a person is more interested. At the initial stage, you need to convince the brain of the necessity of this knowledge, and the more frequent the signal, the more it is connected to other neurons (facts, images, sounds), the better it is remembered. Let this be my opinion, otherwise they will ask for facts again… but these opinions are formed after reading dozens, or even hundreds, of books about memory, where each author tries to understand what our memory is and how it works.

1 Like
9

One thing to keep in mind is that we use Anki to learn information—not merely to answer flashcards correctly. We want to be able to recall the information in a certain real-life, non-Anki, context. So, while a fixed ordering of flashcards may help us answer the flashcards correctly, it might not help us recall information in real life.

1 Like
10

It depends on the card types. With “cloze” cards, we see a sentence or even a large chunk of text in a specific order, meaning there’s context. For a single note, the order of the cards isn’t important, since we’re repeating one part of the entire text and visually repeating the rest.
You could even create a card with a video, which would illustrate the situation and prompt us to continue. But that wouldn’t allow for a complete sequence. No one’s stopping you from showing a video clip, and if you want, you can continue watching outside of the Anki, but then the algorithm won’t take into account the other parts of the video. I wonder, for “cloze” cards, does the algorithm take into account the words we’re forced to look at? Some of these could be from other cards.

11

Sorry for the late reply, and thank you for raising this point — it is very relevant. I agree that what matters is being able to recall information in real life, and as a general rule I do use random order when reviewing cards. However, what I am describing is a very specific case — call it an “emergency” — where the backlog is so large and the material so remote that the situation more closely resembles learning from scratch than reviewing previously acquired knowledge.

Let me illustrate why this matters using the First World War example, which is simple but captures my experience well — though in practice the situations I encountered were often more subtle. Suppose I originally learned the end date of the war through the following reasoning: “the war lasted about 4 years, so if it started in 1914, it ended in 1918.” This works fine as long as you remember that 1914 is the starting point.

Now consider what happens with a large backlog. If I use “order added” — assuming these two cards were created sequentially — I first relearn that the war began in 1914. Then, if the underlying reasoning comes back to me, recalling 1918 on the next card is straightforward. If the reasoning doesn’t resurface immediately, seeing 1918 revealed will likely trigger a eureka moment: “right, the war lasted four years.” If instead I use descending retrievability, the cards for 1914 and 1918 might be reviewed in two separate moments, even at different days. Under these conditions, it becomes much harder for me to reconstruct the original reasoning that made 1918 easy to recall. The risk is that for a certain period I will fail to recall that reasoning entirely, and will therefore attempt to re-memorize both dates inefficiently, either by constructing a different reasoning from scratch or by resorting to pure rote memorization with no reasoning at all.

In normal situations, when I use “random” order, I do so because I want to train my ability to flexibly retrieve from my own brain the underlying context of the card and to reason about that context in order to recall the card correctly. But in the case of a huge and remote backlog, it is too difficult to do that — you have to relearn from scratch the reasonings and contexts you were using. Shuffling cards is a second step.

The reason why using the ‘’order added’’ increases average retention is that foundational knowledge is encountered immediately before the more advanced knowledge that depends on it — which reflects a natural learning progression, at least when you are at the very early stages of learning.

12

On the one hand, I agree with you.
But still, Anki is not a tool for learning new things, but for reviewing what you’ve already learned. In fact, you could even consider it a knowledge test (and that’s what it is, since you’re graded and your memorization is assessed). Like a test, you’re randomly given a question and you have to answer it. You’re not surprised that the questions on tests are given in random order, are you?

In an ideal situation:

  • you make the cards yourself, and you already know half of them and haven’t forgotten them yet.
  • you introduce cards daily, a few at a time, about 10 at a time.
  • the order of the cards is natural, and therefore you memorize them in the correct order.

If you use a pre-made deck and then shuffle it, it’s similar to a student sorting through an entire book, then shuffling the pages and trying to study them.

I’ve long suggested that complex subjects should be divided into approximately one-month study periods, using subdecks. Approximately one month corresponds to a knowledge assessment at an educational institution. Perhaps a subject could be divided by topic, with each topic then serving as a subdeck.

If you have a pre-made deck, it has a pre-defined order of cards, so you should select all of them and hide them from study. Choose 10 for yourself each day and study new things when you’re rested and have plenty of energy. The cards can also contain links to original articles in textbooks; you should visit them all or watch videos. Think of each card as a plan of what you need to remember (the most important thing). Think of each card as something you can glance at, and it will remind you of your entire study history from other sources.

For review, choose a time when you’re running low on energy, or when you’re traveling and can’t watch anymore, only listening.

Now consider the situation where you’ve completed the deck, put it aside for a month due to vacation, and have already forgotten half of it. You don’t yet know what you’ve forgotten, just as Anki doesn’t yet know and gives you the wrong order to review. Again, you can’t remember anything because the information is being given essentially randomly.
Unfortunately, Anki won’t tell you what to do in this case, as its perfect algorithms simply can’t allow for such a thing. But even if it’s given randomly, you’ll eventually understand everything, so how can Anki know you’re the dumbest of the dumb (well, that doesn’t apply to you, anyway :).

What does a student’s tutor do? They know how each piece of knowledge is connected to the other, which is exactly what you want. If a student needs to study card “C” to master card “D,” then it’s logical that if they fail card “D,” they should be tested on card “C.”
Perhaps the relationship there is more complex, say: D->C1,C2,C3
And then each “C” might depend on “B2,B8,Bn”
A tutor has to check everything, but Anki won’t do that.

In Anki, I break a subject into subdecks, say, 10 topics with 10,000 cards each. I intentionally wrote such a large number of cards to make the difficulty clear. Anki assumes your memory is perfect, and you shouldn’t have forgotten almost anything from those 100,000 cards. But how can you tell if this is really true? You need to take each subdeck and randomly select about 50 cards. If you’ve forgotten more than 20%, you should re-read the subdeck in the order you originally assigned and repeat the test. Sometimes, a simple quick scan is enough to The brain, so they remember.

So, there’s a real problem with Anki. The data isn’t organized in a tree, since it’s not a study, but essentially a test. It would be nice to have links linked to this card at a lower level (let’s say this card uses knowledge from other cards and is dependent on them), and the user could decide for themselves that if they don’t know a certain card, they can open a related card and review it again. Of course, Anki can theoretically figure out which card you should know less well, but I’d recommend giving the user the choice so they can decide what exactly they forgot or didn’t quite understand and want to review again.

All that remains is to create such an algorithm. This can be done as an add-on by storing a field in the card with the cards this card depends on, then creating a filtered deck and reviewing these cards, either with or without saving viewing statistics.

But many people are too lazy to add a field to the card with information about which cards determine good memorization and understanding of this card. It’s good if tags are added and exist. An add-on where you can go through tags if needed and review other cards.
So a lazy tutor might say that if you don’t know “Topic 5,” then go and study “Topic 4” first.

1 Like