Ideal retention rate for Oral Exams

I have a very massive oral exam in the next 2 months and it is not allowed to commit many mistakes. I had a retention rate of 95% set for my written exams. It served me well, but I still believe I committed way too many mistakes for it to be acceptable for my oral exams.

With that in mind, what should I set my desired retention rate to be? And what is the problem with setting a 99% retention rate?

The effect of increasing retention rate is that the number of reviews required increases exponentially:

You can do it, but you will end up with a much larger workload.

2 Likes

So is 97% a good compensation?

You should also keep in mind that the target retention rate only triggers when each card is beeing scheduled for review. (e. g. 85%)

However the average predicted retrievability over all cards in the deck is greather than that (if there are no overdue cards). (e. g. 93%)

(the numbers are guessed / made-up)

You can experiment with that yourself. In the deck options, you can set the desired retention, and you will get a preview of how much a new interval after grading will be affected:

image

As an example (based on my Anki), if the good button would result in a 100 day interval at retention of 0.9, it results in a new interval of 46 days at 0.95.

From there the number of days decreases very quickly:

0.96 - 36 days

0.97 - 27 days

0.98 - 18 days

0.99 - 9 days

Your values may be different depending on your FSRS params.

You could always try it, and if it becomes unmanageable you can reduce retention and reschedule your cards (using FSRS helper add-on).

You can also consider other options like,

  • Reducing maximum interval so you will get an additional review before your exam.

  • Study with lower retention up to the last eg. week before the exam, then using a filtered deck to review all cards before the exam (how viable this is will depend on the volume of cards you have).

I would also suggest that Anki shouldn’t be the only way you study. If you have access to example questions for the exam (eg. older exams) review those as well, as Anki only tests recall of atomic facts, and exams are likely to test how you combine and apply information in new settings (at least my exams usually took that form).

1 Like

Well the volume is great, like 15000+ cards great. So overdue cards are bound to pile up as I only restrict my self to 2 houts of review time per day (I manage about 500 cards on average within that time). This wasnt a great problem for written exams, as FSRS kept those overdue cards in check and filters out the most overdue from them.

But now for oral exams where I need to bring my A-game (it is an Anatomy exam-I am studying Medicine), I must change something.

-I have once lowered my maximum interval to a very low 2 day interval. This gave FSRS a very skewed perception of my cards. And I basically forgot them all over again.

-This was a strategy I used for my written exams. But I am afraid the stakes are high something like that. You see, I already failed this Anatomy exam twice. I only have 2 chances left or I would be exmatriculated.

-You are also right about Anki not being the sole way to study. This has royally effed me up. I did not use what I learn to formulate a conncise answer orally. I learned that the very hard way.

Make sure that you understand what the numbers mean.

Desired [/Target/Requested] retention (and True retention has the same definition):

the proportion of cards recalled successfully when they are due.

Average Predicted retention:

the average probability of recalling a card today. In most cases, it is higher than requested retention because requested retention refers to retention at the time of a review, whereas average retention is calculated based on all cards, including undue cards. Not all cards are due, that’s why these two values are different.

For an exam on a date certain, average predicted is going to be a much more relevant number for you. If you Shift-click stats, you should see the FSRS version of the stats page that will show you True and Average Predicted figures.

Disregarding for a moment that anecdotal examples are of limited usefulness :sweat_smile: – in my collection:

  • Requested = 92%
  • True = 92.1%
  • Average Predicted = 96%

Consider this advice from the tutorial

However, setting the desired retention above 0.97 is still not advised for two main reasons:

  • Such a high desired retention will significantly increase your workload (cards per day). The repetitions will be so frequent that you will dread doing your reviews before you even discover the power of spaced repetition.
  • With such high retention, each review will contribute minimally to your overall learning. This essentially transforms the spaced repetition system into a massed repetition system, thereby undermining the advantages of the spacing effect.
2 Likes

So I checked my biochemistry deck that I have been reviewing over the last week (about 13000 cards, each card represent ) and it turns out I have a 99% average predicted retention. So I have probably went ham on it without noticing.

But why do I have this nagging feeling that these numbers dont actually represent the amount I actually know, especially when I get down to reviewing and be met immediately with cards that I get wrong

And how can I get my average predicted retention way up quickly without unnecessarily increasing the work load for me?

  1. Averages are average – predictions are predicted. Neither of them is a card-by-card guarantee, and 1% of 13,000 is still 130 cards.
  2. Aren’t most of your cards significantly overdue? Have all of them even been studied with FSRS?

how can I get my average predicted retention way up quickly without unnecessarily increasing the work load for me?

You can’t. First, I suspect it doesn’t go above 99%. Second, it’s already higher than it needs to be, and you could probably turn your Desired retention down from 97% and still maintain the same Average Predicted. Lastly, you can’t get something for nothing – increasing your retention and having a higher workload go together.

Aren’t most of your cards significantly overdue? Have all of them even been studied with FSRS?

Yes, but not ALL of them. Because I had other subjects, each with a huge amount of cards, I needed to keep all of them somehow in check without spending way too much time on revieweing for all of them. I let the FSRS do its magic there and it managed to keep by Biochemistry deck at a 80% retention rate even with thousand of cards being overdue.

You can’t. First, I suspect it doesn’t go above 99%.

This was me speaking about my biochemistry deck and not the anatomy deck which I have an incumbent oral exam on. I am basically starting from scratch there, and because I only have two chances remaining, I really do not have much leeway to f*** around :smiling_face_with_tear:. I already found out.

increasing your retention and having a higher workload go together.

Yeah, nothing comes for free, eh? I only just need this sweet spot where I have just good enough on me without doing way too much than necessary. If there is a sweet spot above 95% desired retention, please let me know. If there isnt, well bummer.

you could probably turn your Desired retention down from 97% and still maintain the same Average Predicted.

How does Desired retention actually correlate with Average predicted retention. This is new to me, and I feel like one of them does not represent actually reality but rather for mathematical purposes.

As per the definitons above, Desired retention is your goal for share of cards you remember when they are due. [True retention is what share of cards you actually have remembered when they were due (or overdue, I suppose).] Average Predicted retention is the predicted probability of you remembering any given card today. The definition above also explains how they correlate.

So the only one that represents reality after-the-fact is True retention. As noted on the FSRS Stats page, “The average predicted retention is calculated using FSRS formulas and depends on your parameters.”

Isn’t there a significant chance that 95% Desired retention, resulting in something close to 95% True retention, will give you (has given you) 99% Average Predicted retention? I’m not sure why you think you need to stomp on that ceiling. You can check your own stats and see what is currently happening.

Average Predicted retention is the predicted probability of you remembering any given card today .

I sitll dont see how True Retention affects Average Predicted Probability. I will probably just assume it is some weird mathermatical formulae in the background doing all the work.

Isn’t there a significant chance that 95% Desired retention, resulting in something close to 95% True retention, will give you (has given you) 99% Average Predicted retention?

Well it did work for my other subjects after I crunched down on my overdue reviews. I just need to figure out how to make Anki prioritise these cards automatically over others, such that any significant decrease in my ability to recall these cards would be diminished quickly.

Mind you, these cards are not new. These are cards that I have learned from a year ago for my previous anatomy exam which I failed twice. The cards are hard to retain. And not being able to recall minute details is fatal for my chances to pass the exam.

Back then, I did not know about FSRS or it propably didnt even come out. Now I am starting from scratch. One thing I am certain about is that using the same settings for my anatomy oral exam deck as the settings for my biochemistry written exams deck would not work for me, as even after going through all the cards, I still ended up with a worrying amount of failed cards.

Maybe I am just expecting too much from myself and that my expectations are furthest from reality and maybe I should just get down to eliminate all overdue cards prioritising this deck above all for my dire situation. I just really hope that I dont fail the exam a third time, lest I will have only my fourth chance left, after which my studies will end abruptly… :cry:

Also, is there by chance any other way to increase my average predicted retention without raising my true retention in the settings?

Another question I have is, if true retention is just a measured value set by the user, then theoretically all desired-true retention rates should be able to achieve an approx. 99% average predicted retention rate over some time. The question is how much time at each given retention rate, am I right?


I made this. Note that since FSRS isn’t perfectly accurate, it may under- or overestimate average predicted retention. But your numbers match this graph almost perfectly: 92% desired retention and 96% average retention. Also, note that while it looks linear at first glance, it’s not quite linear.
Btw, I haven’t posted this graph anywhere, maybe it should be added to the guide or somewhere, but I’m not sure where.

Nope.

True Retention is measured from your review history. Desired retention is a setting. Average predicted retention is an algorithmic prediction. The first two should (ideally) match, the third one is greater than the first two because it fundamentally measures a different thing. I know it’s confusing, but me and LMSherlock couldn’t think of a way to “un-confuse” people.

4 Likes

Woah! This is exactly the graph that I needed!

Also, note that while it looks linear at first glance, it’s not quite linear.

Is there a methodology, like a formula or smth to convert desired retention into average predicted retention if it is not linear.

So according to this graph, not any desired retention will lead to a 99% average retention rate after all. The difference between the two keeps on decreasing as the desired retention increases.

Huh, so is 95% the absolute most once could increase to as efficiently as possible without increasing the workload exponentially? And that it is not worth increasing after all?

There should be a closed-form expression, but I don’t know it, so I just do cumbersome stuff with tens of thousands of cells in Excel.

Are you referring to this?


Because this graph is different for every user, what you see is just an “average” graph.

I am aware of the presence of this graph and as Dakika mentioned above, more retention means more workload. What I’d like to know is if there is any graph that shows a workload to gain in benefit ratio. Kind of like a risk to reward.

Kind of like a sweet spot of sorts.

I am all for doing more work. Just need this guarantee that it yields a great benefit and that I am not just any time.

You can use the “Compute optimal retention (experimental)” feature to find which value corresponds to the highest speed of knowledge acquisition. But it only goes as high as 95%.

Is there any chance that this will change soon?

?Also, if there is no sweet spot and let’s just say 95% is that sweet spot, how would you go about my situation if you were in my place? Like how could I get the remaining 5% into my brain, since it is very critical for me not to mess up this upcoming oral exam? My forgetful brain is giving me hell.