I have the same issue with FSRS. I am trying to reach retention 90 %. I gradually set the desired retention from 90 % to 93 % and recalculated the intervals every time so the intervals of all cards should be correct. My workload is higher and higher but my retention of mature cards is lower and lower.
I didn’t notice your comment because it was under someone else’s post about their situation. I split you off to your own thread.
To take a look at your situation, let’s start with–
- Screenshots of all of your Stats page. I don’t know how long you’ve been using FSRS or how long you were using Anki before enabling FSRS, but how about you show the graphs for 12 months?
- Your parameters (as text please, not a screenshot).
- Run Evaluate on your parameters. What’s your RMSE and based on how many reviews?
- How long ago did you last Optimize your parameters?
Thank you Danika for your answer.
-
I use Anki almost daily since 2016. I use FSRS since first AnkiDroid beta with FSRS.
-
0.4291, 1.8639, 3.3095, 13.6797, 7.8687, 0.2630, 2.3619, 0.0356, 1.5476, 0.1597, 1.2115, 1.8688, 0.0643, 0.2903, 1.5532, 0.5881, 2.9959, 0.6452, 0.7636
-
log loss: 0.2584, RMSE 2.66 %, 218,866 reviews.
-
I optimised the parameters nine days ago.
Maybe I realised the issue. I use the hard button quite often (but not as again). With SM-2, I have set the hard button to 1.1times of the previous interval. FSRS provides longer intervals for the hard button.
Is it possible to manually adjust the FSRS parameters so the hard interval will be as short as possible?
It is not. It sounds like you were trying to adjust your your Hard button down a bit so you could use it as a “soft”- Again. Since FSRS handles the new interval after a lapse so much better, it’s probably a good idea to just use Again for that now.
Your parameters and RMSE look fine.
The first things I notice in your stats are that you have a backlog, and you’re only studying 85% of the days. How big is your backlog right now?
What I can’t explain is that your Avg. Retrievability is 97% – which is about where you would want it for a 93% DR. And FSRS thinks it’s doing a good job predicting your memory, even though your actual retention is much lower. I could see how that would make sense with a lower Avg. R, and a big backlog, but something isn’t matching up.
The only thing that I can think of that combines those facts is mis-use of Hard. I know you don’t think you were doing that – and your parameters don’t show the typical signs of it, but I wonder if that’s still the issue. Especially when I see how your Answer Buttons graph switches from Learning (so many Again) to Young/Mature (2x as many Hard as Again) – it seems like you were trying to avoid grading cards Again after they graduated.
FSRS should be looking at your review history with shorter-than-usual Hard intervals and understanding your success/failure rate based on intervals of that length. But instead it’s still over-estimating your intervals, which is giving you more lapses and lower retention results.
[@Expertium @L.M.Sherlock – any insights on what is happening here?]
Some suggestions:
- I suspect that if you tried the default parameters, they would give you terrible RMSE, but it would be interesting to find out. [You can cut yours out of the field, run Evaluate, and then paste yours right back in.]
- Looking back at your Reviews, it doesn’t look like your workload is increasing much, at least for the past year [and you’ve been using FSRS for almost a year – AnkiDroid 2.17 betas started Feb 2024]. Maybe there’s something in Reviews, Answer Buttons, and True Retention that we need to see “all history” to detect?
- How long ago did you increase DR to 93%? I assume when you say “recalculated the intervals every time so the intervals of all cards should be correct” – you mean you’re running an FSRS reschedule which changes when the cards are due to be studied, not just their intervals? If that’s been a while, then you might have to try an increase to 94% to offset whatever else is happening.
It is, there is a parameter that is very similar to the hard bonus or whatever it’s called in SM-2. But I don’t recommend tweaking it manually.
Honestly, idk. I guess this is one of those cases where FSRS just isn’t accurate enough for some reason that I don’t understand.
Yes, that was where I was headed. I suppose you can manually change any of the parameters – but it doesn’t mean it won’t end in disaster!
I use Hard for cards where I have to think about the answer but I recall it correctly. Good is for cards where I know the answer without doubts. Easy is for cards which bother me (too easy).
My backlog are temporarily buried siblings only. If you mean the first column in Review Intervals stat, the more than 2000 cards, these are suspended cards. I agree that the 85 % of the days could be one of the reasons of my low retention.
- My values with the default parameters are Log loss 0.2638 and RMSE 3.79, pretty good.
- It is true that the work load is not drastically increasing.
- You can see it in the Reviews stat. The tips over 1000 reviews are the days when I increased the desired retention, recalculated and studied the new backlog. Yes, I just increased to 94 %.
One more thing: check the Card info below. When I switched to FSRS, the easiness 145 % was recalculated to difficulty 80 %. Seems correct. But there was an interval 5.7 months. After I hit Good, the new interval was 1.56 years. It is approx. 3times of the previous interval which seems to be too long for a card with the difficulty 80 %.
Can you show me the whole revlog, from the start?
I didn’t find the same card but I found another example. When I see the graph at the end of the Card info, I probably use the Hard button too much…
One theory: maybe the algorithm works well only if someone study new cards daily. Let me explain.
If someone studies some amount of new cards daily, it meas that there are also new mature cards daily. These mature cards are reviewed and because they have quite short intervals (1 - 2 months), the learner remember them well which increase the retention. On the other side, I mainy study mature cards with long intervals (3 years in average) where there is higher probability to forgot.
It would be interesting to check the retention not only for learning cards, young cards and mature cards, but also split the mature cards to more groups (21 days-3months, 3-6months etc.).
Maybe. But I personally don’t learn new cards daily and sometimes there are weeks without learning new cards. FSRS seems to handle that just fine for me.
Agreed. Maybe even a custom interval that the user could choose (e.g. from 60 to 90 days).