I think my FSRS settings are messed up - Help with fixing same time choices for most cards, same times on same cards for some, and too large of time intervals for others

First off, sorry if this is under the wrong category. I thought at first it belonged under basic help, but seeing as it’s related to FSRS, I decided to put it here. Basically, I need help with Anki giving me strange time interval choices, with a significant proportion of cards offering the same choices respectively, having huge time gaps between choice intervals (for new cards I found that especially strange), and choices on the same cards even being exactly the same.

I’m not just a n00b but I’m just plain stupid when it comes to numbers and math, especially statistics and calculations, so how FSRS works (aside from the concept itself) is like rocket science for me. I read the FAQs and guides but to be honest it’s like in one ear and out the other, so I apologize in advance if this question is so stupidly easy that it’s a waste of time for you.

Context and Background:

  1. I’ve been using Anki to study Thai for about 3 months now. I have 2332 cards, which are Basic and Reverse, so I guess technically I’ve only created 1166 cards myself.
  2. I didn’t learn about the reverse cards option until about a month or so in, so I changed around a thousand cards at the time from basic to basic and reversed.
  3. A couple weeks ago I added a 3rd field to the cards, an IPA/transliteration field for the Thai. I exported the entire card set to a CSV, populated it with the 3rd IPA field, and then imported it again.

Modifications:

  1. The fact I changed Basic to Basic and Reverse a good halfway into the study set, doubling the cards at the time.
  2. A month or so in to studying, I turned on FSRS after reading how it can be more useful for languages. When I first did it, I clicked “Optimize”, thinking that’s what I was suppose to do (I think I read that I should). That may have been when the problem started but I’m not sure.
  3. I added a 3rd field to the Anki set and repopulated the entire existing set with the updated field a couple weeks ago.
  4. I added the FSRS Helper add-on about a month ago and did “reschedule all cards” and “disperse all siblings” thinking it might help but it didn’t. I haven’t touched it since for fear of screwing things up more.

My observations:

  1. Lots of cards I review jump from 10m to 8d - a significantly, and suspiciously, larger proportion than what I’d expect - I’d probably say between 20 to 25% of the cards I review every day give those choices, whether I’ve reviewed it 4 times or 40 times.
  2. Even many new cards, ones that I look at for the first time, will eventually be <1m, 10m, 8d, and 10d, which is very annoying. Since they are still fresh, I would like to review them a couple of times within the near future, not a week later. For these cards, I end up having to click More > Set Due Date > 1 day or 2-3 days depending on comfort level.
  3. Cards that I have reviewed a dozen times already, within a week or so, but are still somewhat new or difficult for me, also have confusing time jump choices, such as going from 10m to 7d or more. A card that’s still new to me, I would think it’d be best to review it again within 2-4 days, not a week or more later.
  4. Some cards have the same time choices for both good and easy options.
  5. Some cards, regardless of whether I choose “again” or “hard” will consistently display the same unchanging choices. I tested this hoping to get a new card that gave me the choice of <1m <6m <10m and 16d to eventually give me something less than 16d, but no matter how many times I clicked again or hard, it never changed.

I’m sure I could probably provide other useful information to any kind soul who dares to help, but I’m not sure what exactly you need, so rather than posting a dozen more screenshots of various possibly useless Anki stats, if you (the kind soul) would be willing to help me fix these time interval issues, just let me know what exactly you need and I’ll do my best to provide it (you might need to tell me where to look to find it though).

Example screenshots of the time intervals are below. I would really appreciate any help on this. I’ve been hesitant to try to fix it myself for fear of making it worse, hoping maybe the problem would solve itself after a couple weeks of reviewing and setting days, but nothing is changing unfortunately. So here’s hoping someone out there understands the issue and maybe even knows a quick fix for it. Thanks a bunch to anyone willing to help.

Example pictures:
Cards with the same good and easy options:


Lots of 8d cards:




Confusing jumps:


New Cards, first time seeing them:



1 Like

I don’t see anything obviously out of the ordinary here, but let’s talk about a few things. (And I promise the most you’ll have to do with numbers is compare which one is bigger or smaller!)

Intervals in FSRS are generally longer than intervals were using the original algorithm. It’s very common (Frequently asked questions about FSRS - Anki FAQs), and nothing to be alarmed about. What you can consider is that just because you would like to study them sooner doesn’t mean you need to study them sooner. FSRS is scheduling cards so that you can study them as little as possible to maintain your desired retention.

But studying is no fun if you don’t feel confident about what you’re doing, and no one is going to force you to trust this algorithm before you’re ready! Here’s some things you can try in the meantime –

  • Yes, optimizing was a good idea when you started. It’s probably time to do it again.

    • You only had 1 month of review history when you optimized your parameters, and now you’ve got another 2 months, so that gives FSRS a lot more data to consider.

    • [Rescheduling with the Helper add-on wasn’t necessary, but the only downside is that it can surprise you with a backlog of overdue cards. Since you didn’t mention that, it sounds like that wasn’t an issue for you. Regardless, it’s water under the bridge.]

  • Before you do that, let’s see how FSRS is doing!

    • Below your parameters, click “Evaluate” – make a note of the RMSE(bins): ##.##% figure.
    • Now click “Optimize” and FSRS will give you new parameters. Click “Evaluate” again, and note your RMSE.
    • Just for fun, blank out the parameters field (use the reset-circle-arrow on the right, or just delete what’s in the field) so you see the defaults in gray. Again “Evaluate” and make a note of your RMSE.
    • [When you’re done with all of that experimenting, you can close Deck Options without saving.]
    • What is your RMSE for your current parameters, new parameters, and default parameters?
  • Change your desired retention (as the link said). You didn’t say what your DR is, so I can’t give you any suggestions about what to do with it yet. [And you might want to re-optimize first and see how things go.]

To talk about the specifics of any card, we’ll need to see –

  • Your Learning and Relearning steps
  • Your parameters (as text, please) – the current ones you’ve been using up until now.
  • Your Desired Retention
  • The Card Info for the card you want to ask about.
4 Likes

Hi Danika. First and foremost, thank you, sincerely, for replying. I know I posted a lot, and I know you took time out of your day to reply to me, so I’d just like to know I truly appreciate that, regardless of the outcome.

On to your post.

You mentioned that I myself didn’t mention a backlog of overdue cards. To be honest, I’ve never seen a log of any overdue cards, but I also haven’t been looking for it. Is this the “Due” stat on the Deck page? I checked under the Statistics options but didn’t see any there. If it is the “Due” state, then right now it’s only at 86, so I guess there’s not much of a backlog.

As for FSRS, these are the values for my current, new, and default parameters:

  • Clicking “Evaluate” displays: Log loss 0.5226, RMSE(bins) 14.22%. (no idea what that means though).
  • After optimizing, clicking “Evaluate” displays: Log loss 0.4578, RMSE(bins) 8.74%.
  • After blanking out the parameter field, clicking “Evaluate” displays: Log loss 0.5258, RMSE(bins) 14.30%
  • I closed and did not save, so the values are still 0.5226, etc.

My desired retention was and is currently set to .90. Sorry for having failed to mention that earlier. I haven’t re-optimized it yet, I’ll wait to see what your thoughts are on the current FSRS values. Quite honestly, I’m not sure if I should change it or not - having over 2000 cards, I definitely don’t want to review ones I know more than necessary, but at the same time, I don’t like how new cards, one’s I’ve only seen a couple times, will jump from 10m to 8 or even 16d. So I’m still undecided here, perhaps you could give me your opinion?

Are the “Learning steps” and “Relearning steps” you mentioned the values under “New Cards” and “Lapses” respectively? If so, here are all the values on the deck to cover all parameters, but just let me know if I’m missing something:

Daily Limits

  • New cards/day: Preset 50
  • Maximum reviews/day: Preset 200
  • New cards ignore review limit: On
  • Limits start from top: Off

New Cards

  • Learning steps: 1m 10m
  • Insertion order: Sequential (oldest cards first)

Lapses

  • Relearning steps: 10m
  • Leech threshold: 8
  • Leech action: Tag Only

Display Order

  • New card gather order: Random cards
  • New card sort order: Card type, then order gathered
  • New/review order: Mix with reviews
  • Interday learning/review order: Mix with reviews
  • Review sort order: Due date, then random

FSRS

  • FSRS: On
  • Desired retention: 0.90
  • FSRS parameters:
    0.6276, 1.5183, 5.7129, 13.8206, 5.1618, 1.2298, 0.8975, 0.0310, 1.6474, 0.1367, 1.0461, 2.1072, 0.0793, 0.3246, 1.5870, 0.2272, 2.8755
  • Preset: “Default” -is:suspended
  • Optimize / Evaluate: Optimize (selected)
  • Number of reviews: 6,504
  • Reschedule cards on change: Off
  • Compute minimum recommended retention:
    • Days to simulate: 365
    • Computed minimum retention: 0.81
    • Iteration: 5

Burying

  • Bury new siblings: Off
  • Bury review siblings: Off
  • Bury interday learning siblings: Off

Audio

  • Don’t play audio automatically: On
  • Skip question when replaying answer: Off

Timer

  • Maximum answer seconds: 60
  • Show answer timer: Off
  • Stop timer on answer: Off

Auto Advance

  • Seconds to show question for: 0.0
  • Seconds to show answer for: 0.0
  • Wait for audio: On
  • Question action: Show Answer
  • Answer action: Bury Card

Easy Days

  • All set to Normal

Advanced

  • Maximum interval: 36500
  • Historical retention: 0.90
  • Ignore cards reviewed before: 01/01/1970

And I know you didn’t ask, but for the sake of covering all ground, I figured I’d post my Statistics at the end of this post as well. I’ve never once looked at this page as I don’t understand any of the stats, it’s all Greek to me, but perhaps it can give you some insight. I know you prefer text, but I used image version to keep it short and easily skippable if you don’t need it. If you’d like it in text version however, let me know and I’ll post it that way.

One other note I failed to mention last time, which will explain why I have a New cards/day present of 50 and a Maximum reviews/day of 200. So I had about 1000 cards in the deck, most reviewed by the time I converted them to “basic and reversed”. I continued to study for a month or two with default parameters until one day I noticed there was almost 1000 “new” cards that I never reviewed, plus hundreds of new ones I had put in by the time I saw this. Over the weeks, I realized I was getting a handful of new “recently added” words and a ton of new “old but reversed” words in each review - and to be honest, I know most of those new “old but reversed” words really well already, even though I haven’t seen that particular reverse card in Anki yet, and I’d like to get them out of the way and off the “new” list so I can work on the new recently added cards, so I upped the new cards per day to 50, and until I whittle away the over 1000 new cards list, I’ve found this to be a good way to get that 1000 deep backlog of ‘new but old’ cards out of the way sooner rather than later. So each time I see one of those pop up, which always says <1m <6m <10m and 16d, I just choose the 16d and move on. Truly new ones to me, I’ll just do again or hard so it’ll come up again sooner (but like mentioned, after that, it still goes from 10m to 8d/16d, which I thought is a huge jump for a new card, but I digress). At this rate I should be all caught up with ‘new cards’ within about 2 weeks.

I had hoped in my next set of reviewing today I would come across one of those cards that had the same time intervals for both good and easy, as to help determine why it’s the same like that, but unfortunately none popped up today and I’m not sure how to go about finding those ones yet. Is this situation also normal and nothing to worry about?

I suppose that’s everything for now, all the information you requested and my replies to your questions or comments. I hope it’s useful. Thank you again so much for the reply, explanations, and guidance on the matter. I’m curious to know what you think and what you recommend as the best course of action to get my time intervals back in order (if necessary).

My stats:



1 Like

Your card stability is too low. 26 days is very very low. My average for japanese is months. You probably need to improve the quality of your cards. More refinement = more memory. Also, read this ENTIRE article, do not skip:

twenty-rules-of-formulating-knowledge

I’m not allowed to link so just search that to find the webpage. It’s a webpage on supermemos website.

1 Like

Here it is:

3 Likes

Excellent! Now we know that you should definitely re-optimize. When you optimized to get your current parameters, you had little enough review history that FSRS only optimized the first 4 parameters – the rest are defaults. That’s why you see that your current parameters and defaults have about the same RMSE – on average, FSRS misses predicting your memory curve 14% of the time.

The re-optimized parameters are better! FSRS misses 8% of the time, but you’ve only got 6500 reviews yet, so that will keep getting better the more reviews you accumulate and re-optimize.

Recommendation #1

Re-optimize your FSRS parameters and Save. [All of your decks are assigned to the same Deck Options preset group, right?] Post your new parameters so we can take a look at them – but only the 17 numbers from the “FSRS Parameters” box this time. That’s all we need.

[For now, I’d leave your desired retention where it is. Let’s see what the new parameters look like.]

Recommendation #2

Use the Helper add-on to “reschedule all cards.” You already have a snapshot of your collection Stats, so you can compare right away – how did the Average Review Intervals, Stability, Difficulty, and Retrievability change? How did your Future Due change – and most importantly, do you now have a significant backlog? After you look at all that, if you like how it looks, you can leave it. But you can also Edit > Undo to put things back, if you want to discuss it first.

It’s not really a log – “backlog” is the word we use to talk about an accumulation of overdue cards that have not been studied yet. If you’re studying all of your Due cards every day, and graduating all of your Learn/Relearn cards to Review – you don’t have a backlog.

That’s more than I needed, and more than you needed to type (sorry for all that work!) – but I appreciate the thoroughness. Those daily New and daily Max limits were the only things I saw in your Deck Options that was worth taking a look at. Now that I understand why you’re doing that, I won’t quibble with you about the daily New limit. [I’m sure you saw the warning message in your Deck Options telling you that those numbers don’t go together.]

Recommendation #3

Regardless of what you do with the daily New – you should change you Max reviews/day to unlimited (9999). You want to be able to study all of the cards that are Due each day, and if you set a lower daily Max limit and reach it, that means Anki is just hiding those cards from you.

[If it helps you to know what happened with that 1000 extra New cards – those were created when you changed your note type to “basic and reversed.” Since you’d already introduced so many of their “sibling” cards (made from the same note), you’ve got a lot of them waiting. But it sounds like you’re making good progress. ]

Review Intervals: I don’t agree with the commenter who said your average interval is too low – you’ve got a very young collection, so I wouldn’t expect anything longer than that. It’s always good to consider whether you could improve your cards, but it sounds like you’re studying vocab, so there aren’t many ways you can go wrong!

Answer Buttons: I’m a little bit concerned about how often you are grading your answers Hard. So let’s just confirm – when you click Hard, that means you got the answer correct, but it was a struggle. If you got the answer wrong, the only grade to use is Again – see: Studying - Anki Manual. If that’s what you’re doing, then everything is fine, and you can go about your business.

Recommendation #4

If you’re doing something else, please say so, because you should nip that in the bud – and there might be some other things that need fixing.

True Retention: Looks good. And it tells us that 90% desired retention appears to be a good fit for where you are now.

I know I gave you a long shopping list last time, but without the Card Info, there’s not much we can do there. If you want to find those cards, you have screenshots of them above, so you can look them up in the Browse window and look at their Card Info from there. But, at this point, since you’re changing your parameters anyway, let’s put a pin in that. We can diagnose individual cards if we need to once your using the new parameters.

3 Likes

Thanks for the advice. Since I’ve only been using these cards for about 3 months, would it even be possible to have an average stability in months at this point? And to be honest, I don’t know what you mean by “improve the quality of your cards”. How can a card with a word on one side and a translation on the other even have levels of quality? Considering they have sound and IPA, more than most other Thai flashcards sets out there have, I considered these to be pretty good quality. :stuck_out_tongue: How can it be improved? But I might be getting ahead of myself and the answer to that question might be in the article you sent. I’ll give that a read soon, sounds interesting.

Thanks so much once again Danika for the wonderful, logical, friendly reply. Really appreciate it! Now I’ll respond to your recommendations:

Recommendation #1

I went ahead and re-optimized the FSRS parameters and saved. And to answer the sub-question, I only have one deck, so all the deck options are assigned to it (I assume). The new parameters are as follows:
0.4891, 0.8131, 1.6377, 1.9440, 7.4304, 0.6496, 1.4157, 0.0235, 1.4064, 0.3957, 0.9005, 2.1567, 0.0429, 0.5320, 2.4682, 0.4987, 2.6082, 0.3681, 0.6465
Clicking Evaluate now shows: Log loss 0.4562, RMSE(bins) 8.59%.

Recommendation #2

I used the Helper add-on to “reschedule all cards.”
Here are how the stats compare in picture format, since I wasn’t sure if it would be better or not to convert it all to text, I left it as Anki displays. I took the old stats screenshots and edited them to be side by side with the new stats screenshots for easy comparison. However, I’m sorry to say, this is were my brain fails me. I don’t know what these numbers or graphs mean. I mean, I get the English, sure, but not the relevance of the changes. Are they better or worse? And I’m not even sure what the backlog is either, only that now there is a “backlog” box whereas before there wasn’t. Is this saying I have a backlog of almost 1500 cards? :open_mouth:


At this point, since I don’t really know what most of these changes really mean for my studies, I can’t really say if I like how it looks or not. With that said, I undid the reschedule, because I’d like to wait for your feedback on it before going through with it or not, if you don’t mind. Based on what you said though: "“backlog” is the word we use to talk about an accumulation of overdue cards that have not been studied yet.” I would guess the “New” stat under “Card Counts” would be my backlog, which is currently at 517, 22.17% (but as mentioned, the majority of those are “basic converted to basic and reverse” cards that I’m just trying to plow through so I can get to the truly new stuff).

Recommendation #3

I changed the Max reviews/day to 9999. I see what you mean about how Anki would end up hiding any cards beyond the number I set.

Answer Buttons : I think I’m doing everything fine here, but I admit sometimes I just click the button according to when I personally feel would be a good time for me to review it again, so I’m probably being biased in some of my answers. In general though, I usually reserve again for if I’m wrong, and hard for when I can’t recall immediately, good for when I’m satisfied but still want to be reminded again in the somewhat near future, and then easy for when I know I’ve 99% mastered it but still want it in back in the far future. Once I know I’ve got it and never want to see it again, I suspend it. Buuuuuut, like I’ve mentioned, sometimes those time intervals don’t work for me, so I also frequently click again and hard, just for the sake of it, until a lower time interval for good becomes an option and then I’ll click good. So yeah, once in awhile I just spam again and hard until I see a time interval I like.

Recommendation #4

I think I explained the “something else” I might have been doing that your referring to above. Unless you were referring to another something else, in which case just let me know.

Excellent idea about just using the screenshots of the cards I posted to find them in the browser as to let me post the card info. Like you said though, I’ll forego posting those until we figure out the best parameters. And I think I’ve responded to everything for now, but if I missed anything let me know and I’ll provide ASAP. Thanks again Danika, I can see that I’m very lucky to have your help with this. I owe you a beer or twenty! :slight_smile:

Sorry, I missed your response yesterday!

#1

Those parameters look good, so I hope you started using them! Your collection is still quite young, so as you accumulate more history, I think you’ll see that 8% RMSE start dropping.

If you plug your new parameters into the FSRS Visualizer, and compare that to your old ones (or even the defaults), you’ll see immediately that your intervals will be shorter.

#2

You can see that difference in the Stats changes too – like average interval dropping from 26d to 17d. Your intervals should start out shorter and grow slower, which I think is what you were looking for.

Goodness no! :sweat_smile: That total is how many cards you have due in the next 30 days. The backlog part is only the ones due before today, so to the left of 0 – it looks like about 200 cards? You can see that more clearly on the main decks screen – your “Due” (green) cards today will jump from 80 to 200-something, because all of the overdue cards are counted there.
image

I think, if you don’t find that high number of Due Reviews too intimidating, you should go ahead and stick with the reschedule. That is what is going to bring all those future cards down to more reasonable intervals. The reason you get a backlog is that some of the cards were pulled so short that you’re already past their due dates. I think you’ll be happier studying those sooner rather than later too.

If it’s about 200 extra cards, it won’t take long to work through. Put your New cards on 0 for a few days and use the extra time to catch up. Even if you only do 30 extra per day, you’ll finish in less than a week.

[If you do find the high number of Due Reviews intimidating, there are ways to make that less apparent while you to catch up, so let me know.]

No, New cards wouldn’t be in the backlog, because you haven’t started studying them yet. It’s only Review cards that have gone past their due date.

#3

:see_no_evil: I hope that you won’t feel the need to do that anymore now that your intervals will be a better fit. Think of it as garbage-in, garbage-out – if you give FSRS incorrect data, it will reward you by being bad at scheduling your cards! :wink: Now is a good time to get into a better habit.

If the intervals are still distracting, you can turn them off in Preferences, and see how that feels. The best way to check the algorithm’s effectiveness isn’t how you feel about the intervals on the buttons, but how you actually do studying those cards when they are due. So keep an eye on your True Retention and as long as you like those outcomes, you can trust that the algorithm is getting you there with the intervals it is setting.

3 Likes

As usual, thanks a ton for the insightful reply! It really means a lot, you’re willingness to help a stranger. Talk about the Christmas spirit! Haha. Right then, time for my reply…

#1

I hadn’t before, not the “Re-schedule” that is (I saved the re-optimize though) but with that said, I am as of this post. Looking at the FSRS Visualizer instantly gave me a migraine so I think I’ll refrain from trying to understand that for awhile.

#2

But I can definitely see the shorter intervals, which is definitely better with new words. And I think I get it about the backlog too, which sadly jumped from 116 to 487 after doing the re-schedule above. It’s a lot, and I know I can’t do it all today (especially since it’s Christmas, I’ll be lucky to get in 50), but over the next several days I can probably whittle that down to a reasonable amount. So I’ll stick with it for awhile and hopefully these numbers will balance out, especially now that they’ve been re-optimized and re-scheduled.

#3

Thanks to you, I’m now being more discipled with my answers. I’ve caught myself time and time again wanting to choose the time interval that I’m personally comfortable with rather than answering based on how easy it was for me to remember, but I think the re-optimizing from our last chat helped with that, I’ve seen far less cards with seemingly random time-intervals these days, so that definitely helped. And like you said, I’ll start keeping an eye on True Retention. Which brings up a quick question though - under True Retention, there’s a lot of stats:


Which one’s should I be looking at to see how good I’m doing? The Total? If so, I assume as long as the most recent reviews (yesterday, last week, and maybe last month) are around 90%, then I those are good outcomes, because my desired retention set under the FSRS settings is .90. Please correct me if I’m wrong though.

And it looks like that’s it. You’ve run me through all the steps and helped me figure out how to fix these crazy time intervals. I’ll do some more reviews over the next couple of days and see how the new intervals are working for me, but even from just half of the advice you gave me I already started seeing improvements and better results, so I’ve no doubt with the rest of the changes it’ll only get better from here. Nevertheless I’ll update you on the results a bit later, for kicks and giggles, and to remind you of how thankful I am. For now, well, thank you yet again, and wishing you the absolute best Christmas and naturally a very very happy New Years. :smile:

Fair enough!

Yep, that’s plenty. Don’t base it on one day, because that can fluctuate too much – but a week or month gives you a good sense. [Hopefully that table will look better in future releases – until then, if you hold down Shift when you click Stats, you’ll get the Helper add-on’s version of the stats page, and the table looks a bit better.]

Happy New Year and happy studying!

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.