With apologies for reviving this topic (explained later) I’ve also encountered this strange event where reoptimizing a preset I’d recently renamed and done more than a thousand reviews on suddenly told me that I’d only done (an arbitrary-looking) 92 since the 20th of October (I’m doing 200+ different cards every day). Since this is the only thread that comes up when I search for the problem, can you explain a bit more what the solution was? How did you get Anki to “update the preset name change?” I’ve tried optimizing by deck name instead but the result is the same. The deck doesn’t have subdecks but is split into two note types – I don’t know if that’s important. Going back by more months pre-preset name change drags in many more reviews (thousands) and successfully optimizes but I need the more recent date because I’ve significantly changed my question workflow (limiting the time to answer to 10 seconds from 120) which has affected my retention.
Hi kettle, I believe your error is a bit different from mine, with mine being simpler:
In my case I expected Anki to automatically update the name change in the search field in the FSRS section (it did not), and because it was such a minute change I didn’t notice the difference even when I was staring at it.
The point is that I was asking Anki to optimize a preset that didn’t exist, so it gave me that 0 reviews found error. But your situation is more nuanced, Anki still recognizes something and gives you a number different from 0, which puzzles me…
To answer your question
I never got Anki to do it, I did it manually (typed the correct name).
This bit I don’t understand
Let’s say you made the name change this October,
if you tweak the period to consider from, let’s say, 1970 to Sep 2024 it finds all the reviews you expected
but if you ask from 1970 to Oct 2024 it gives you only 92?
[I split you off to your own post. Feel free to re-title it if I didn’t understand your question.]
I don’t think your issue has anything to do with the change of the preset name. Are you using “Ignore reviews before”? That’s the change that matters!
That feature is pending a name-change of its own because what it actually does is ignore all review history for any card introduced as New before that date. It’s an extreme measure to cut out review history that would be a complete disaster for FSRS to consider.
I’m not sure you need to exclude your review history for this, but first, can you clarify – Did you change your “Maximum answer seconds”? Or are you using an Auto Advance feature where a card automatically gets marked wrong if you can’t answer it fast enough?
Ah, I understand now, sorry for dragging you into this then and thank you for the reply!
I think the new cards being introduced does have something to do with this in some way, as depending on the cut off date the amount of reviews it considers seems to vary considerably. This is how many the error 500 reports for each date of October:
22nd - 0
21st - 92
20th - 92
19th - 92
18th - 92
17th - 220
16th - 371
15th - It optimizes successfully.
This is despite me doing a similar number of unique reviews (200+) each day. But the changes seem to correspond with days when I also learned some new cards. As I’d made the change from 120 seconds to 10 my retention had dropped from the 90% I wanted to less than 70%, so I wasn’t adding new cards as often, but I did add 35 on the 21st, the 17th, the 16th and the 15th, which are the days when it brings in more reviews… But I didn’t add as many as the number is bumping up by, it was always 35. (And in case it matters, I wasn’t “adding” new cards, since it’s a premade deck I downloaded, but I did choose to see them for the first time.)
Just to clarify, do you mean if the “ignore reviews before” is set to October the 20th it ignores all review history for a card first seen prior to the 20th both before and after that date? As in, if a card was new on the 17th, it would even ignore reviews I made of it on the 22nd? I thought it was acting more like a cut-off, where reviews for all cards before that date aren’t considered, but all reviews afterwards are.
At the time my maximum answer seconds was still set to 120 as I thought auto-advance would overrule it, and auto-advance was 10 seconds, although it wasn’t automatically answering it for me as “seconds to show answer for” was set to 0, it was just displaying the back, then I was manually marking it as wrong after examining it first (which would take a different amount of time depending on how hard it was).
Yep.
I believe it was intended to be more like a cut-off, but it turns out that FSRS can’t deal with “headless” review history, so it has to exclude everything from that card.
“Maximum answer seconds” is for record-keeping, statistics, nothing more. If for some reason you feel like auto-advance is a good fit for you – at least it sounds like you’ve figured out how to use it correctly.
But I’m still not sure why you decided to exclude your review history. Were you getting odd parameters or outcomes when you optimized on your whole history?
Thank you for the clarification, I now understand how it works and why it wasn’t computing the other reviews, as well as how to get it to do so in future (add new cards).
I agree with you that renaming “ignore reviews before” may be something worth considering, together with updating the description for the option currently given by Anki:
“If set, reviews before the provided date will be ignored when optimizing & evaluating FSRS parameters.”
It doesn’t say that reviews of cards after the provided date will also be ignored if they were first seen before it – that was the source of my confusion.
Essentially yes, but it was more because I changed my workflow rather than something FSRS should have detected; it’s a deck I’ve been using for years with a 120 second limit per question. A combination of now having less spare time as well as a dawning that taking 100+ seconds to remember what a word means isn’t exactly a sign of language fluency meant that I reduced it to only 10 seconds, and as a result my retention went from a consistent 90% each day to <70%.
FSRS, taking account of the previous years’ reviews, was I think still treating me like I was going to continue getting 90% correct, even though I knew I was now very unlikely to as I’d changed my question workflow, which is why I wanted to get it to reoptimize only from the date after I’d made the change, because I don’t think other parameters like time limit for answering a question changing is something it takes account of.
I’m sure it would have cottoned on to things after a while given how big the retention change would have been, but it seemed a bit of a waste of time to go through those weeks with what seemed to me the wrong parameters, so I was hoping to save time and jump ahead using only data from my answers with the strict time limit, and I’ll still do so, but I’ll make sure I have enough reviews of new cards first.
Thanks again for your help!
There are so many other places to stop between 120 and 10 – I think there could be less extreme ways to handle that. But if getting answers inside 10 secs is important to you, I won’t try to stop you.
That’s not really what FSRS does. It looks at your history to see how you typically perform on cards that get a certain sequence of grades, and then it looks at the sequence of grades on the current card to figure out when you need to see it again. But if you get it wrong, that card now has a different sequence of grades, and FSRS would schedule it (after Relearn) differently.
This just doesn’t seem like a situation FSRS that can’t handle, but maybe I’ll be proven wrong.
Try this –
- As you have it now – with an ignore date and the default parameters – click Evaluate – and make a note of the log loss and RMSE, and how many reviews.
- Then, remove the ignore date, keep the default parameters – click Evaluate – and make a note of the log loss and RMSE, and how many reviews.
- Then Optimize on your full preset (leave it as the default, grayed-out query) – click Evaluate – and make a note of the log loss and RMSE, and how many reviews.
It would be interesting to know what the Evaluate and reviews are at each stage, and what your optimized parameters are [please post them as text, so I can copy them].