I updated from the official Anki version to beta 4. I’ve scrolled through the forum for a few hours, but there’s too many changes, so I’ll ask here instead:
I would definitely do that, as you want to have the best paramaters for this version of the scheduler.
Is that before or after optimizing? The old parameters are expected to give different results with the same day reviews taken into account.
This isn’t a help thread, so if you aren’t reporting issues with the Beta builds that need to be addressed before release, this isn’t where you should be posting. (I’m splitting your post into a new topic.)
If you want to use a Beta build, you should read the release notes for this whole cycle of Betas yourself (especially about things like the warnings about optimizing if you sync) – Releases · ankitects/anki · GitHub . If that’s too much work, then consider waiting for the official release.
Thanks for the split. I wasn’t sure where to post, and I couldn’t make my own post because my account is new.
Something I’m worried about is that “compute minimum recommended retention” gives 0.91 when simulating 10 years. Is this normal? With FSRS 4.5, the minimum computed retention was 0.75, so I’m not sure what change caused such a significant jump.
Here is the timeline of what I did:
FSRS 4.5: optimize parameters (17k reviews). RMSE(bins) 3%.
Compute minimum recommended retention 3650 days: 0.75
Update to 24.10 beta 4.
Optimize parameters (Almost 30k reviews). RMSE(bins) 7%.
Compute minimum recommended retention 3650 days: 0.91.
All I can say is that the minimum retention also went up for all my decks with newly optimized parameters, but in my case it seems to make sense.
That being said, 7% sounds like a relatively poor fit. I found that you can often get better parameters using the optimizer tool in the FSRS repository. Make sure to plug in your time zone and wether or not to ignore suspended cards (for example leeches). Then just run it and copy paste the parameters (without the surrounding brackets) and see if the result is better.
RMSE(bins) is now down to 2.94%. Minimum recommended retention went down to 0.87.
Thank you.
@MaleMonologue, log loss is not measured in %. I am assuming that you are talking about RMSE.
Such a difference between the built-in optimizer and the Python optimizer is not expected. Can you share your collection file using a Google Drive link so that @L.M.Sherlock could try to find out the problem?
I’ll give you a brief overview of what’s new in 24.10
Slip of the tongue. I meant RMSE(bins). When I used the optimizer after updating to Anki beta 4, and optimizing the deck, it showed it as 7%. But once I used the optimizer tool oktoberpaard linked, and pasted the parameters it gave me, it went down to less than 3% which is even better than what I had with FSRS 4.5.
Please export your Anki collection, upload it to Google Drive (or Mega or whatever cloud storage) and share the link so that L-M-Sherlock can take a look
In-built optimize makes the RMSE(bins) 7% (6% now that I checked again).
The parameters I got from the link which takes more time brings it down to 2.95%
parameters:
0.0600, 0.3690, 3.4252, 7.3442, 7.4576, 0.1680, 1.2247, 0.0586, 1.4001, 0.0171, 0.9224, 1.8133, 0.2455, 0.2576, 2.2416, 0.6477, 3.8554, 0.9509, 0.9145
However, the minimum recommended retention for 3650 days is 0.87 even though in the old version with FSRS 4.5, it was 0.75.
It says deleted for me, too. Please reupload it.
@L.M.Sherlock please take a look. This user is getting way better metrics using the Google Colab optimizer.
I thought it was normal, since the google colab optimizer took more time to get the parameters. What I was confused about was how minimum recommended retention jumped from 0.75 to 0.87 after updating to FSRS 5.
Before I go to sleep, is there anything else I need to upload? There’s the debug info:
Anki 24.10 (23b7d636) (ao)
Python 3.9.18 Qt 6.6.2 PyQt 6.6.1
Platform: Windows-10-10.0.22631
===Add-ons (active)===
(add-on provided name [Add-on folder, installed at, version, is config changed])
AnkiConnect [‘2055492159’, 2024-07-26T14:25, ‘None’, ‘’]
===IDs of active AnkiWeb add-ons===
2055492159
===Add-ons (inactive)===
(add-on provided name [Add-on folder, installed at, version, is config changed])
… I find that changing the seed of FSRS-rs could reduce the RMSE from 7% to 4% in that case.
Random Seed Is All You Need
How the heck?
I mean, in the article the author is talking about a 2% difference, but we are observing a crazy big difference, not just 1-2%.
How much increase in rmse should be considered normal when going from fsrs v4. 5 to fsrs v5?
In my case, the rmse increases slightly from 2.27% to 2.75%.
On average, FSRS-5 is more accurate, not less, so…
Did you optimize parameters?
Sorry, I mixed up few values in above comment. The below is correct version.
0.2283, 1.5549, 21.3778, 36.7666, 4.4554, 1.3250, 1.0327, 0.0059, 2.4182, 0.2906, 1.6419, 2.2392, 0.1747, 0.3667, 2.8691, 0.1053, 6.0000
fsrs v4.5 rmse 2.41
fsrs v5 rmse 2.27
optimize gives “The FSRS parameters currently appear to be optimal.”
so i reset the parameters and the optimize to get 19 values parameters
0.3736, 5.9044, 24.2286, 38.0612, 7.1393, 0.7132, 1.1613, 0.0000, 2.3532, 0.2738, 1.6630, 1.7925, 0.2365, 0.3899, 3.0824, 0.0837, 6.0000, 1.1413, 1.5148
fsrs v5 rmse 2.75
You shouldn’t reset parameters if you see “The FSRS parameters currently appear to be optimal”. It’s not an indication of a problem, and, as you can see, it can make the results worse sometimes.