( I’m not sure if I should post here in Anki > FSRS, in Anki > Help, or inside an existing post. Am I in the right place? )
Hello. I’m considering moving from Anki 24.11 to Anki 25.02 because it’s about time to optimize my parameters again.
However, when I ran “check database” in Anki 24.11, uninstalled Anki 24.11, installed Anki 25.02, ran “check database” again, and ran the optimizer on individual decks:
- The optimizer automatically ran a second time on some decks. It went to 100% and then started again.
- On those decks that it automatically ran a second time on, the log loss and RMSE appeared to get worse (increased in value). On the decks that it ran only once on, the log loss and RMSE either stayed the same, because parameters were already optimal, or got better (decreased in value).
- On those decks that had an increase in log loss and RMSE values, the last two parameters were set to
0.0000
(18th and 19th values, w17 and w18 from FSRS 5’s formula). - Here are the results. I omitted some for not being relevant. In each section, Anki 24.11 and Anki 25.02 both have the same “before” parameters. The “before” log loss and RMSE only appear different because of the different evaluation results between versions:
2025-03-09 --- kanji a (57,530 reviews) Anki 24.11 optimizer runs ONCE before: 0.3078 4.36% (ALREADY OPTIMAL) before: 0.3590, 0.5282, 0.9424, 1.3864, 7.7088, 0.4937, 2.3647, 0.0230, 1.1735, 0.1263, 0.5781, 1.8452, 0.1751, 0.3855, 0.9342, 0.0079, 3.9109, 0.5301, 0.5805 Anki 25.02 optimizer runs TWICE (automatic second run!) before: 0.2784 3.74% after: 0.2801 4.20% (WORSE, LAST TWO PARAMETERS SET TO 0.0000) before: 0.3590, 0.5282, 0.9424, 1.3864, 7.7088, 0.4937, 2.3647, 0.0230, 1.1735, 0.1263, 0.5781, 1.8452, 0.1751, 0.3855, 0.9342, 0.0079, 3.9109, 0.5301, 0.5805 after: 0.5533, 0.7786, 0.9872, 1.2084, 7.4332, 0.3943, 2.1594, 0.0274, 1.1543, 0.1721, 0.6164, 1.9848, 0.0938, 0.5042, 1.4209, 0.0000, 3.4684, 0.0000, 0.0000 --- kanji b (43,646 reviews) Anki 24.11 optimizer runs ONCE before: 0.2833 3.91% after: 0.2830 3.90% (BETTER) before: 0.5893, 1.2798, 4.0960, 24.1313, 7.3079, 0.1973, 2.3113, 0.0194, 1.3269, 0.0965, 0.7964, 1.7688, 0.2125, 0.2988, 1.6172, 0.0118, 4.8276, 0.5757, 0.8310 after: 0.5598, 1.2615, 4.2674, 24.2626, 7.3330, 0.4326, 2.4665, 0.0226, 1.3028, 0.0797, 0.7430, 1.8164, 0.2238, 0.3036, 1.4658, 0.0073, 4.3935, 0.5132, 0.8183 Anki 25.02 optimizer runs TWICE (automatic second run!) before: 0.2578 3.62% after: 0.2601 4.19% (WORSE, LAST TWO PARAMETERS SET TO 0.0000) before: 0.5893, 1.2798, 4.0960, 24.1313, 7.3079, 0.1973, 2.3113, 0.0194, 1.3269, 0.0965, 0.7964, 1.7688, 0.2125, 0.2988, 1.6172, 0.0118, 4.8276, 0.5757, 0.8310 after: 0.7790, 1.9671, 3.7444, 23.4165, 7.1470, 0.4380, 2.3402, 0.0174, 1.4060, 0.1139, 0.8506, 1.9492, 0.1259, 0.4012, 1.5203, 0.0170, 3.5602, 0.0000, 0.0000 --- vocab a (74,848 reviews) Anki 24.11 optimizer runs ONCE before: 0.3568 4.46% after: 0.3556 4.19% (BETTER) before: 0.6185, 0.8989, 1.5747, 44.2150, 7.0304, 0.4272, 2.2767, 0.0129, 1.5717, 0.2753, 0.9731, 2.0137, 0.0179, 0.3050, 0.7159, 0.0266, 4.8286, 0.3252, 0.1962 after: 0.6957, 0.9843, 1.6565, 48.9230, 6.9911, 0.5258, 2.7288, 0.0071, 1.6408, 0.2830, 0.9951, 2.1167, 0.0161, 0.3620, 0.1724, 0.0254, 5.1615, 0.2858, 0.1151 Anki 25.02 optimizer runs ONCE before: 0.3173 3.82% after: 0.3163 3.56% (BETTER) before: 0.6185, 0.8989, 1.5747, 44.2150, 7.0304, 0.4272, 2.2767, 0.0129, 1.5717, 0.2753, 0.9731, 2.0137, 0.0179, 0.3050, 0.7159, 0.0266, 4.8286, 0.3252, 0.1962 after: 0.6191, 1.0413, 1.4945, 48.7713, 7.0307, 0.6117, 2.3952, 0.0053, 1.6310, 0.3060, 0.9964, 1.9954, 0.0568, 0.3488, 0.6312, 0.0218, 3.3989, 0.3629, 0.2057 --- vocab b (59,458 reviews) Anki 24.11 optimizer runs ONCE before: 0.3303 4.51% (ALREADY OPTIMAL) before: 0.4403, 6.8131, 16.4443, 100.0000, 7.2472, 0.4877, 1.5236, 0.0146, 1.2928, 0.0000, 0.7570, 1.6987, 0.2186, 0.1348, 0.5454, 0.0214, 5.1275, 0.9048, 0.7033 Anki 25.02 optimizer runs TWICE (automatic second run!) before: 0.2952 4.59% after: 0.3031 4.92% (WORSE, LAST TWO PARAMETERS SET TO 0.0000) before: 0.4403, 6.8131, 16.4443, 100.0000, 7.2472, 0.4877, 1.5236, 0.0146, 1.2928, 0.0000, 0.7570, 1.6987, 0.2186, 0.1348, 0.5454, 0.0214, 5.1275, 0.9048, 0.7033 after: 0.7892, 7.8490, 26.6964, 100.0000, 7.1073, 0.3950, 1.9624, 0.0187, 1.3732, 0.0000, 0.8216, 1.9310, 0.0986, 0.3034, 0.5492, 0.0006, 4.0268, 0.0000, 0.0000 --- vocab c (49,279 reviews) Anki 24.11 optimizer runs ONCE before: 0.3189 5.01% after: 0.3171 4.78% (BETTER) before: 0.3116, 1.0480, 6.4631, 100.0000, 6.8019, 0.2849, 2.1378, 0.0075, 1.7005, 0.0004, 1.1392, 1.9142, 0.0814, 0.1911, 1.5837, 0.0000, 4.4897, 0.5012, 0.4408 after: 0.3550, 1.2864, 8.8747, 99.9999, 6.8671, 0.5838, 2.2402, 0.0031, 1.6890, 0.0369, 1.1443, 1.9644, 0.0296, 0.1991, 1.4039, 0.0000, 4.6054, 0.4230, 0.3997 Anki 25.02 optimizer runs ONCE before: 0.2922 4.94% (ALREADY OPTIMAL) before: 0.3116, 1.0480, 6.4631, 100.0000, 6.8019, 0.2849, 2.1378, 0.0075, 1.7005, 0.0004, 1.1392, 1.9142, 0.0814, 0.1911, 1.5837, 0.0000, 4.4897, 0.5012, 0.4408 --- vocab d (43,965 reviews) Anki 24.11 optimizer runs ONCE before: 0.3153 6.10% (ALREADY OPTIMAL) before: 0.6954, 2.1809, 2.1809, 98.5326, 7.0345, 0.9863, 2.5210, 0.0022, 1.8257, 0.0860, 1.2461, 1.9353, 0.0040, 0.2541, 1.8685, 0.0007, 3.7187, 0.3759, 0.1265 Anki 25.02 optimizer runs ONCE before: 0.3021 5.98% after: 0.3018 5.97% (BETTER) before: 0.6954, 2.1809, 2.1809, 98.5326, 7.0345, 0.9863, 2.5210, 0.0022, 1.8257, 0.0860, 1.2461, 1.9353, 0.0040, 0.2541, 1.8685, 0.0007, 3.7187, 0.3759, 0.1265 after: 0.6228, 2.4299, 2.4299, 97.3825, 6.9264, 0.8577, 2.4740, 0.0010, 1.8663, 0.0795, 1.2817, 2.0056, 0.0618, 0.2537, 1.9052, 0.0089, 3.1609, 0.4408, 0.2293 --- vocab e (5,308 reviews) Anki 24.11 optimizer runs ONCE before: 0.2289 7.04% after: 0.2298 6.97% (BETTER) before: 0.6146, 1.9837, 11.5027, 57.4142, 7.1394, 0.6637, 1.6189, 0.0010, 1.5890, 0.0578, 1.0589, 1.8717, 0.1991, 0.2597, 2.1564, 0.0000, 3.2597, 0.5024, 0.6668 after: 0.5414, 2.4369, 23.2735, 58.3397, 7.1479, 0.5046, 1.6477, 0.0010, 1.5312, 0.1054, 1.0089, 1.8551, 0.1978, 0.2253, 2.1788, 0.0185, 3.3407, 0.4260, 0.5532 Anki 25.02 optimizer runs TWICE (automatic second run!) before: 0.2176 6.74% after: 0.2202 6.77% (WORSE, LAST TWO PARAMETERS SET TO 0.0000) before: 0.6146, 1.9837, 11.5027, 57.4142, 7.1394, 0.6637, 1.6189, 0.0010, 1.5890, 0.0578, 1.0589, 1.8717, 0.1991, 0.2597, 2.1564, 0.0000, 3.2597, 0.5024, 0.6668 after: 0.3576, 4.1873, 23.1463, 58.0611, 7.1141, 0.5395, 1.6885, 0.0010, 1.5612, 0.0618, 1.0363, 1.9709, 0.0926, 0.3613, 2.3176, 0.0673, 3.0746, 0.0000, 0.0000
My questions:
- Is there a reason that the optimizer would run to 100% more than once?
- Is there a reason to expect an increase in log loss and RMSE when optimizing? For example, is this something caused by the recency weighting mentioned by @Expertium ?
- Is there a reason to expect the optimizer to set the last two parameters to
0.0000
despite all 19 parameters already being used? - Are these “worse” results in Anki 25.02 actually worse, or should I trust the optimizer and save?