When/How to separate presets for FSRS

Anki FSRS
11

@sorata @DerIshmaelite here are the results:

image
image1400×800 32.7 KB

Correlation coefficient=-0.056

The correlation coefficient between the average RMSE and the number of presets is virtually 0. Visually, I was expecting to see a U-shaped curve with a minimum that corresponds to the best number of presets, but nope. And according to the benchmark, RMSE is actually performs worse by 3-4% (relative) when FSRS is optimized on several presets rather than on the entire collection.

Note that I can’t extract the number of reviews per each preset from the file Jarrett gave me, only the total number of reviews across all presets.

P.S. Out of 9999 collections, the maximum number of presets is 130. So DerIshmaelite, your 273 (or whatever number it was) is literally off the charts.

EDIT: here’s the difference between the RMSE of FSRS-5 optimized on the entire collection and the average RMSE of FSRS-5 optimized on each preset.

Figure_1
Figure_11400×800 40.1 KB

Average difference (unweighted)=0.003
Average difference (weighted by n reviews of each user)=0.002

Positive difference means that FSRS-5-preset is worse than FSRS-5.

1 Like