FSRS 5: Ideas and Sims

[DerIshmaelite]

Current FSRS Ideas and Simulations at work (list being updated…)

1-Seed Collection Testing

FSRS 5: Optimizer

Right now I’m just gonna wait for like 30 hours to test 1 vs 10 seeds. That stuff takes forever even on a relatively small dataset EDIT: update, I finished doing it with 2 seeds on 200 collections, some with (relatively) few reviews, and some with a lot of reviews. Note that I deliberately selected collections close to the extreme ends - either a lot of reviews or not a lot. Here is the graph of the differences as a function of the number of reviews: [Figure_1] For each collection, RMSE(2 s…

2-Preset Size

Number of Cards worthy of a Preset

That’s something me and LMSherlock would like to investigate, but since our dataset doesn’t have deck/preset names, we can’t find out which cards belonged to which decks, and can’t measure how much splitting things into a million presets helps.

3-Short Term Memory Model

FSRS 5: <1d Scheduling and Learning Steps

If I figure out the short-term memory model, I will decrease the minimum stability limit in the same version.

FSRS 5: <1d Scheduling and Learning Steps

Found it: The median short-term stability of a card whose first rating is ‘again’ is 38 seconds. The median short-term stability of a card whose first rating is ‘again’ and second rating is ‘good’ is 24 minutes. [image]

4-Sorting Order (PSG, PRL)

Ordering Request: Reverse Relative Overdueness

So, turns out PSG cannot maintain desired retention at the specified level. I don’t think we should continue this. I will tell Dae to just implement reverse relative overdueness. P.S. I know sorata will disagree and say that we should try more sort orders. I will not be doing that.

5-Improving Calculations for Retrievability and Difficulty

[Enhancement] Improving the function for calculating difficulty · Issue #352 · open-spaced-repetition/fsrs4anki

[Feature Request] Improve Calculation of Retreivability · Issue #703 · open-spaced-repetition/fsrs4anki

6-Influence of Increasing Collection Size on Retention (Interference - Retrograde Inhibition)

7-Influence of Time of Day (Fatigue) on Scheduling Time Intervals

[Expertium]

1. Here are the results so far
   
   
   
   I’ll finish this test, but it seems pretty clear that it’s not worth it even for people who don’t have a lot of reviews, and definitely not worth it above 5k reviews.
2. Still waiting for the dataset
3. That’s Jarrett’s territory, not mine. I don’t have much to say
4. Dae will implement reverse relative overdueness, but probably not anything beyond that. And probably not my idea with only having 3-4 sort orders and “Custom”, sadge
5. Both of those are dead ends. We might be able to reduce RMSE by 1-2% (relatively), but not more than that, and even that will be difficult to do. That is, unless someone smarter than me and Jarrett decides to work on FSRS, but I doubt that will happen any time soon.

[DerIshmaelite]

6-Influence of Increasing Collection Size on Retention (Interference - Retrograde Inhibition)
7-Influence of Time of Day (Fatigue) on Scheduling Time Intervals

[Expertium]

6. I’m not sure what you mean.
7. I’ve experimented with that and got nowhere, and I don’t plan to continue working in that direction.

[DerIshmaelite]

I think that the size of the collection and the rate at which it grows (number of new cards per day) influences the retention of older learned cards.

New cards interfere with retention of older cards → Retroactive Inhibition

Retroactive inhibition | psychology | Britannica

Learning theory - Stages, Acquisition, Retention | Britannica

This also applies in the opposite direction.

Older cards interfere with learning of new cards → Anterograde Inhibition

So I believe that collection size (Older cards) and Number of New Cards per day are variables that should be looked into.

[MaleMonologue]

I was just doing my reviews today and I mixed up 連絡 and 連続. This one is due to visual similarity. However, more importantly, I have even mixed up visually dissimilar vocab because the meanings are so close to each other, despite the appearance of the word being completely different. Similar to the chair-table example, although it was for more abstract vocab, not nouns.

[Expertium]

Alright, I finished testing 1 vs 10 seeds
@L.M.Sherlock @vaibhav @sorata

For each collection, RMSE(10 seeds) is calculated simply as min(RMSE(seed 1), RMSE(seed 2)…RMSE(seed 10)).
I deliberately chose 100 collections with a small numer of reviews and 100 collections with a large number of reviews.

Difference in RMSE and the number of reviews.

Distribution of differences

Improvement and the number of seeds used

As you can see, it is absolutely not worth it even for people with <5k reviews (for other people the average is even lower).

Note that this is different from what I proposed here. It’s not the same idea.

[DerIshmaelite]

So is the idea done for good What happens if you test against 100 seeds

[Expertium]

I will get grey hairs before the optimizer finishes. And the improvement will be small anyway, probably around -0.2%

[DerIshmaelite]

probably around -0.2%

Yeah, no that is not worth the effort.