FSRS 5: Continuity of Card Difficulty Distribution

[DerIshmaelite]

Anki 24.10 Beta

Unrelated, but that is the most perfect distribution of difficulty I’ve ever seen.

Is there a reason why the distribution of Card Difficulty is wonky like this

or this (where there are 0 cards in some card difficulty ranges for no apparent reason)

and that they are not distributed in a “continuous” fashion. I feel difficulty should be continuous and with no break-offs.

[Expertium]

Difficulty update (from old D to new D) consists of 2 parts:

1. The update that depends on the grade. Since there are 4 grades, there are 4 possible values
2. The “mean reversion” as LMSherlock calls it, which is usually small.

If we assume that 2 is negligible - and it often is - it turns out that D doesn’t change continuously and only changes by 4 discrete values.
However, this is somewhat different in the latest version of FSRS, where we added another term to D to make it approach maximum asymptotically, so that D is never exactly equal to it’s maximum allowed value. This makes the distribution smoother. I suggest using the latest rc, optimizing parameters and checking the distribution again.

Last but not least, D cannot reach 0% if you never use Easy.

[DerIshmaelite]

@Expertium

I suggest using the latest rc, optimizing parameters, and checking the distribution again.

I already did. This is the distribution after optimizing with the new RC (Optimsing All Presets) and then pressing Reschedule All Cards on my Main Deck(with the help of the FSRS helper addon)
.

However, this is somewhat different in the latest version of FSRS, where we added another term to D to make it approach maximum asymptotically so that D is never exactly equal to its maximum allowed value.

Shouldn’t this apply in both ways A card should never be able to achieve maximum ease (or minimum difficulty) at all, even with pressing easy infinite times. “Maximum” ease or difficulty shouldn’t be a thing.

[Expertium]

Shouldn’t this apply in both ways

Nope. It only applies one way. A card can still reach exactly 0% difficulty - as long as you press Easy - but not exactly 100% difficulty.

[DerIshmaelite]

Nope. It only applies one way. A card can still reach exactly 0% difficulty - as long as you press Easy - but not exactly 100% difficulty.

I don’t think that that should be the case though. Is this a mathematical restriction, or a theoretical one Practically speaking, things can always get easier just as things can always get harder. They don’t hit a magical boundary of ease, is what I am saying.

[Expertium]

We haven’t tried making it so that D approaches the minimum value asymptotically

[DerIshmaelite]

I think it is worth trying (even though it may reduce RMSE minimally just like asymptote at maximum difficulty). @L.M.Sherlock sorry for pinging you.

[DerIshmaelite]

The point I am suggesting is that an ideal shape would be like the one @Aleksej had. But that happens so rarely.

FSRS 5: Continuity of Card Difficulty Distribution

Difficulty update (from old D to new D) consists of 2 parts: The update that depends on the grade. Since there are 4 grades, there are 4 possible values The “mean reversion” as LMSherlock calls it, which is usually small. If we assume that 2 is negligible - and it often is - it turns out that D doesn’t change continuously and only changes by 4 discrete values. However, this is somewhat different in the latest version of FSRS, where we added another term to D to make it approach maximum asymp…

Having asymptotes on both ends (max and min) should help, I believe.

[oktoberpaard]

I would think that a normal distribution only happens if all the cards have the same complexity and all the facts are equally unknown and have equal interference with real life exposure to those facts or related facts. An extreme example: a deck of randomly generated made up words. Some might click a bit better than others, but the chances of that happening are fully random.

I’m learning a language and I get much more exposure to words that are used frequently and there are also a lot of words that are similar to words of other languages that I speak. There is also a group of words that can’t relate to anything, or words that are similar but have a different meaning. My difficulty distribution doesn’t remotely look like a smooth normal distribution.

[DerIshmaelite]

That is not my point. My point is that it for example cards are either assigned a difficulty of lets say 68% or 75%, none in between. So my question, is why is it that there is a “void” for some D values.

Based on what @Expertium says, it is a restriction based on FSRS itself and how it assigns values.

I don’t know if this is good or bad, but I feel like difficulty is always continuous. There can always be difficulties between two values, even 2 consecutive difficulty values. For example all D values between 58% and 59% → 58.8% 58.76% 58.665% and so on… let alone D values between values as far from each other as 68% or 75%.

[Expertium]

I agree that continuous D is more intuitively pleasing. If D was also based on R at the time of the review, that could result in a continuous (or at least somewhat more smooth) distribution, but I’ve tried like 20 different ways of using R for updating D, and nothing helped.

[DerIshmaelite]

Yes. The scheduling is still static. It is not “real-time”. If the next interval was 2h on 10:00 AM and I do not press good, then come back later in the day 5 PM and then review it, the button will still say 2h.

• If there is a way to make FSRS “real-time”,
• that and the making an asymptote at minimum difficulty could perhaps lead to smoother difficulty curves.

But I don’t believe that’s all there is to it.

---

• I think this also has something to do with FSRS not being to factor in Short-term memory and how it behaves “real-time”

FSRS 5: <1d Scheduling and Learning Steps

FSRS-5 doesn’t consider the real interval of your learning steps because I haven’t figured out how to model the short-term memory accurately. So only the ratings matters in FSRS-5.

[Expertium]

• I think this also has something to do with FSRS not being to factor in Short-term memory and how it behaves “real-time”

It’s also a programming problem as well, since Anki counts intervals in “days”, and FSRS cannot get access to real time data.

[Expertium]

@DerIshmaelite

that and the making an asymptote at minimum difficulty could perhaps lead to smoother difficulty curves.

I benchmarked two-way damping. It made results worse. One-way damping is better.

Here are different functions that I used for damping: Two-way (non)linear damping | Desmos

The first one is the one currently used, and the best one. The damping factor goes to 0 (which means that the change in D approaches 0) as D approaches 10, and goes to 1 as D approaches 1 (which means that the change in D approaches whatever value it wants). Hence why I call it one-way damping - D approaches 10 asymptotically, but it can become exactly 1 quickly if you press Easy a lot.

The second one is non-linear and looks like a triangle. At 5.5 D is most volatile. As D approaches 1 or 10, the damping factor approaches 0. This means that here D approaches 1 asymptotically and 10 also asymptotically. It didn’t improve the results.

The last one is non-linear and smooth. I just wanted to test it because why not. It made results worse.

[DerIshmaelite]

I still feel like there is something more to this, but okay…If an asymptote at the other end doesn’t help, then leave it.

How else could one smoothen a curve in mathematics Taking derivative functions

---

I am not a mathematician, so I used ChatGPT because why not

Apply a Smoothing Function:

• Use a mathematical smoothing function such as a sigmoid or Gaussian to reduce sharp transitions in difficulty levels. For example, a sigmoid function could help map card characteristics to difficulty levels more gradually.
• A function like
  
  can help control the steepness of the transition, where k controls the slope, and x0 shifts the midpoint.

Use Interpolation:

• Interpolate between difficulty levels based on card attributes instead of assigning discrete jumps in difficulty. For example, you could use linear interpolation between difficulty scores of nearby cards.

Adjust the Scoring Model:

• Introduce more granular scoring criteria for difficulty calculation. This can help reduce the number of cards that fall into the same narrow range.
• For example, instead of assigning difficulty based on only a few factors (e.g., number of mistakes, time taken), consider incorporating other aspects such as card type, length of content, or review interval history

Add Randomness with Constraints:

• Adding a small amount of controlled randomness to the difficulty assignments can prevent clustering in specific difficulty bands. For instance, after calculating the initial difficulty score, you can add a slight random variation (e.g., ±5%) to reduce clustering.

Refine Your Binning Process:

• If you are binning cards into specific difficulty ranges, try increasing the number of bins or creating non-uniform bins. This can make the difficulty distribution appear more continuous.

Exponentially Decaying Weight:

• Weight earlier mistakes or intervals more heavily than recent ones by using an exponentially decaying factor. This can help prevent the effect of difficulty suddenly spiking due to recent performance and smooth out the curve over time.

Adjust Difficulty Growth Factor:

• If your function includes a growth factor (e.g., increases in difficulty based on past performance), try reducing the rate at which difficulty increases, especially for high-difficulty cards. This can prevent clusters at the higher end and allow for a more gradual difficulty curve.