Suggestion about the learning steps after analyzing the Anki Dataset

Findings

For newly introduced cards, the average intraday retention corresponding to different initial ratings are:

  1. Again: 74.2%
  2. Hard: 91.7%
  3. Good: 96.2%
  4. Easy: N/A (the card is graduated immediately)

For lapsed cards, the average intraday retention is 85.3%.

Data and code are available at: Anki-button-usage/analysis.ipynb at main · open-spaced-repetition/Anki-button-usage · GitHub

The current default learning steps are set at 1m 10m, with the default relearning step at 10m.

This translates to the following intervals for new cards based on ratings:

  1. Again: Review after 1 minute
  2. Hard: Review after 6 minutes
  3. Good: Review after 10 minutes

Lapsed cards are reviewed after 10 minutes.

Suggestion

For typical users:

New cards:

  • The ‘Again’ interval is excessively long
  • The ‘Good’ review interval is insufficiently long

Lapsed cards:

  • The interval is excessively long

To elevate the intraday retention from 74.2% to 90%, the interval should be reduced by a factor of 3.5. Thus, 60s / 3.5 ≈ 17s.

To decrease the intraday retention from 96.2% to 90%, the interval should be increased by a factor of 3. Hence, 10m * 3 = 30m.

To raise the intraday retention for lapsed cards from 85.3% to 90%, the interval should be reduced by a factor of 1.6. Therefore, 10m / 1.6 ≈ 6.25m.

Given these calculations, the proposed adjustments are:

  • Default learning steps: 17s 30m
  • Default relearning step: 6m

For a more moderate approach, consider 30s 15m as the default learning steps.

11 Likes

Have you figured out what retention level is most optimal for intraday learning?

It’s still unclear. But high retention is definitely not optimal.

As the figure below shows, the stability increases 42x when reviewed in 30m, or 30x when reviewed in 10m.

Did you not see the real delays but the intervals instead? With Anki’s 20 minute learn ahead, that will be really inaccurate but you already know that I guess?

It’s the real delay.

Here is an extreme case:

My suggestion only applies for the default setting for average users. For your own case, I need your data to draw the conclusion.

1 Like

To get this conclusion, you need to make an assumption that a very large fraction of these reviews were done after 60s.

But, what if a significant number of users in the dataset are not using the default learning steps. Also, what if they are using the default learning steps but they allow a longer time to pass before they get back to the reviews (if they are busy with other tasks).

As an extreme example, what if all of the reviews that were actually done after 1 min were Pass and only those which were delayed (because of different learning steps or busy schedules) were Fail. In this case, retention after 1 min is 100%.

1 Like

You are right. According to my initial analysis, the median interval after pressing again is 114s. Seems most people tend to delay it. However, it could be another reason to shorten the learning steps because if people tend to delay, a short step may make up for it.

1 Like

Please wait for Anki 24.11. It will provide a API to export your data in specific format.

1 Like

This is very interesting! Are you implying that shorter again intervals might be better to prevent incorrect memorization of information?

Any thoughts regarding 17 seconds leading to brute force memorization?

He is aiming for 90% retention I think.

Consider that even if you do your reviews in one go, you’ll have a queue of learning cards getting due around the same time. Going through them means what was a 30 second interval becomes, say, 60 seconds. So, some cards will have a shorter delay and some will have a longer delay.

Although that’s not always happening. If you spend the same amount of time for every single review, cards will appear at the right time and will get reviewed at the right time.

I guess from a mathematical point of view the extremely short intervals make sense, but 17 seconds is just absurd. That would mean 1 or 2 flashcard reviews in between… there is no chance a user can experience a genuine memory recall, and the number of total reviews will increase significantly.

Aiming at a 90% retention rate or any specific target shouldn’t be enforced in the very short term. Seeing a card again in 17 seconds would not have a more meaningful impact than reviewing the card in 10 minutes or the next day (after memory consolidation during sleep).

1 Like

Fine. I think I cannot figure out a reasonable short-term memory model before I quit…

But it’s a good choice to increase the second steps.

3 Likes

Fine. I developed an experimental feature in the FSRS helper add-on:

The "1": 48 means your stability of new card which you rated again firstly is 48 seconds.

Feedback is welcome:

4 Likes

This is excellent, thank you!

From the stats shown for your deck, would the correct learning steps for 90% retention then be 48s,8083s? Is there enough information in these stats shown to give the correct relearning step(s)?

Are these values expected to match the intraday intervals given by FSRS when I leave leave learning steps blank?

It’s hard to query the necessary data to calculate the stability fro relearning steps. But I will try.

1 Like

I have always known that the again interval was way too high for my liking even though people were trying to tell me off it. Since my memory is on the other extreme end of being borderline fish-memory, I am currently using 1s 3m as my learning intervals with my automatic learning step scheduler handling the rest.

Would I also be able to know if this is optimal in my case by any chance :question: How does this work :question:

How did you get this to work if I may ask