[23.10 Desktop] Add a feature to automatically periodically re-optimize FSRS parameters

[voczi]

cant this automatically be done on the sync server itself shortly before start of day?
for ankiweb it makes sense to limit this feature depending on the deck size, so that too much computational power isnt used. but for a selfhosted sync server, any limit should be decided by the user.
or is it maybe more complicated than that? i see theres a lot of opinions in here >_>

[vaibhav]

@BrayanDSO

Preset/card changes can also end up being reverted if user was not already in sync.

Can’t they be reverted without automatic optimizations as well? Has that created any problem until now?

When making changes manually, the user will ensure that the changes from the first device are synced before making further changes on a different device. However, with automatic optimization, the change happens without any intervention by the user, making sync conflicts more likely.

I am in favor of automatic optimization but we must find a solution to the issue of sync conflicts before we can start developing the feature.

After reading the discussion about the issue, I think that there is no way to prevent the sync conflict. However, I think that we can modify the sync code to change how Anki deals with the sync conflicts.

My idea:
If the same card was reviewed on one device and its memory states were changed on another device without any review (due to automatic optimization), then keep all the revlogs and recalculate the memory states from the new parameters using the complete review history (after including the latest revlog entry from the other device).

[vaibhav]

@dae

Depending on the timing of a sync before/after optimizing, you can end up optimizing on multiple machines, or undoing the optimization.

One way to solve this is to make the user choose ONE device on which automatic optimization should occur.

[Expertium]

make the user choose ONE device on which automatic optimization should occur

No. Automatic optimization should be automatic, as in “literally no user intervention is ever required”. The goal is to make FSRS usable with zero actions from the user. Not two actions. Not one action. Zero. FSRS should be usable even if the user has never once touched deck options. Of course, that won’t happen until FSRS becomes enabled by default, but that itself won’t happen before automatic optimization.
So basically it should be like this: automatic optimization reduces the number of actions required to one (enabling FSRS), making FSRS the default algo reduces the number of actions required to zero. So we need automatic optimization first.

[Keks]

@Expertium

Of course, that won’t happen until FSRS becomes enabled by default, but that itself won’t happen before automatic optimization.

I have never seen anyone write that automatic optimization is required to enable FSRS by default.

[Expertium]

Optimization is necessary to find the best parameters for a given user, and if FSRS is ever going to become the default, then obviously it needs to optimize parameters automatically. I don’t know why anyone would think that FSRS can be the default algo without automatic optimization, that’s extremely user-unfriendly.
Remember that the vast majority of Anki users don’t read the manual or any guides. Anki (and FSRS) needs to work for them as well. Right now FSRS is great for the kind of person who doesn’t mind reading a 4000 words guide, not so great for everyone else.

[sorata]

I said that once because newer users might not know about optimisation to start with. Also asking them to optimise manually just makes it user unfriendly.

New users should be given more thought when talking of default features IMO.

[Keks]

@Expertium

I don’t know why anyone would think that FSRS can be the default algo without automatic optimization, that’s extremely user-unfriendly.

@sorata

I said that once because newer users might not know about optimisation to start with. Also asking them to optimise manually just makes it user unfriendly.

That is, in your opinion, SM-2 is preferable to FSRS without optimization?
As far as I know, FSRS has better default settings than SM-2. When a new user has to figure out the algorithm, it will be much easier to do it in FSRS because there is only one button for configuration.

[sorata]

That is my argument too. That FSRS is easier to use. But I believe a lot of users wouldn’t like the idea of doing “optimisation” on their settings every now and then. The want to get a setup and be done with it. +we have the “optimise all presets” not in the main screen but hidden in the deck options.

[Expertium]

That is, in your opinion, SM-2 is preferable to FSRS without optimization?

No. FSRS is better even with the default settings. But optimization is still very important. Without optimization, the user won’t get the full benefit of FSRS. And we need to ensure that not only very tech-y users get the full benefit, but users who never touch the settings as well.
Again, keep in mind that the majority of Anki users don’t put a lot of mental effort into learning about Anki, and don’t read the manual.

[Keks]

@sorata

The want to get a setup and be done with it. +we have the “optimise all presets” not in the main screen but hidden in the deck options.

Sounds like an idea for an addon. Once every N days, a button appears, when clicked, a window appears with a warning that your devices must be synchronized before optimization, the button performs full optimization and disappears again for N days.

[Expertium]

No. No add-ons. No stuff that requires extra mental effort to setup. FSRS needs to work out-of-the-box aka without the user doing anything. The more actions are needed, the fewer users will do all of them.

[Keks]

@Expertium

No. FSRS is better even with the default settings. But optimization is still very important. Without optimization, the user won’t get the full benefit of FSRS. And we need to ensure that not only very tech-y users get the full benefit, but users who never touch the settings as well.
Again, keep in mind that the majority of Anki users don’t put a lot of mental effort into learning about Anki, and don’t read the manual.

I do not argue with you that it is necessary to improve the user experience whenever possible. But I don’t understand why not do it right now by enabling FSRS by default.

[Expertium]

I’m not saying that FSRS has to become the default right now. I’m saying that automatic optimization is a necessary step towards that.
EDIT: I misunderstood you, sorry. It’s possible to make FSRS the default algorithm right now, but it’s not ready yet. FSRS Helper add-on features and stats aren’t implemented yet, plus FSRS-5 will come out in October (or earlier). Then there are the really distant plans for implementing a full simulator based on FSRS. So there is still work that needs to be done. But automatic optimization is the most important feature of all the features that are yet to be implemented. It will have the highest impact on non-tech-savvy users.

[sorata]

Actually even if we don’t get auto-optimisation I think such a time to time reminder might be enough for getting FSRS as the default scheduler.

By the bye, I don’t understand why isn’t it documented anywhere (I havent found) that even manual optimisation should be done only when all the devices are in sync? Is it a non issue when we’re not talking of auto optimisation?

[BrayanDSO]

One way to solve this is to make the user choose ONE device on which automatic optimization should occur.

Or run the automatic optimization if the user uses only one device and doesn’t sync at all (I know that this far from the ideal scenario, but it is closer to a MVP than the current status)

[sorata]

How is the “no sync” condition necessary then? I use one device and everything can just be simply uploaded to AnkiWeb without any conflict?

[BrayanDSO]

Another alternative is to only run the automatic optimization if the collection is synced.

Kind of

1. Sync
2. Check if it is time for the auto optimization and optimize if necessary
3. Sync the optimization changes

[vaibhav]

@BrayanDSO

Another alternative is to only run the automatic optimization if the collection is synced.

The problem is that you can’t know whether all the other devices are synced to AnkiWeb or not.

It seems to me that the following is the only way to deal with sync conflicts.

@vaibhav

After reading the discussion about the issue, I think there is no way to prevent the sync conflict. However, we can modify the sync code to change how Anki deals with the sync conflicts.

My idea:
If the same card was reviewed on one device and its memory states were changed on another device without any review (due to automatic optimization), then keep all the revlogs and recalculate the memory states from the new parameters using the complete review history (after including the latest revlog entry from the other device).

[jambamboleo]

@dae

That benchmark ignores the I/O costs of reading that data from disk. If the user’s disk isn’t able to respond to that request promptly, answering the card is going to block until it completes.

How long could that take? I feel like up to 500 ms on a card’s front side screen, for example, is fine, given that it will most likely be less than the time it takes the user to process the question, recall the answer and press the reveal button. In fact, there is an addon called No Space Dash: Prevents you from seeing answers without reading a question.

If it takes a second or more, then yes, it can be a noticeable annoyance to the user.

Please excuse me if I misunderstood the issue.