[23.10 Desktop] Add a feature to automatically periodically re-optimize FSRS parameters

Okay, thank you for taking the time to explain the issues.

On another point, regarding your reply to zZH2z’s comment, I think that his idea is actually quite reasonable, as we should expect people can make a sufficiently informed decision for that. For example, a warning next to the button to enable it PLUS a pop-up that provides another warning should be sufficient. I think at that point, if they enable it and it messes something up, they can realize their mistake to avoid it in the future (similar to a mistake of not syncing cards on other devices, before manually clicking it).

I’m curious if say they miss a sync on one device B, and the auto-FSRS syncs to all cards on device A, if there’s a way to program it so that once they sync on device A, it retroactively updates the cards to how they should have been done. Perhaps, something that gets it to approximately close enough might do.

So for example, I imagine that if they miss a sync on device B, then they’re likely using a different deck A on device A, because otherwise they would just realize they’re redoing the exact same cards from deck B that they had just completed on device B in the exact same order. And so, perhaps, in that case, the FSRS would just “take into account” the cards from deck B that had been previously completed but not synced previously. It wouldn’t be too different than say if they had completed the decks in the reverse order, say deck A first, sync, then deck B, sync. Perhaps (I’m not sure), the overall stats on the FSRS might be ever so slightly different in that case, but I don’t think it would be overall super significant since: (i) the sync mistakes are more likely than not to be an occasional exception rather than the rule, and (ii) FSRS operates within a margin of error (as per the Log loss and RMSE(bins) values presumable).

Perhaps, another potential option is simply to have the Anki app to check other devices to see if they are not synced, prior to auto-updating the FSRS.
This might be a desirable option regardless, as it would prevent manual FSRS sync conflicts from occurring as well.

I’ll be honest, I’m not interested in rehashing the points I’ve already made (and that Damien and others have made up-thread).

You imagine a very narrow band of use, and certain functionality that wouldn’t be possible without fundamentally re-designing/re-structuring how Anki syncs data. If you want to work on solutions to this, you need to learn more about how users really use Anki, and about what Anki is capable of as a cross-platform, offline-first program.

Alright, thank your for your time

I think that we can simplify this suggestion. We don’t really need to check whether any review was done or not.

Simplified version of the suggestion:

During the sync, if there is a sync conflict in a card, recalculate the values of memory states, desired retention and decay for that particular card (after the sync of all the other data such as revlogs, deck configs, etc. is completed).

Rationale:
Sync conflict in a card can be caused by doing one or more of the following on two or more devices:

  • changing decks (which changes did)
  • reviewing
  • rescheduling
  • optimizing

All of these except rescheduling affect the memory states, desired retention or decay. So, it is better to recalculate these values.

Relevant code: add_or_update_card_if_newer in rslib/src/sync/collection/chunks.rs

Possible implementation:
Modify the add_or_update_card_if_newer function to return a list of cids where a conflict was detected and recompute the values in those cards after all the other parts of the collection are synced.

@jakep is working on a solution (well, getting started). I suggest asking him

please don’t set people up for disappointment

Keeping track of what causes a sync conflict is such a good starting point for getting past this roadblock! It’s basically anything that causes an update to the record in the Cards table (and with it, the “Card Modified” timestamp mod), right?

I don’t know if it’s important to have a thorough list, but if so, I’ll add:

  • Bury/unbury, suspend/unsuspend, flag/unflag
  • Filtered deck build/empty (which does not change did)
  • Set Due Date (or is that included in “rescheduling”?)
  • Reset
  • Reposition
  • Changing DR for the preset

[I suppose if a user still has SM-2-releated add-ons bumbling around behind the scenes updating “Ease” automatically (after sync, on-close, etc.), that could do it too.]

Rescheduling does affect the memory state if done through the Helper add-on in the Browse window – “FSRS: Update memory state and reschedule”.

⚠️ scope creep ahead

I know this idea is only meant to address the simple case – change on one device, automatic optimization on another device – but if someone is implementing something to handle that, it would be nice if it would also cure the biggest bad of sync conflicts – mis-scheduled (un-studied, un-“done”) cards.

The only (partial) cure I’ve found for that is a Browse > select > Cards > FSRS: Update memory state and reschedule. Which leads me to tentatively wonder aloud – should we also reschedule cards when a conflict is detected and the memory state is recalculated anyway? But that would obviously a big change, and we should generally avoid rescheduling cards out of view/with no user interaction …

I don’t know how I missed these. Given that we have so many potential sources of sync conflicts, my suggestion in [23.10 Desktop] Add a feature to automatically periodically re-optimize FSRS parameters - #107 by vaibhav shouldn’t be implemented as it is, because it would lead to unnecessary recalculations.

To prevent the unnecessary recalculations, we need to have separate usn/mod time for the memory states.

Relevant: Feat/infer memory state in batch by L-M-Sherlock · Pull Request #362 · open-spaced-repetition/fsrs-rs · GitHub
Jarrett (and other people) is working to speed up calculation of cards’ memory states (Difficulty, Stability, Retrievability), which would make recalculating memory states after every sync practical.
If anyone is good at Rust, feel free to help Jarrett.

EDIT: some gifs from Jarrett
Anki 25.09

before

After these changes

after

This is only a 10x speedup though :pensive_face:
So let’s wait for Jarrett and others to do some more magic