This issue is like a reminder for LMSherlock, it’s not exactly aimed towards Anki devs.
Some people use “Set Due Date” without being familiar with the card, while others use it once they have already reviewed the card outside of Anki. There is no way to differentiate between the two based purely on cold, hard data. But we can try different approaches to see which one results in better log loss/RMSE, on average. Of course, we won’t be able to find a solution that works amazingly well for everyone, only one that works ok on average, but it’s better than using a solution that works poorly on average.
@L.M.Sherlock take the Anki 20k dataset and put all users who have at least one rating=0 review in a new dataset, we’ll use that for benchmarking.
Following on your discussion thread from Discord –
a couple of things I didn’t see you mention about “Set Due Date” –
-
Using “Set Due Date” on a New card also sets the interval to that number of days. Some users know about that and do it deliberately – other users are surprised to find it out. It sounds like FSRS disregards that interval and always starts over from 0 because there was no “review” that caused it. I can see why that’s necessary for the 2nd group, but I’m not sure it fits the needs of the 1st group.
-
Currently, both Anki and AnkiDroid use the same “Set Due Date” functionality. But before 2.18 (which was just released this week), AnkiDroid had a different functionality instead, called “Reschedule” [probably for the best to have moved away from using that name for it 
]. If you’re trying to make something that fits past revlogs, it might be worth checking what an initial AnkiDroid-“Reschedule” revlog entry looked like compared to an initial Anki-“Set Due Date.”
1 Like
Regarding 1, that’s the issue: we don’t know whether users do that while knowing the card already or no. My idea is to try different assumptions and formulas, and see what results in better metrics on average.