Currently, when using the simulator, only one graph absed on oen simulation is plotted. However, I think it would be beneficial to use 5-10 simulations and make the final graph an average of those, plus confidence add intervals.
Kinda like this:
Use different seeds for each simulation.
This also means that the simulation will be 5-10 times slower, however. I think we should try it first and see how bad the slowdown is. @sorata pinging because this was originally your idea @L.M.Sherlock pinging because you are The FSRS-Man @A_Blokee pinging because you are the only one who knows how to plot stuff, or at least the only one who puts that knowledge to use for the sake of Anki
EDIT: you can kind of do that by running several simulations with a different value of “Days to simulate”
Yeah, but I wonder if there might be something I’m missing. I still would like to see a proper graph where everything is the same, only the seed is different.
If the variance is small even then, then we can ditch this idea.
Well, at least you got the idea out from the deep abyss of discord. The concerns you had are still valid, no way to do this without making the compute time longer. Or we only do this conditionally in some cases thereby confusing users.
I don’t know how to build from source code
Also, I’ve tried this (using different “Days to simulate”) on another preset, and it seems like there’s more variance
Maybe a checkbox saying something like “Compute confidence intervals” could be helpful? Or maybe something like “Compute confidence intervals (might take a long time)” to emphasize that it might take longer?
Yeah, but there are two many options already and this one wouldn’t even make sense to a normal human. I bet you’ll see a lot of questions on the forums about it.
Ah wait, @rossgb do you remember that issue with number of reviews you have for a particular day changing when you change days to simulate? Isn’t that it? I guess I solved the issue? (Or all of you knew it before)?
This might need some work otherwise we’re gonna get bug reports from confused users who’ve got no idea what’s happening.
The above downloads the repo from Luc-Mcgrady, but only the branch random-simulator-seed, which is downloaded into the folder random-simulator-seed-folder.
cd random-simulator-seed-folder/
The above enters the folder.
./run
This command builds the entire branch and then opens the newly compiled anki. It might take quite a long time depending on your hardware.
I’m confused. I’m not sure what you are talking about.
It turned out that the issue I had earlier was because Jarrett was smoothing the data with a moving average. I don’t see how this applies to what we are discussing now.
That’s the other one where tooltip data was one day off. You also had the data for same day appearing different when you put days to simulate 30 versus 60. And I don’t remember Jarrett talking about that.
That is not clear to an average user (how many people know what’s a “confidence interval”?), and splitting the UI into Beginner/Advanced mode is something I’ve proposed a long time ago and the responses (especially from Dae) were lukewarm at best.
