Desired Retention UI Overhaul

(partially backed up by @David. We don’t agree on the details, but we agree that something like this is necessary)

Right now a lot of users don’t realize that desired retention is related to interval lengths and workload. I could write a list of 100+ posts where someone has DR at 90% and asks “Why are my intervals so long? What do I do?”.

My initial idea was like this:

But David thinks that’s probably too much information, and that some people may still end up being confused. So here’s a more condensed way to present the same information + a hint about interval lengths.

Question 1: Will the interval hints always be there? Can’t we disable them?
Answer 1: No. Disabling them defeats the whole purpose, which is making it obvious what desired retention does.

Question 2: It’s obvious to me!
Answer 2: It’s not obvious to hundreds of other users.

Question 3: Is the slider necessary?
Answer 3: Yes, otherwise interval length hints will look out of place. Where would they even be then?

Question 4: Do you see any way to implement this overhaul without giant interval hints that cannot be turned off? Like, any at all–
Answer 4: Absolutely not. I understand that this will upset power users, but that’s a necessary evil.


The user needs to think whether he understands FSRS enough to mess with it? Then it doesn’t solve the problem.

The user needs to think what desired retention does before using your solution? Then it doesn’t solve the problem.

The user needs to think? Then it doesn’t solve the problem.

The point is NOT to add more customization. The point is to throw “DESIRED RETENTION AFFECTS INTERVALS AND WORKLOAD” in the user’s face so hard that he won’t be able to miss it.

Question 5: So we would be able to use both a slider and an input field for DR?
Answer 5: Yes. If you use the input field to change the value, the slider should instantly change as well.

Question 6: When will the workload value be recalculated?
Answer 6: Ok, now we are getting to the important part! I have no way of measuring how long the simulations take, at least not with the kind of millisecond precision that I need. That would require scavenging that particular part of the Rust code and running it manually. So depending on how long it takes to run simulations for 29 values of desired retention (as for the duration, 90 days is reasonable, IMO), we have 2 options.

  1. If it takes <40 milliseconds (on a low-end or mobile device): just re-run simulations every time the user changes max. interval, learning steps, relearning steps, new card limit, review limit, Easy Days, or anything else that affects the simulations. The amount of lag won’t affect user experience much. This would be ideal. We could even reduce the duration of the simulation to 30 days, if that’s what it takes to get under 40 ms. As to why 40 ms: that’s a good baseline for “amount of lag that isn’t annoying”.

  2. If it takes >=40 milliseconds: then we have no choice but to recalculate them only when FSRS parameters change. In other words, after each optimization. That would be very unfortunate, since that would mean that the user won’t immediately see how changing settings affects the workload, and likely won’t realize that the values only change if parameters are recalculated.

Time per simulation is the main factor that will decide whether this overhaul is a good idea or not.

I tried a very crude estimate - run the simulations in Python, calculate time and divide it by 50 because Rust is 50 times faster than Python*. But I ended up with a completely unrealistic estimate: for 29 values of DR and 90 simulated days, it would take 8 seconds. I am willing to bet it’s much faster than that in reality, hopefully Jarrett can estimate it properly.

*source: the universe has revealed it to me

@dae @L.M.Sherlock

1 Like

You’re in luck, I prototyped this already.

As you say, for each data-point, you have to run an entire simulation. That is massively unresponsive, even if as you suggest you recalculate after every optimisation. Here’s how it runs with 1 data-point per percent on my actual computer. (It takes 40 seconds)

3 Likes

No. Come on man, do you think an average user is going to click that and mess around with it? We need something that requires zero actions from the user. Anything more than 0 is not acceptable.

The user needs to think whether he understands FSRS enough to mess with it? Then it doesn’t solve the problem.

The user needs to think what desired retention does before using your solution? Then it doesn’t solve the problem.

The user needs to think? Then it doesn’t solve the problem. We are trying to make a solution for people who don’t understand that desired retention and workload (and intervals) are related in the first place.

The point is NOT to add more customization. The point is to throw “DESIRED RETENTION AFFECTS INTERVALS AND WORKLOAD” in the user’s face so hard that he won’t be able to miss it.

I don’t see “Days to simualte” in your video, btw. I guess you are using some unreleased version and Days to simulate is higher, and it got cut off.

Yep, I just copied the simulator modal for this prototype.

The 40 second wait time, or even just the simulator wait time is in my opinion too long to be feasible to do anything “magically”.

The 40 second wait time, or even just the simulator wait time is in my opinion too long to be feasible to do anything “magically”.

Yeah, that’s a big problem. I was just talking about how having a separate option/menu/whatever is not a good idea.

1 Like

Nothing more than UI clutter. It is already stated in the docs that DR affects intervals.
One way to do something without cluttering the UI is to create a new space on the open-spaced-repetition hugging face page that takes the user’s weights and outputs the DR curve.

3 Likes

I tend to agree somewhat, though I think a UI overhaul could be a good idea if it’s implemented right. I do wonder though if it’s really necessary.

If you click on the “Desired Retention” string it already shows you the explaination and relationship between DR and workload + intervalls. So it’s not just in the online docs, but in the in-app docs as well.

This can be accessed from the prominent question mark icon as well.

At least some thinking is neccessary – for basically everything in the world. Maybe a proper breakdown of the problem would be appropriate, to understand why those users are having issues.

To me it is pretty much straight forward (though I don’t mind thinking…):

  1. I want to change a few settings.
  2. I encounter one which I don’t understand.
  3. I read the in-app doc (either by clicking the string or the question mark icon. As a sidenote: it’s pretty common to have explainations shown onHover – if you hover over the string in anki it gets underlined and the pointer shows a question mark symbol, too. Pretty hard to miss in my opinion)
  4. If I now understand the setting, I configure the setting as I want – else I look online (a user could just search something like “what does the desired retention setting do in anki”, which returns Deck Options - Anki Manual as one of the first results).

In summary: It is neccessary to understand why those certain users are having difficulties – just changing stuff without knowing their problems isn’t likely going to change anything.

2 Likes

Does it consider existing cards and all settings from the preset?

1 Like

Just in case you’re asking about my implementation then yes.

Most users don’t do that.

It should, yes. Can you test how long it takes to run 29 simulations (for each value of DR) with duration=90 days? Idk, maybe a_blokee somehow screwed it up, though it’s unlikely.

I made a survey and posted it on r/Anki: Anki "on hover" tips

I am very surprised by the results. I’ve seen tons of people who don’t know about “on hover” tips, so idk where they are.

The sample size is small, but still

EDIT: so far seems like around 25-30% of users don’t know about “on hover” hints.

So my take is that if something seems “obvious”, there will be a lot of users who don’t know about it. If something seems “not very obvious”, likely >50% of users won’t know about it.

EDIT 2: less than 25%, it seems

2 Likes

But how to make it more obvious (but at the same time consistent with the rest of the ui)? The only thing I can think of is adding the string “help” next to the question mark icon:


(Note: this screenshot was taking from version 24.11, not the latest build. It just serves as a visualization.)

That would be too minor. We need a major overhaul like what I proposed above, like a slider with two hints on the sides.

As for the workload, I asked Jarrett to double-check how long the simulations take (in Rust).

1 Like

Alright.

I don’t think it’s better, though. It’s not immediately obvious that the “Desired Retention” setting and the slider are related – and if you were to change the DR percentage, you’d get an info message anyways, telling you how intervalls are affected. So you’re proposed change doesn’t really offer value here.

The “Anki will take X minutes per day” is probably poorly worded (you’re probably talking about reviews), but it could indeed be a nice addition.

If I had to choose between the slider thingy you suggested and the graph thingy, I’d choose the graph.

This one

Maybe someone with great ui skills finds a great way to portray this better.

1 Like

Time is more intuitive. “I have to do 85 reviews today” isn’t quite as intuitively clear as “I have to do 10 minutes of studying today”. And since we can estimate it with the simulator, why not?
(aside from the problem of the simulator being slow)

As for the graph, according to David, some people have a hard time reading graphs, so it’s better to use a bar with a little text on top of it.

I mean as in “Anki will take X minutes per day” should be “Reviews will take X minutes per day”

A lot of them. When I studied psychology there were at least 30% that still struggled with graphs after we learned – and were exposed to – a variety of different graphs.

The bar thingy still is worse in my opinion – by a huge margin even.

1 Like

Man, there’s gotta be something wrong.
I did it manually and it took less time! I have no clue how it took you 48 seconds. Maybe some post-25.02 change made it slower, idk.

Based on counting frames with “Processing…” in them, which is usually 2 frames, I estimate it takes about 2/60=0.033 seconds=3.33 milliseconds per simulation, around a second for 29 simulations. What unspeakable horrors did you and Jarrett do to the poor simulator after 25.02? :laughing:

EDIT: ok, this was a bad example because this preset doesn’t have a lot of cards. I tried it on another preset that is applied to more cards and got around 4 frames with “Processing…” on them, so 4/60=0.066 seconds=6.66 milliseconds per simulation. Still roughly 2 seconds for 29 simulations.

Like, ok, let’s be turbo-pessimistic and say the frame before “Processing…” appears counts and the frame after it disappears also counts. That’s 6/60=0.1 s per simulation, or 2.9 seconds for 29 simulations. Still not even close to 48 seconds, not even within the same order of magnitude. And I doubt that your PC is 15-25 times slower than mine.

(everything was recorded at 60 FPS)

Your computer must just be built different :man_shrugging:
I added a quick progress bar.

I did originally run it with 1 year of days to simulate, here’s how it performs with 90.

Based on what I can see in your video, I’m running it with 4-5* more cards than you.

hang on “/31” seems weird.
(Edit: I accidentally simulated DR=100% so ignrore the last % for timings :sweat_smile:)

I did originally run it with 1 year of days to simulate, here’s how it performs with 90.
Based on what I can see in your video, I’m running it with 4-5* more cards than you.

Hm, maybe.

There is an alternative - instead of calculating all 29 workload values and storing them we can just re-calculate the workload value when the user adjusts desired retention, so that we only calculate it for the current retention. But that would cause lag when the user is tinkering with desired retention.

I guess for now using simulations isn’t a good idea. I still want to do something about the second half of the overhaul - making it clearer that DR affects interval lengths. Especially at DR=90%, where the “a 100 day interval will become X days” thingy doesn’t show up (and a lot of users find it unintuitive anyway).

I really prefer having the graph:

Maybe we can simplify it a bit?
Things to consider:

  1. x-axis should be “Desired Retention (%)” and should be in % as well (“0.90 → 90”). This makes it consistent with the latest anki version, which shows a percent for DR as well:
    anki
  2. y-axis should be “Daily Study Time (minutes)”. That is probably less confusing than the current title.
  3. A very short description somewhere like “The higher your Desired Retention, the higher is your Daily Study Time.”. That way people won’t have to understand the graph fully and can use the concise written info instead, while still having the visualization that that graph provides.
  4. Maybe remove the “minimum recommended retention” part, unless the graph is actually based on the users review history.

I have no idea how you colored your graph, so I summoned my inner artist and just drew over it using a free hand filling tool… But you might get the idea anyways:

(though I’m wondering whether the coloring would need to be explained as well. Maybe in a popup?)