I am saying that adding a flag is not necessary. Anki can internally convert a query like is:leech into something like prop:leech-p<0.05.
“Hey guys, we made a leech detector that doesn’t mark cards as leeches”
You are misunderstanding. From the user’s perspective, nothing would change. I am only proposing to change what Anki does internally.
But you’re saying we shouldn’t use flags/tags
Yes, but how does a user know if a card has a flag or tag? They don’t open up the database file in a SQL viewer. They see what the UI tells them. The UI can read the p value from the data column.
So the UI would display a pop-up based on the p-value stored in card info, and when searching prop:is_leech, Anki would convert it to prop:p<0.01, correct?
Actually, no, that still won’t work. In order to reduce the amount of time the leech status changes, in my specification of the detector I wrote that we should use two thresholds. If p<threshhold_1, card’s status is changed to “leech”. if p>threshold_2, card’s status is changed to “not a leech”. The second threshold can be 15% or 20% or something like that. Otherwise, some cards can jump around a lot. This means that we need to keep track of how p changes after every review.
For example, if the first threshold is 5% and the second one is 25%, and p(normal) is 10%, whether it’s a leech or not depends on whether it has crossed the first threshold before or not. If it has crossed it before, it’s a leech, otherwise it’s not a leech.
As an alternative solution, we could use one threshold but update the leech status once every 3 reviews. But that is also incompatible with your method of not using a tag/flag, since it requires keeping track of how many reviews ago the leech status changed.
why is this a problem?
If a user sees that a card has been going “leech” → “not a leech” → “leech” → “not a leech”, he will think that the detector is bugged or not working very well.
And also it can be just plain annoying.
2 Likes
Are there actual cases where normal anki usage would lead to it continually bouncing between leech and not-leech?
also I figured leeches would have notifications, but unleeching would happen silently.
I was thinking of having a pop-up for both leeching and unleeching.
As for bouncing: Discord
Btw, based on this chart, I’d say the idea of having two thresholds seems reasonable.
By the way, is this really a leech detector? In this method, we are just looking for cards where FSRS’ predictions are off. But, even if FSRS is predicting R accurately for a card, the card can still be a leech if it requires a very large number of reviews.
Consider a card whose S grows extremely slowly. The p value won’t be low because FSRS is correctly predicting the R, but this card sucks up a lot of time that can be better spent elsewhere. So, this card is a leech even if this automated detector won’t detect it.
Example card:
I have done 68 reviews but the card still has a stability of just 30 days (and the last lapse was in November).
The p-value for this card is 0.21 (calculated using GitHub - rbrownwsws/leechkit with default values of --skip-reviews and --max-reviews)
Example 2:
p-value = 0.29
Example 3:
p-value = 0.43
Example 4:
p-value = 0.43
Example 5:
p-value = 0.44
As a counter-example, here is an anti-leech:
That being said, this method can be useful to improve FSRS by identifying the type of cards that FSRS struggles with.
1 Like
Yes, “leech” here means “a card for which FSRS is overestimating p(recall)”. It’s possible that FSRS predicts p(recall) accurately, so it’s not a leech in that sense, but the card feels “leechy” subjectively.
That being said, I’m sure if you choose a threshold like 1%, 2% or 5%, the cards that fall below this threshold will most likely feel “leechy”.
We really need a proper implementation in Anki or in an add-on to collect feedback.
2 Likes
I am sorry for bothering you. I don’t understand most of it.
But when that smart leech detector would appear in Anki? At least approximately?
And how it will improve FSRS? Like how significant it would be for a user on a daily basis? Will it be noticeable or nothing will be changed for a user?
1 Like
I don’t think anyone is working on this actively. And any approximation of when it’s happening is meaningless IMO. It seemed easy until it wasn’t until it was then again until it wasn’t.
It won’t improve FSRS but it’ll hopefully detect time-consuming cards better which you can suspend or edit or come up with mnemonics for. FYI, leech detection is referring to how Anki defines and tags cards as leech. Currently, we just look at again ratings which isn’t very good.
5 Likes