Suggestion: Lapse to Review Ratio

Anki Add-ons
1

Lapses pile up after a long time for each card regardless of its ease.
For example. every once in a while I would have a brain fart and lapse in recalling an easy card. The lapse counter, as Anki does it, just piles up cumulatively and in turn would have sooner or later the same amount of lapses as a hard card.

What differentiates between the two, is how many reviews it takes for it to reach a certain lapse count.

For example (this is only hypothetical and completely arbitrary), to reach a lapse count of 20:

A hard card requires only 30 reviews to do so (Ratio 2:3)
Whereas an easy card requires 100 reviews (Ratio 1:5)

This is a much better indicator of how much of a leech card a card is and I figure this could be done through an add-on, with a color indicator, and suggestions for actions to be taken on a card.

So where is the threshhold, I dont know, I cant think of a threshhold which would determine that a card is wasting you too much time unless someone has an idea.

I only know of @Shigeyuki who does add-ons here.

4 Likes
2

I find it amusing how similar our thinking can be. You saw me asking whether we can use the FSRS parameters for leech threshold. Then I saw someone else talking about it in Anki discord.

Well FSRS parameters can not be used as we learned.

Anyways I was thinking about a similar problem tomorrow. The one you mentioned. Yes I was thinking about it in terms of Lapse: Review ratio sort of thing. But a better idea Danika proposed is to just make “lapses” expire after a user-defined period. I think this is a better way to prevent lapses piling up and it solves the problem. Review: Lapse is kind of unintuitive. Like do you actually have an idea of what would make a good ratio? I know I don’t. Data can solve the problem; but also the ratio would depend on the time intervals between reviews and that is something different for different target retentions.

1 Like
3

Yeah I guess the ratio will have to be arbitrary.

Like do you actually have an idea of what would make a good ratio?

The greater the ratio, as demonstrated above, the better.
Whether the ratio is optimal depends on several factors like:

-number of relearning steps
-date first learned (the ratio could change from lower to higher until it stabilizes after some time)

You dont want Lapsed cards to expire because that doesnt solve the problem. You would still have to learn those (I am a Med student, there is only so much things where I could have the freedom of just ignoring them).

+What I would want is some sort of in-reviewer indicator (green or red or yellow) to show that this card that you are learning is harder than normal, based on a lapse:review ratio (with your normal being a desired ratio set by you).

+This only should come into play after a certain number of days has passed. (7 days for example, also set by the user).

+If the card is still harder than normal afterwards, the whole card should be deleted or completely reformulated or a wide range of mnemonics should be introduced by the user. Perhaps also suggested by the add-on.

Shige, if you find this idea interesting, could you please perhaps turn this into an add-on? :smile:

5

Perhaps you mentioned this my add-on, counting the number of lapses today for each card and show it in the menubar.

image

My next development idea for this add-on is to auto-bury after a certain number of lapses.

For example, if the user makes a mistake 10 or 20 times, it will be auto-bury to the next day.
Or pop up a dialog asking the user to choose the action [1]Again, [2]Bury, [3]Supend, [4]Edit.


But all these functions use data within today’s range, not past data.

For mature cards, I previously created a prototype add-on that displays Ease percentages(EaseScouter, not yet released to AnkiWeb).

Text color changes depending on the card’s difficulty status, like this.

image
image1633×450 64.1 KB

My next development plan for this add-on is to try to get the FSRS difficulty etc., in some way.

Perhaps it is possible to create Lapse and Review ratios as you mentioned, but calculating the numbers from a database becomes more difficult and fragile.
As sorata mentioned, it seems easier to use existing difficulty levels, perhaps the objective is much the same.

2 Likes
6

Assuming there are dozens of cards with revlogs, you can label them with leech or non-leech. Then we can train a classifier based on machine learning from your labeled cards’ revlogs.

3 Likes
7

As sorata mentioned, it seems easier to use existing difficulty levels, perhaps the objective is much the same.

If you think it does the same thing, sure why not. :sweat_smile:, though I would think a lapse to review ratio is much more intuitive. If it makes your add-on more easier to make, who am I to complain.

I will be looking forward to your works.

8

This would be revolutionary as it would save users like me so much time. Is there a chance this gets incorporated into Anki, because the Leech feature as it is right now, feels so arbitrary and therefore once could never know whether the leech threshhold is set too far high or set too far low.

Or rather where could I find this machine outside of Anki? (assuming that I know jack about model training) :sweat_smile:

One problem though (and this is why I dislike leech a bit): All my cards are cloze cards. A leech tag would apply to the whole note. :expressionless:

1 Like
9

One more thing, perhaps one of the actions that you could incorporate for leech cards is that leech cards could perhaps be sifted and moved into another deck automatically. Just a thought. Would be a really helpful feature though.

A flag and not a tag (as tags apply to the whole note including all of its cards) for leech cards would be nice as well!

10

Hmmm, maybe you and I have different ways of doing Anki reviews.
In my case, it leech when lapse exceeds 7, so all Leech tagged cards are at max difficulty.(In my case, they all need to be edited or given up to suspend.)
20 or 100 Review cards should already be a few years away.
So I probably don’t have the ratio of Lapse to Review.(or maybe I’m misunderstanding it.)

The only warning I need is when I press Again on a Leech tagged card. That is already wrong a lot, so I am most likely wasting my time. When I don’t press Again on a Leech tag card, it doesn’t warn me about anything, because I have nothing to do.

11

20 or 100 Review Cards was just an example :sob:. Just wanted to hint at one of the functionalities that you could add, like moving leech cards to some deck made for leeches if that is not too hard.

2 Likes
12

I think there is already an add-on that auto moves leech tag cards to another deck, Anki developer Damien created it for examples of how to write add-ons.

Edit : Maybe this.
ANTI-SUCK: automatically move leeches to a separate deck

Edit2 : The author of the add-on sample for moving leech card to the deck was the same as the author of the Leech toolkit add-on.

1 Like
13

But leech tags dont work well with cloze notes, it marks the whole thing as leech. :face_with_peeking_eye:

Ah forget this functionality, just having the difficulty-lapse to review ratio with the indicator thingy in your add on would be nice

14

Always happy to have credit where credit is due – but this was not an idea Danika proposed. I merely recognized it as a more logical conclusion.

1 Like
15

The Leech card has the lowest Ease (130%), so all the colors are the same (all purple!). Probably FSRS difficulty is also the same.

In this case, it is not possible to track cards repeatedly Leech, so as you said, it may be necessary to set a unique difficulty based on the ratio of Lapse and Review.
( In short, difficulty factor for Leech cards only).

If you want to set Leech only for cards, it may be necessary to create a dedicated flag. (For example, to auto set a flag of a specified color when becoming Leech). But I think it would be a distraction if the Leech flag is displayed on the Front, so I think it would be better to hide it.

1 Like
16

In this case, it is not possible to track cards repeatedly Leech, so as you said, it may be necessary to set a unique difficulty based on the ratio of Lapse and Review. ( In short, difficulty factor for Leech cards only)

Yes please!

If you want to set Leech only for cards, it may be necessary to create a dedicated flag. (For example, to auto set a flag of a specified color when becoming Leech). But I think it would be a distraction if the Leech flag is displayed on the Front, so I think it would be better to hide it.

You are right. Would be better to have both options though, as I can think of scenarios where I wouldnt want to press spacebar to reveal the backside in order to see if it is leech or not.

Like a check box whether back-only, or front and back would do.

17

Also here is a simple use case for why this could be great for time allocation @sorata

image
image1304×513 10.7 KB

I hope you could understand it

2 Likes
18

@DerIshmaelite I’m gonna sleep now it’s already 5 am but before that have a look at the code sherlock shared. This is my invite link: Anki

Well it’s the Anki server, sherlock shared it in #general today. He called it Lapse rating(?). It takes time into account so things get harder or easier with time. I mean to me consuming. pinging @Shigeyuki too.

1 Like
19

I saw this code in the Discord server. This you mean?
Could @L.M.Sherlock please explain this :sweat_smile: and how we can we use this?

>>> import numpy as np
>>>
>>> # sigmoid function
>>> def sigmoid(x):
...     return 1 / (1 + np.exp(-x))
...
>>> def leech_score(r_history):
...     score = -12
...     decay = 0.8325
...     threshold = 0.65
...     rating_to_score = {
...         "good": -4.0424,
...         "hard": 5.9002,
...         "again": 9.6398,
...         "easy": -25
...     }
...     for rating in r_history:
...         score = score * decay + rating_to_score[rating]
...     return sigmoid(score) > threshold, score
...
>>> leech_score(["again"])
(False, -0.35020000000000095)
>>> leech_score(["again","again"])
(True, 9.348258499999998)
>>> leech_score(["again","again","good"])
(True, 3.740025201249999)
>>> leech_score(["again","again","good","good"])
(False, -0.9288290199593754)
1 Like
22

@shigeyuki

I just want to sum up everything because this could get very confusing very quickly:

  1. the add-on should be able to calculate the lapse to review the count ratio

  2. Preferably also the average lapse to review ratio to be used as a reference by the user

  3. Allows the user to set their own preferred lapse to review ratio

  4. The add-on should use this set preferred ratio and compare it to the current one and should display an indicator on the back side (or front side of the card or both) that it is either a leech, borderline leech, or not.

  5. It would be nice if the add-on also marks the leech cards with some sort of a flag so that they are searchable in the browser. I say flag because tags are horrible with notes, it tags ALL of its cards.

  6. Could also have some set automated functions like moving the cards to a certain deck so that they could be learned with a much tougher preset (I suppose this is too much, just making them searchable in the browser so that they could be filtered out and be put into an another deck would suffice, I would say)

  7. Multiple thresholds can be used for different timeframes. (for example within the first few days a threshold of 1:3 because the number of reviews should be good, whereas within the next 18 days a threshold of 1:6 should be good)
    If this is too much, stick with just having one threshold @Shigeyuki :sob:

Also. Sherlock just posted this python code in the Discord server. Perhaps you could turn this into an add-on as well Shige, all credits of course to Sherlock.

>>> import numpy as np
>>>
>>> # sigmoid function
>>> def sigmoid(x):
...     return 1 / (1 + np.exp(-x))
...
>>> def leech_score(r_history):
...     score = -12
...     decay = 0.8325
...     threshold = 0.65
...     rating_to_score = {
...         "good": -4.0424,
...         "hard": 5.9002,
...         "again": 9.6398,
...         "easy": -25
...     }
...     for rating in r_history:
...         score = score * decay + rating_to_score[rating]
...     return sigmoid(score) > threshold, score
...
>>> leech_score(["again"])
(False, -0.35020000000000095)
>>> leech_score(["again","again"])
(True, 9.348258499999998)
>>> leech_score(["again","again","good"])
(True, 3.740025201249999)
>>> leech_score(["again","again","good","good"])
(False, -0.9288290199593754)

image

1 Like
23

I think 4, 5, and 6 would be an add-on to manage Leech.

There is an add-on that does this, the Leech Toolkit, but it was broken so I created a simple Fork and uploaded it to AnkiWeb.
(I didn’t add any functionality since I just fixed it.)

These features include.

  • Flags, deck moves, leech marked.

Flags are not transparent, but can be set automatically.

  • Code by L.M.Sherlock

Very nice, I think it can be used for the core part of calculations, it is also customizable. I will try to create some prototype later.

4 Likes