How to prevent users from misusing Hard? Ideas are welcome

Or very plain and simple, a pop-up : “Cautious : Hard is meant only to mark as successful, but you’d like the system to know it was harder than the expected”, with a nice “Do not show this pop-up anymore” below it.

I’m a backend guy, sorry.

1 Like

IMO something needs to be done asap to address this.

Something which doesn’t seem very hard to change nor controversial is just changing the button outline and/or text colors. I think this would have the biggest impact by far.

Red for Again and Green for the rest, like in the original post.

This makes it very clear that red = failed to remember and green = you remembered the card.

Other things could be changed later like the spacing of the buttons, tooltips etc. Just getting this merged would be huge and a great start.

2 Likes

If it weren’t for some people’s near pathological aversion to having anything in their lives be labelled a “fail”, this would’ve been easy enough to solve: Fail (or Bad*), Pass, Good, Easy.

Most people of sound mind won’t even be looking at the buttons after the first few hundred repetitions, so whatever emotional impact the labels might have will pretty much be irrelevant in any case.

(Or, at least, that’s how I feel it should be. You could argue that the intervals being displayed might still be of interest to the user. But in my opinion, they should be hidden by default. With FSRS, the interval numbers add nothing of value, aside from helping with debugging, which normal users shouldn’t be doing out of the gate anyway. People are inherently terrible at judging how long they’ll remember something, and so the numbers only end up causing fear and uncertainty once the intervals suddenly start to feel “too long” even though they aren’t.)

Apologies for my harsh tone. It’s just that I feel like this is all a bit silly. All this, and it feels like it’s really just to avoid using the word “Fail”, and spare the feelings of a few timid people. (Okay, I admit, users shouldn’t be excluded just because of their emotional idiosyncrasies, but still! :expressionless:)

*borrowed from SuperMemo, which has several failing grades, the least severe of which is termed “Fail”. It’s followed by “Bad”, and lastly, the dreaded “Null”, which is for when you don’t even remember having ever seen the question or the answer before.

3 Likes

The intervals on answer buttons show people what Anki is doing when they’re new to it. Not everyone knows Anki is SRS and how it works.

I keep going in circles with this conversation. I would just trust the main dev’s instinct on what should be the priority for this project.

4 Likes

Nope, that’s controversial, too

Increasing the distance between Again and Hard? People with OCD and people who like symmetry will hate it (also doesn’t work very well on mobile).

Giving Again a red outline and giving other buttons a green outline? Colorblind people will hate it.

There is nothing (that I can think of) that makes everyone happy.

4 Likes

I think this problem is urgent for FSRS but not for SM2. Anki’s default SM2 does not do advanced calculations so Hard can be used in replace of Again, which means that the problem of misuse of Hard only occurs when FSRS is enabled. SM2 has been working that way for over 18 years so I think there is no reason to rush into it now.

And even if not, Anki’s reviewer is in the process of major changes (Svelte), so it is reasonable to start features like the reviewer buttons after they are completed. If they work quickly they will probably have to develop the same feature twice (Now Reviewer + New Reviewer), so development resources will be wasted.

According to gamification research punishment does not reinforce user behavior, also mainstream research shows that pedagogy through punishment has many negative effects, so I think Again is more likely to be effective in helping users learn than Fail which is reminiscent of punishment.

5 Likes

For SM-2, the user who misused hard pressed hard again and again, and the interval increased slowly (but still increased). It’s still a wrong way to use SM-2. They just hurts themselves but unaware of that.

However, they have hurt themselves for such a long time that it’s not urgent to remedy it because the injury is too small to perceive.

3 Likes

But Anki already has colors so would It really be that big of a deal? I mean, only about 5% of people have some sort of colorblindness according to google. If it bothers them, then there could be a toggle for it in preferences, or a thing where you could pick which kind of colorblindness you have (although I realize this would be much more work and require changing all of the other colors as well).

And although I don’t know much about colorblindness or how it is for people who have it, wouldn’t they be kind of used to the idea that what they see red as = bad and what they see green as = good? Like traffic lights and whatnot.

I did not know about this, but if it were to take a relatively long time, which it seems like it will, then It would be worth doing it twice IMO since this is such a big problem.

2 Likes

Though I don’t know exactly, I guess users may be doing it intentionally.

I previously researched the medical student Anki community and the add-ons for EaseHell to find out why EaseHell occurs in SM2. e.g. If the user presses Easy, Ease increases and the problem is solved so there should be no need for add-ons for EaseHell in the first place.

But users intentionally do not press Easy because they are worried they will forget it in the exams (or many cards are too difficult so they cannot press Good or Easy, or their workload is too high so they don’t have time to press Again and relearn).

Pressing Hard a lot increases the workload but it has the advantage of ensuring that they will remember them on the exams. (and when they finally can’t handle the reviews, they use add-ons to force the Ease to change and reduce the workload)

So perhaps some users intentionally avoid Easy and press Good and Hard to avoid forgetting on the exams. In such cases it is the user’s intended behavior that increases the workload.

I think this is possible but official Anki already refused, so to do this without us developing it we need to find a generous volunteer developer.

1 Like

I mean the user who misuses hard presses hard when they forget. Then the interval given by SM-2 is too long, so they forget again and press hard again.

2 Likes

bruh

Yes, it’s a small fraction of all people, but some of them have already expresssed concerns (I can’t find where, though, but it was in relation to my idea with outlines for buttons), so…

3 Likes

Deuteranopia, the colorblindness in your images seem to be just a fraction of people with colorblindness.

From wikipedia:

Comparison between normal color vision and the most common type of colorblindness (I realize the severity of colorblindness is probably very different from person to person but still as a reference):

I get that but if something will work for 99% of users It should be implemented IMO. Also having a toggle in preferences would solve that. After changing the colors, then spacing and stuff could be looked in to. This is just my opinion tho and i’m not a dev or have any experience in UI stuff.

The specific red and green color could be picked so its most clear to people with colorblindness as well.

I guess we’re stuck with how things are right now then :person_shrugging:

3 Likes

@dae

To summarize the past few comments: another user on Reddit suggested giving Again a red outline and giving other buttons a green outline, and perhaps increasing the distance between Again and other buttons, which is exactly what I have proposed before.
Answer buttons design to prevent misues

If we can’t satisfy everyone - such as people with OCD and colorblind people - but can satisfy >90% of all people, then this seems like the best solution. If we can’t satisfy everyone, this solution is still better than not doing anything at all.

4 Likes

Yes, well this is just speculation, I mean they know the right way to push the buttons but for some reason they are doing it intentionally. If the learning workload is too high there are several possible cases where it is reasonable to push it that way.

1 Like
  1. Maybe the Hard, Good, Easy buttons can have different shades of green. Wouldn’t do anything, just be aesthetically pleasing.
  2. Instead of a huge gap that separates Again from Hard, there could just be a dashed light-grey line separating them instead? This could also work on Ankimobile by making the line between Again and Hard bolder.
2 Likes

Improving the reviewer UI would be ideal but so far there are no plans for development, so I think that displaying notifications as in proposal #3 by damonfernandez would be a realistic alternative:

As Expertium has already said auto detection would be difficult, but it might not be too difficult to show a notification the first time users enable FSRS, like this:

To make FSRS work effectively, you need to press "Again" when you cannot remember.
[OK] [Help]

In this case SM2 user misuse is not improved so it is not ideal, but this problem has been around for a long time so I think it ignorable.

1 Like

I don’t know if anyone has done this before or if this is stupid, but can’t you use shigeyuki’s leaderboard addon too get a rough estimate on how many people are misusing hard?

If someone has done more than 50 reviews and has a retention of 99% or above it’s pretty much guaranteed they are abusing hard right? IIRC the addon uses again count as retention.

Total people with 50+ reviews: 354
Total people with 50+ reviews and 99%+ retention: 30

So at least ~8.5% of leaderboard addon users are misusing hard.

I think 99% is pretty conservative but even so the real figure would be higher since people who use the leaderboard addon are more likely to be interested in Anki and read the wiki, watch yt vids etc.

1 Like

It’s around 10%, according to my survey

This was conducted on r/Anki, which is full of Anki nerds, so it’s probably worse for Anki users “in the wild”.

4 Likes

Technically possible but retention rate could be almost 100% in these cases:

  1. Learner is reviewing many easy cards prior to the exam to get a high score.
  2. Learner is learning a new language and reviews thousands of simple words.
  3. Learner is taking the time to learn and making high quality cards so they almost never make mistakes.

And these other concerns:

  1. The data LeaderBoard is collecting is inaccurate because it includes relearning cards. (Perhaps only mature cards need to be collected)
  2. Sharing Leaderboard data with third persons requires a change to the privacy policy (technically possible).
  3. Perhaps L-M-Sherlock has already received a large amount of deck data (20,000) from the official Anki so he should be able to extract data from it.

I think the leaderboards are relatively more power users, basically competitive gamification increases the motivation of users, but users who score low may be less motivated by relative grading. (so ideally there should be an additional evaluation system or compete in small groups.)

3 Likes

Edit: This comment was not very relevant to this thread so I moved it to a new suggestion.

1 Like