How to prevent users from misusing Hard? Ideas are welcome

[Expertium]

@dae today I will post a survey about this answer button design on r/Anki. In the meantime, I came up with a way to algorithmically detect whether someone is misusing Hard.
Just do 3 checks:

1. Check if w_15 (a parameter in the parameters field) is <0.01
2. Check if RMSE is >5.0%
3. Check if the number of reviews is >1000, just to avoid false positives

If all 3 are true, display the following pop-up message:

There is a possibility that you are using the Hard button incorrectly. When you press Hard, Anki assumes that you have successfully recalled the card.
Please keep in mind that Hard is not “fail”, it’s “pass”.

It’s not 100% reliable, but it should be reliable enough. And it’s as simple as it gets: three else-if statements and a pop-up. Doesn’t get much simpler than this.
EDIT: nevermind, LMSherlock found a problem with my idea.

[sorata]

1. But it will be hard to do in mobile clients where most reviews happen IMO. You don’t have enough space to put a gap between Again and other buttons and similar coloured buttons would be a nightmare to distinguish from each other.
2. Dae is simply prioritising other stuff, so you are not convincing him with a survey. Might be useful for future reference though.

[Deranged6605]

Just put the again on top

[ZpLiu]

I can not appreciate my thanks for your effort in this picture presented.

[Expertium]

Here are the results so far: Answer Buttons Design

8-9% of users thought that Hard = fail

Around half of users like this design

Around 75% of users think that it’s helpful for solving tbe problem with misusing Hard. I deliberately asked two different questions - do you like it vs do you think it’s helpful. There are more people who think that it’s helpful than people who like it.

@dae this should give you a good idea how many users are misusing Hard, and maybe nudge you towards a UI redesign.

@L.M.Sherlock so do you think my idea with detecting who is misusing Hard based on w_15 and RMSE will work? I think it will.
It will most likely produce more false negatives (the user IS misusing Hard, but the system didn’t detect it) than false positives (the user isn’t misusing Hard, but got a pop-up). I think that’s good, it means that it won’t be an annoyance for normal users.

[Whar]

I think a good solution for the mobile UI could be this:

[Album] imgur.com

[Expertium]

Good should be the biggest button because it’s the one you are expected to press most often:

1. Again will only be used 15-5% of the time, for most users
2. Hard and Easy are expected to be used when Good is not good enough for whatever reason

[sorata]

we can just do the colours. that should be better.

[Expertium]

AnkiDroid already has different colors for buttons

[sorata]

I feel like now I know why this happens. Look at these buttons from the new reviewer —

I mean look at the colours we have. I already think it’s easy to categorise the buttons into Good+Easy/Again+Hard. If you now look at the colours Again+Hard have very similar looking colours.

I’d expect the colours to communicate something but here they have the potential to miscommunicate. Making Hard+Good+Easy different shades of the same colour (or different but similar colours) might be a good idea (I mean, all green can confuse you). But clearly, something needs changing here.

(I checked and found the same issue with AnkiMobile)

[Expertium]

@David how about this?

Although colorblind people will beat us with rocks for this

[raresxd]

I’m a beginner when it comes to Anki so correct me if I’m wrong but the wording doesn’t seem that intuitive. “Good” should probably be the most casual button when it comes to grading a card as correct, so “I hesitated a little” might spark some confusion to some users. Removing that part and only leaving “My answer was correct” is better imo. I agree with the other buttons, though the “easy” one might benefit from some other wording.

[Keks]

Ankidroid:


Different colors have the advantage of making it easier to aim at the buttons.

[Keks]

It takes up a lot of space on the screen.

To be honest, the only viable option is a two-button mode for beginners.

[sorata]

My screenshot was from the new reviewer which has colours similar to the AnkiMobile app. The current one is better.

[Deranged6605]

This is what I’m talking about🤣. And to be honest, this is basic form of it. No changes to colours or wording, just design

[sorata]

@Expertium

Although colorblind people will beat us with rocks for this

u come up with some hilarious phrases. tbh, we can have some accessibility setting to change colours. well, we wouldn’t need to do any of this if 2 button mode became a default option.

[L.M.Sherlock]

We don’t have the real labels in the dataset, so we don’t know how accurate the detecting method is.

[Expertium]

We do though. Or by “real” do you mean “extracted directly from the user’s brain”? In which case, I agree, we don’t.

[L.M.Sherlock]

How confident do you think about the detecting method? My prior is 50%. If we could collect some data from those respondents and check whether their w_15 is close to 0, I will be more confident.