Before I start, @dae is it fine to rate a deck that I personally don’t use? There is a server where most people have such an opinion so I just wanted to confirm.
Now, I recently came across one deck on AnkiWeb that got a lot of ratings in a very short period of time. When I was looking through the reviews one of them ended with —xelieu. I know this person called xelieu and no way he actually uses this deck. He’s a quite experienced Japanese learner and the deck I’m talking about is for Japanese beginners. He actually personally knew the person who created this deck. I read through some other reviews and many of them sound like experienced learners. I bought this topic up in the discord server from where this deck originates and the server owner re-assures me all of this is fine as they are trying to bring a revolution.
I don’t know if I’m suffering from 被害妄想 (persecution complex) as the server owner says but I just thought I’d bring it up here.
When I went back to the deck some more reviews were there with the person’s name being written, ofc people who would never use this deck. One of them was a sarcastic
I wish this was my first deck
I think rating decks shouldn’t be allowed unless you downloaded a deck while logged in. Otherwise it gives unfair advantages to some decks over others.
There should be a report button for both decks and non-sensical reviews like the one I quoted above.
AnkiWeb doesn’t track who downloads which deck. Doing so would consume more resources, and I’m not sure the loss of privacy is worth keeping out a few bad actors.
sorry for that. I don’t know why I thought otherwise. I was also thinking of showing downloads but once people abuse the rating system downloads will increase anyway.
@dae I just came up with this, how about having a maximum cap on the number of ratings? I know Anki keeps track of the number of downloads. My idea is the number of downloads should be the maximum number of reviews that are shown/used for sorting.
But won’t that mean that bad-actors can essentially block any negative reviews by flooding their add-ons with fake positive reviews? That would make the situation worse.
In the end, you will not stop someone who heart is set on committing fraud. Fake reviews cannot really be prevented, they can only be dealt with.
It’s not impossible for an experienced learner to review material intended for beginning learners. They can review the material without studying from the deck – and even without downloading the deck directly from the shared site. I’m not saying that happened in that person’s case – just pointing out that it is possible. Those reviews, whether positive or negative, can even be quite useful, because beginners don’t necessarily know what they need to know.
Yes some are useful because they talk about how sorting is better in that deck. They used data from mainly jpdb frequency dictionary. It’s useful because words common in Manga/Novels come in top. Another talked about native audio. So yes two of those were useful. But xelieu said something like “Yes my guy did it” so no I don’t think he actually used this. And others are similarly useless comments.
I thought about this situation but no. They can’t. Guess why? Because to do that they’d need to download the deck first. Understand the chronology. You first download, downloads increase, then you rate it. Yes the bad actors can block future negative reviews. BUT only those coming from people who didnt download the deck. If you download it then max cap increases and you put a rating.
The only situation where this doesn’t work is when they’ve bots that rate decks as soon as someone downloads it.
As long as useless reviews stay looking as useless as that – I’m not concerned we need to overhaul the system to prevent them. That one is easily disregarded. My point was that garbage-reviews are a different issue than reviews by someone too advanced to have used the deck.
If someone is posting a review immediately after having downloaded the deck, that is probably a garbage-review too. It takes time to test a deck out, even if you find issues with it. There’s no way to ensure that the review spot for my download is still there when I come back with something to say.
I don’t think unless the effort is organised we would see this problem that much. Also this I mentioned, you can write the review but the rating would not be used in sort. Others can view it though. This does not completly solve the problem I understand but this will still prevent bad decks and paywalled ones coming on top. Actually I saw a group called Languages on Fire and they’ve made a two decks last year both of which come at the top. Never seen a Russian deck from them before so most of the ratings were new and potentially “fake” (which they disagree with because they are from “friends”). So maybe one more thing that can be added is a time cap. Once again, if they spam fake ratings they’ll do it regardless of whether you implement such a system or not.
The issue with reviews on Ankiweb is pretty complex. Yes, you can make a deck appear popular by asking a bunch of people to write reviews. But I’ve also seen many really bad decks with lots of positive reviews. So those aren’t even fake reviews, those are genuine reviews made by people who are not experts in the field of study a deck covers, and, consequently, they can only write a misleading review.
I agree my solutions aren’t good enough. What I was saying though is that people who spam fake reviews under such systems will be doing them regardless of what we do. But there will also be people who will be discouraged to attempt such things if we do have counter measures.
Tatsumoto’s point is also valid. Bad reviews are often also written by people who genuinely believe something is good (when it’s not).
Consider this me brainstorming, I think I have two great ideas.
Don’t let new accounts rate decks on AnkiWeb.
Allow people to comment on decks without rating them. I see a lot of people rating decks because they wanted to say something and by default they rate it positively. (I do that )
(edit: this apparently has been suggested in Jan 2023)
If anyone can add something, please do so. Brainstorming helps.
Edit: I just thought of another thing. And this might solve the problem with putting a cap on rating. Instead of a deck specific cap, the cap should be user-specific. This also means privacy won’t be a concern, unless some people think tracking how many downloads they had is a breach of privacy.
I think rating decks shouldn’t be allowed unless you downloaded a deck while logged in. Otherwise it gives unfair advantages to some decks over others.
Suppose my egotistical “BFF” creates a deck and asks me to give it five stars. So I download it, review it, and never use it. What was accomplished by the restriction?
)