https://github.com/open-spaced-repetition/review-sort-order-comparison/blob/main/notebook.ipynb
Hard to see much differentiation in the Cost per Day graph, but in the Retention per Day graph you can see retrievability_desc hovering above the rest. difficulty_asc looks much more unstable on that graph.
Biggest takeaway for me though, the current “Relative Overdueness”, which is basically retrievability_asc is terrible, even compared to some of the more arbitrary seeming like add_order_desc
LMSherlock updated the graphs to include sorting by stability
You want to maximise knowledge though, or more like knowledge acquisition rate. I haven’t read that link.
Can we make one properly written post in a different topic? This one has 90 replies. Most of which is casual rant.
That “don’t comment here” sounds a bit rude but I get what you’re trying to say, just that the way you did was too… ykwim.
Can you also mention that we can possibly cut some fat in new card sort order too? Especially, with the two different Random orders (Random cards; Card type, then random). The later helps keep siblings distant from each other but we can possibly do the same in one single “Shuffle” order.
If the user can do literally any number of reviews per day, the order doesn’t matter.
This argument is flawed because it doesn’t consider many of the other factors that affect learning.
For example, a Random sort order is beneficial because it leads to better memorization. Several studies have shown that interleaving improves learning. (Also see Practicing problems over sessions that are spaced increases learning | Open Learning)
I agree but with review cards we are already doing “randomisation” with Fuzz. But even without it, reviews should be properly interleaved with most sort orders (maybe except order added/latest first).
Here, you are assuming that the Random sort order is somehow producing more interleaving between topics than other sort orders. Sure, for new cards it may. But I don’t expect the same for review cards.
The following suggestion by @Keks makes the most sense to me.
We should have a few pre-configured sort order and then a “Custom” option that allows the user to use a custom-made sort order depending on their needs.
4 Likes
But, why should I take chances when it is easy to just add the Random option?
Also, cards added recently will likely have smaller intervals/stabilities than cards added months/years ago. So, a sort order based on interval/stability will not interleave older and newer cards.
The cards are sorted by the first sorting, if the cards claim the same place, then the sorting takes place according to the second sorting (it doesn’t always make sense). The direction is also important. Give experienced users the opportunity to choose the first sort, the second sort , the direction of the first sort, the direction of the second sort. Then you will be able to satisfy any desires.
Then, instead of the whole zoo of sorts, you can leave a few recommended ones and a custom one. When selecting a custom one, advanced settings would appear.
A simple option is to simply combine all the non-recommended sorts into one group that.
2 Likes
It’s not beginner-friendly. We need pre-defined sort orders
If for the sake of a little convenience for beginners it is necessary to remove a lot of useful sorting, then it is not worth it.
1 Like
Ok, I added this to my other post:
If you really want to keep all this zoo of sort orders, you can add “Custom” that opens a new window where the user decides the metric as well as the order (ascending/descending). So there would be 3-4 pre-defined orders and “Custom”.
I have seen your tables and can’t help but get confused. In your first table, it appears that Difficulty Ascending is the best option. In your other one, it appears that the ability of sorting by Retrievability Descending to maintain retention is conclusive, whereas Difficulty Ascending is noisy.
Is Retrievability Desending really the better option 
This is hard to read, but yes, Retrievability Descending is much less noisy than either of difficulty sorts
Even in the tables, Retrievability Descending beats everything by having the least total time spent studying.
I think a better simulation would be to run it for a set time instead of a set number of cards. Then, judge by how many cards are learned over that set time.