# New Sort Oder: PSG **URL:** \ **Category:** Suggestions\ **Created:** [October 11, 2024, 8:22pm UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600 "2024-10-11T20:22:06Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![sorata](https://sea2.discourse-cdn.com/flex002/user_avatar/forums.ankiweb.net/sorata/32/34115_2.png) [@sorata](https://forums.ankiweb.net/u/sorata)\ **Post date:** [October 11, 2024, 8:22pm UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600/1 "2024-10-11T20:22:06Z")
![1000231586](https://us1.discourse-cdn.com/flex002/uploads/anki2/original/3X/9/d/9d069a7f6ea9c1be11145e6142591cc72771668d.png) ![1000231624](https://us1.discourse-cdn.com/flex002/uploads/anki2/original/3X/5/2/5270c98aa7bf20f3876c89b4b305d3275130a475.png) _(results from the new simulation)_ # Suggestion ## Why change to PSG? * * * In the latest simulation, a new sort order `PSG_desc` performs better than `retrievability_desc` in `seconds_per_remembered_card`. But it’s worse than `difficulty_asc` albeit slightly. Unsurprisingly, maintaining a constant `desired retention` is still the forte of `retrievability_desc` but that’s not a very valuable metric. Now, one of the issues with `difficulty_asc` was mentioned that the sorting doesn’t change with time as `difficulty` only changes when grading cards. Even for very old backlogs, thus sorting stays the same as before. On the other hand, the sorting with `retrievability_desc` is more dynamic but comes with a slightly worse number for `seconds_per_remembered_card`. `PSG_desc` combines the strong points of both of them: - `PSG` value changes with time. - It performs really well in `seconds_per_remembered_card` almost as good as `difficulty_desc`. There have been some aspersions cast about `seconds_per_remembered_card` or the calculation of `total_time` so I’d also point out that when it comes `total_remembered`, `PSG_desc` is performing better than all other sort orders. (Note: `total_remembered` is amount you remember when the simulation ends i.e. when you’ve finished the twenty thousand `is:new` cards) ## What is PSG? * * * PSG stands for “ **P** otential **S** tability **G** ain” and is calculated something like this: ```auto PSG = (S_recall ÷ S) × R ``` Or in other words, ```auto PSG = change in S after recall × probability of recall ``` Purely from intuition, I expect `PSG_desc` to perform better in edge cases as the formula for PSG includes both difficulty and retrievability. Say, when `retrievability_desc` creates an order very much reversed of what `difficulty_asc` does. Now, it will probably be slow but I wanted to inquire if we implement this instead of what was suggested in [Improving sort orders](https://forums.ankiweb.net/t/improving-sort-orders/50081). # Tangent * * * We tried something like `PSG = (S_recall × R – S_forget × (1 – R) ) ÷ S` but it didn’t work out too well. # Discussion * * * For discussion, see [Ordering Request: Reverse Relative Overdueness](https://forums.ankiweb.net/t/ordering-request-reverse-relative-overdueness/50051). ---
**Author:** ![Keks](https://sea2.discourse-cdn.com/flex002/user_avatar/forums.ankiweb.net/keks/32/20537_2.png) [@Keks](https://forums.ankiweb.net/u/Keks)\ **Post date:** [October 11, 2024, 8:42pm UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600/2 "2024-10-11T20:42:36Z")
> [@sorata](#): > > We tried something like `PSG = (S_recall × R – S_forget × (1 – R) ) ÷ S` but it didn’t work out too well. Why is it not so PSG = (S\_recall × R **+** S\_forget × (1 – R) ) ÷ S ---
**Author:** ![sorata](https://sea2.discourse-cdn.com/flex002/user_avatar/forums.ankiweb.net/sorata/32/34115_2.png) [@sorata](https://forums.ankiweb.net/u/sorata)\ **Post date:** [October 11, 2024, 8:57pm UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600/3 "2024-10-11T20:57:30Z")
There are two things going on in a backlog: - `R` is constantly dropping. - `1 — R` is constantly increasing. If `S_recall` is higher, you should do the card sooner assuming `S_recall × R` is decreasing. If `S_forget` is higher, you should do the card later assuming `S_forget × (1 – R)` is increasing. I felt this had a negative relation with what should come first. * * * Your suggestion was what Jarrett initially said but it was never tested. Do you expect this to work well? Also, I am not sure this will work but I still asked Jarrett to try once, ```auto PLSG = PSG(today) – PSG(tomorrow) ``` If you’re wondering, PLSG is “ **P** otential **L** oss in **S** tability **G** ain”. ---
**Author:** ![Keks](https://sea2.discourse-cdn.com/flex002/user_avatar/forums.ankiweb.net/keks/32/20537_2.png) [@Keks](https://forums.ankiweb.net/u/Keks)\ **Post date:** [October 11, 2024, 9:04pm UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600/4 "2024-10-11T21:04:34Z")
I see the point in: S\_recall × R **+** S\_forget × (1 – R) You will get something like the expected stability. By analogy with [Expected value - Wikipedia](https://en.wikipedia.org/wiki/Expected_value) (S\_recall × R **+** S\_forget × (1 – R) ) ÷ S The expected increase or average increase… ---
**Author:** ![sorata](https://sea2.discourse-cdn.com/flex002/user_avatar/forums.ankiweb.net/sorata/32/34115_2.png) [@sorata](https://forums.ankiweb.net/u/sorata)\ **Post date:** [October 11, 2024, 9:12pm UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600/6 "2024-10-11T21:12:16Z")
@L.M.Sherlock Can you also try doing `(S_recall × R + S_forget × (1 – R)) ÷ S` as the formula for PSG today? (apart from trying out PLSG.) ---
**Author:** ![L.M.Sherlock](https://sea2.discourse-cdn.com/flex002/user_avatar/forums.ankiweb.net/l.m.sherlock/32/16770_2.png) [@L.M.Sherlock](https://forums.ankiweb.net/u/L.M.Sherlock)\ **Post date:** [October 12, 2024, 3:39am UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600/7 "2024-10-12T03:39:25Z")
Fine. Will do it. Update: ```python if review_sorting_order == "PSG_desc": card["S(recall)"] = card.apply( lambda row: ( float(student.next_states(row["states"], row["delta_t"], 3)[0]) if row["stability"] != 0 else 0 ), axis=1, ) card["S(forget)"] = card.apply( lambda row: ( float(student.next_states(row["states"], row["delta_t"], 1)[0]) if row["stability"] != 0 else 0 ), axis=1, ) card["PSG"] = ( card["S(recall)"] * card["retrievability"] + card["S(forget)"] * (1 - card["retrievability"]) ) / card["stability"].map(lambda x: x if x != 0 else 1) ``` ![image](https://us1.discourse-cdn.com/flex002/uploads/anki2/original/3X/c/a/ca14b9163371ee30998cc37c0ee78cf78f91c9c7.png) ![image](https://us1.discourse-cdn.com/flex002/uploads/anki2/original/3X/e/c/ec68b62da85ec2b8d231a74d90826555e7298c24.png) ---
**Author:** ![sorata](https://sea2.discourse-cdn.com/flex002/user_avatar/forums.ankiweb.net/sorata/32/34115_2.png) [@sorata](https://forums.ankiweb.net/u/sorata)\ **Post date:** [October 12, 2024, 7:29am UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600/8 "2024-10-12T07:29:32Z")
Okay, so this is clearly worse. Thanks! The formula stays the same. By the way, will you try PLSG? I’m not hoping much but the data will be interesting. ---
**Author:** ![L.M.Sherlock](https://sea2.discourse-cdn.com/flex002/user_avatar/forums.ankiweb.net/l.m.sherlock/32/16770_2.png) [@L.M.Sherlock](https://forums.ankiweb.net/u/L.M.Sherlock)\ **Post date:** [October 12, 2024, 7:32am UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600/9 "2024-10-12T07:32:37Z")
> [@sorata](#): > > PLSG PLSG ASC or PLSG DESC? ---
**Author:** ![sorata](https://sea2.discourse-cdn.com/flex002/user_avatar/forums.ankiweb.net/sorata/32/34115_2.png) [@sorata](https://forums.ankiweb.net/u/sorata)\ **Post date:** [October 12, 2024, 10:37am UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600/10 "2024-10-12T10:37:30Z")
Ascending. ```auto PLSG = PSG(today) - PSG(tomorrow) ``` ---
**Author:** ![zZH2z](https://sea2.discourse-cdn.com/flex002/user_avatar/forums.ankiweb.net/zzh2z/32/23935_2.png) [@zZH2z](https://forums.ankiweb.net/u/zZH2z)\ **Post date:** [October 12, 2024, 10:59am UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600/11 "2024-10-12T10:59:49Z")
Hi, what is the image with table of order list? Is this is some addon? How to interpret this data? Are those statistics how order of the reviews impact learning? Very interesting 😉 ---
**Author:** ![sorata](https://sea2.discourse-cdn.com/flex002/user_avatar/forums.ankiweb.net/sorata/32/34115_2.png) [@sorata](https://forums.ankiweb.net/u/sorata)\ **Post date:** [October 12, 2024, 11:20am UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600/12 "2024-10-12T11:20:03Z")
> [@zZH2z](#): > > Hi, what is the image with table of order list? Is this is some addon? It’s not an add-on. It’s a [simulator](https://github.com/open-spaced-repetition/review-sort-order-comparison/blob/main/notebook.ipynb) developed by Jarrett Ye, the creator of FSRS. See the discussion I linked in OP. We’ll have too much clutter if we discuss it here again. ---
**Author:** ![rich70521](https://sea2.discourse-cdn.com/flex002/user_avatar/forums.ankiweb.net/rich70521/32/1710_2.png) [@rich70521](https://forums.ankiweb.net/u/rich70521)\ **Post date:** [October 12, 2024, 4:09pm UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600/13 "2024-10-12T16:09:38Z")
> [@sorata](#): > > I’d also point out that when it comes `total_remembered`, `PSG_desc` is performing better than all other sort orders. (Note: `total_remembered` is amount you remember when the simulation ends i.e. when you’ve finished the twenty thousand `is:new` cards) I want to reiterate my aspersions about `total_remembered` here (which filter down to `seconds_per_remembered_card `). I don’t think we should be using these as the objective function. [https://forums.ankiweb.net/t/ordering-request-reverse-relative-overdueness/50051/358?u=rich70521](https://forums.ankiweb.net/t/ordering-request-reverse-relative-overdueness/50051/358) ---
**Author:** ![rich70521](https://sea2.discourse-cdn.com/flex002/user_avatar/forums.ankiweb.net/rich70521/32/1710_2.png) [@rich70521](https://forums.ankiweb.net/u/rich70521)\ **Post date:** [October 12, 2024, 4:36pm UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600/14 "2024-10-12T16:36:02Z")
> [@sorata](#): > > ```auto > PSG = (S_recall ÷ S) × R > > ``` > > Or in other words, > > ```auto > PSG = change in S after recall × probability of recall > > ``` I think you’re calculating _percent_ change in S, not _absolute_ change in S. That may be intentional, but I think absolute change in S would be more useful for PSG purposes. ```auto PSG = (S_Recall - S) x R ``` With percent change in S, it’s basically the exact same sort as `difficulty_asc` because that’s exactly the function of D in the algorithm (the sim results show it’s not the exact same, but pretty damn close). Taken from [https://github.com/open-spaced-repetition/fsrs4anki/wiki/The-Algorithm](https://github.com/open-spaced-repetition/fsrs4anki/wiki/The-Algorithm): > 1. The larger the value of D, the smaller the SInc value. This means that the increase in memory stability for difficult material is smaller than for easy material. With _absolute_ change in S, you’d be maximizing the actual time between reviews. That makes more sense to me. ---
**Author:** ![sorata](https://sea2.discourse-cdn.com/flex002/user_avatar/forums.ankiweb.net/sorata/32/34115_2.png) [@sorata](https://forums.ankiweb.net/u/sorata)\ **Post date:** [October 12, 2024, 4:54pm UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600/15 "2024-10-12T16:54:56Z")
I recommend this discussion be done at the associated forum thread. I want to keep it clean. ---
**Author:** ![rich70521](https://sea2.discourse-cdn.com/flex002/user_avatar/forums.ankiweb.net/rich70521/32/1710_2.png) [@rich70521](https://forums.ankiweb.net/u/rich70521)\ **Post date:** [October 12, 2024, 5:03pm UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600/17 "2024-10-12T17:03:31Z")
Was this in response to my first comment, second, or both? ---
**Author:** ![sorata](https://sea2.discourse-cdn.com/flex002/user_avatar/forums.ankiweb.net/sorata/32/34115_2.png) [@sorata](https://forums.ankiweb.net/u/sorata)\ **Post date:** [October 12, 2024, 5:05pm UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600/18 "2024-10-12T17:05:08Z")
No, I am talking about all discussions. I saw Keks typing so was meant for them too. ---
**Author:** ![L.M.Sherlock](https://sea2.discourse-cdn.com/flex002/user_avatar/forums.ankiweb.net/l.m.sherlock/32/16770_2.png) [@L.M.Sherlock](https://forums.ankiweb.net/u/L.M.Sherlock)\ **Post date:** [October 13, 2024, 7:39am UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600/19 "2024-10-13T07:39:50Z")
```python card["S(recall)"] = card.apply( lambda row: ( float(student.next_states(row["states"], row["delta_t"], 3)[0]) if row["stability"] != 0 else 0 ), axis=1, ) card["S(forget)"] = card.apply( lambda row: ( float(student.next_states(row["states"], row["delta_t"], 1)[0]) if row["stability"] != 0 else 0 ), axis=1, ) card["PSG"] = ( card["S(recall)"] * card["retrievability"] + card["S(forget)"] * (1 - card["retrievability"]) ) / card["stability"].map(lambda x: x if x != 0 else 1) card["S(recall_tomorrow)"] = card.apply( lambda row: ( float(student.next_states(row["states"], row["delta_t"] + 1, 3)[0]) if row["stability"] != 0 else 0 ), axis=1, ) card["S(forget_tomorrow)"] = card.apply( lambda row: ( float(student.next_states(row["states"], row["delta_t"] + 1, 1)[0]) if row["stability"] != 0 else 0 ), axis=1, ) card["PSG_tomorrow"] = ( card["S(recall_tomorrow)"] * card["retrievability"] + card["S(forget)"] * (1 - card["retrievability"]) ) / card["stability"].map(lambda x: x if x != 0 else 1) card["PLSG"] = card["PSG"] - card["PSG_tomorrow"] ``` ![image](https://us1.discourse-cdn.com/flex002/uploads/anki2/original/3X/2/a/2a9ddd4369f8931e0a98845e04e0b37e46c3d362.png) ![image](https://us1.discourse-cdn.com/flex002/uploads/anki2/original/3X/b/5/b5ebf810a6348877d804b273e453bdcd2a34da30.png) ---
**Author:** ![sorata](https://sea2.discourse-cdn.com/flex002/user_avatar/forums.ankiweb.net/sorata/32/34115_2.png) [@sorata](https://forums.ankiweb.net/u/sorata)\ **Post date:** [October 13, 2024, 8:40am UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600/20 "2024-10-13T08:40:06Z")
@keks It doesn’t work apparently. It’s worse. Btw, would you try `PLSG_desc` Sherlock? Okay, I don’t get it. What did you try here? Did you try Keks’ suggestion or my suggestion or you tried both? If you tried both, now we don’t know what worked and what didn’t. ---
**Author:** ![L.M.Sherlock](https://sea2.discourse-cdn.com/flex002/user_avatar/forums.ankiweb.net/l.m.sherlock/32/16770_2.png) [@L.M.Sherlock](https://forums.ankiweb.net/u/L.M.Sherlock)\ **Post date:** [October 13, 2024, 8:59am UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600/21 "2024-10-13T08:59:22Z")
I thought I replied your ask. > [@sorata](#): > > Btw, would you try `PLSG_desc` Sherlock? OK ---
**Author:** ![sorata](https://sea2.discourse-cdn.com/flex002/user_avatar/forums.ankiweb.net/sorata/32/34115_2.png) [@sorata](https://forums.ankiweb.net/u/sorata)\ **Post date:** [October 13, 2024, 9:12am UTC](https://forums.ankiweb.net/t/new-sort-oder-psg/50600/22 "2024-10-13T09:12:06Z")
Use this formula though (for calculating PSG): ```auto PSG = S_recall/S × R ``` As the other one with S\_forget didn’t work. [Next page](https://forums.ankiweb.net/t/new-sort-oder-psg/50600.md?page=2)