In this thread@dae mentioned that anki gets very slow when it has to handle a lot of fields.
Could someone explain to me why this behavior is, and how serious it can be depending on the number of fields? I understand that databases and tables, columns tend to slow down more than rows, but I don’t really understand why and if it is related to this. I would like to understand the root of the matter.
Does the same thing happen with clozes or do they have a different behavior? Will the performance of having a note with 200 clozes be the same as that of 1 note with 200 clozes?
To a large extent my interest in this matter comes from the fact that I am trying to migrate the modeling of my collection with closet, the new add-on from @hengiesel. It has many advanced cloze options, but it simulates cloze behavior with a field for each occlusion, and I am concerned that this will affect the performance of my collection.
As a matter of fact I mentioned some delay before, for some of the bigger models, but wasn’t quite sure about whence it came.
Closet is a template engine written in JavaScript, which means it does some parsing, however in the browser environment, it typically executes in <30ms, and only exceeds 100ms if you get close to 1000 tags. So it is reasonably fast.
I personally have a model with 10 fields, and one with 31 fields, and I’ve only ever noticed the delay on the one with 31 fields, even if I use very few tags. Especially in special circumstances (on AnkiMobile on low battery) I’d say the delay can exceed 300ms. So it may have to do with the amount of fields.
I wonder whether this issue can be solved my moving more processing to the Rust backend? Are there any plans regarding this?
thanks for answering quickly @hengiesel as always. While we wait for the developers responses , I ask you if in case anki manages multiple cloze per note much faster than multiple fields per note, would there be a way to transform your addon to work as cloze, and not as fields, or that is something tremendously difficult to achieve?
PD: Up to now I use image occlusion, sometimes exceeding 80 occlusions. This is not a problem with IO as it handles it as different notes, but it could be in closet.
Well I haven’t done that before but I certainly have a few ideas off the top of my head.
What one might loose is the ease of configuring individual cards. Like “reveal this rectangle for card 17”.
I wouldn’t mind tinkering on this for an evening, afterwards I’d be able to tell you more
The card template rendering code is in Rust these days, and handles large template+field combinations much faster than before. It’s still a good idea to store only the minimum data you can get away with - if each note has 100 bytes of field text, and then you change it so each note has 400 bytes of field text, searches will become approximately 4 times slower. If you only have thousands of notes you likely won’t notice, but as your note count grows, it could become a problem.
The delay you’re seeing in AnkiMobile is probably not the template rendering - it’s more likely to be a delay in the storage layer or something webview related.
Let me see if I understand. Today the performance of having
100 notes generating 10,000 cards (100 fields in each note)
Would it be the same as having 10,000 notes generating 10,000 cards (1 field per note)?
Does the answer apply the same for cloze as it does for other types of notes? Sorry for the inconvenience, it was not clear to me.
No, the performance depends on what you’re looking at. The note count * note content size will determine the speed of searches. The card template count * card template complexity will determine the amount of time spent generating cards. If you require more detail, I recommend you create some dummy collections and benchmark them.
Okay, I made some experiments and came to a result.
To shortly explain the note type again, it works similar to the built-in Basic (optional reversed card) note type. There are fields called cmd fields which are numbered from 1 to 25 and optionally turn on Card 1 to Card 25.
I made to derivatives of this note type:
One with fields reduced to 5
One with card types reduced to 5
And as @dae said before, the version with only 5 fields was just as fast as it was before (which means not very much), however the version with only five card types had a significant speed boost.
So it seems like Anki doesn’t like note types with a lot of card types.
What explains this speed penalty? Is there a way to speed this up in the Anki code base?
Regarding a cloze-based solution: there is no way to figure out, which Cloze card-type (Cloze 1, Cloze 2, etc.) you’re on, once you’re in the web view, as far as I see it, which makes this impossible right now.
Edit: It seems I didn’t read his previous answer closely enough, as he already described what I “discovered”. By “card template complexity” do you mean the size of the card template content?
Thanks for the info. It is a great limitation that I hope can be solved.
Regarding cloze, surely there must be some way to identify the active card, maybe with javascript, I don’t know. Someone who knows in depth how cloze works can surely suggest how to do it.
Edit: have you done any experiments with cloze performance? I’m pretty sure, but it’s still unclear to me if it’s much faster handling a lot of occlusions.
Edit2: By the way @glutanimate does this have something to do with why you decided to have Image Occlusion create each occlusion as a different note?
, I ask you if in case anki manages multiple cloze per note much faster than multiple fields per note, would there be a way to transform your addon to work as cloze, and not as fields, or that is something tremendously difficult to achieve?