Issue with huge card IDs

Anki Help
1

Recently I started to hit crashes when opening the Card Info screen of certain cards.

This is the error message:

A fatal error occurred, and Anki must close. Please report this message on the forums.
Anki 2.1.46 (94913ec2) Python 3.8.6 Qt 5.14.2 PyQt 5.14.2
Platform: Windows 10
Flags: frz=True ao=True sv=?
Add-ons, last update check: 2021-08-10 01:25:11

Caught exception:
Traceback (most recent call last):
  File "aqt\browser\browser.py", line 515, in showCardInfo
  File "aqt\browser\card_info.py", line 21, in __init__
  File "anki\stats.py", line 36, in report
  File "anki\collection.py", line 823, in card_stats
  File "anki\_backend\generated.py", line 804, in card_stats
  File "anki\_backend\__init__.py", line 126, in _run_command
pyo3_runtime.PanicException: SystemTimeToFileTime failed with: The parameter is incorrect. (os error 87)

Further search revealed that this is caused by the cards having unusually large IDs (specifically, the crash happens around this line).

If I’m not mistaken, card IDs are just creation timestamps in milliseconds, so I should not have cards with huge IDs such as 1488978224757674 (a date around the year 49153).

The issue only affects cards created in the last two months. I have about 3000 cards affected. A large portion of them are from a shared deck, the others are mostly imported from a text file, while just a couple of them are manually made.
I don’t think I have any add-ons that can be the cause, but I will look further.

Here is a CSV file with affected rows of the cards table: cards-with-large-ids.csv · GitHub
Also a deck package: cards-with-large-ids.apkg

Judging from the fact that Anki uses max(id) + 1 when an ID is already used and that most of the affected cards are imported, I guess it’s all started with one odd card that has a large ID for some reason, then the ID stuck around as the maximum and gradually infected more newly added cards.

I will probably have to do a surgery on my collection to fix the IDs and the revlog.

1 Like
2

It was probably a card created with a third-party tool. :frowning: I could add a check to the import routines when they get an update, but it will still be possible for add-ons and tools opening the collection externally to bypass the check unfortunately.

1 Like
3

It’s most likely caused by some cards from the AJT Kanji Transition deck. I’ll try to report the issue to the author.

Does this code for fixing the cards look right?

# minimum broken id found in the collection
threshold = 1488978224757674

next_cid = int(mw.col.db.scalar('select id from cards where id < ? order by id desc', threshold)) + 1
cids = mw.col.db.list('select id from cards where id >= ?', threshold)

for cid in cids:
    used = mw.col.db.scalar('select id from cards where id = ?', next_cid)
    if used:
        next_cid += 1
    mw.col.db.execute('update cards set id = ? where id = ?', next_cid, cid)
    mw.col.db.execute('update revlog set cid = ? where cid = ?', next_cid, cid)

That would be appreciated anyway.

4

Cards with IDs around the year 40,000+ is something that I’ve been encountering all the time for many many years that I’ve used Anki, so I don’t even pay attention anymore.

Large IDs are nasty and they display incorrectly when you use the Anki Browser (e.g. type added:1 in the search bar and all of them come up at once), but I have no idea what causes them. Apparently, it’s very easy to get such “broken” cards when you import a TSV file, or add cards with AnkiConnect, or create a new note with two cards in it. The first card of the note gets a normal ID, and the second one get a huge id.

It’s most likely caused by some cards from the AJT Kanji Transition deck.

I checked the deck, and it appears to have 89 cards with huge IDs. Other cards are fine. Again, I don’t know why it happened.

It was probably a card created with a third-party tool.

Can’t say for the other cards, but at least I made “AJT Kanji Transition” with nothing but Anki itself.

I’ll try manually fixing my deck since it’s one of the affected, but in the future it would be great to see certain checks run when people upload their decks to AnkiWeb (as well as import old decks that are already on AnkiWeb).

It would also be ideal to prevent such cards from appearing in the first place.

5

I ran some tests on my main profile and detected more than 50,000 cards with enormous IDs. I was able to fix them by running this add-on:

import time

from aqt import gui_hooks, mw


def card_created_now() -> int:
    return int(time.time() * 1000)


def fix_ids():
    problematic_cids = mw.col.db.list('select id from cards where id >= ?', card_created_now())
    if problematic_cids:
        for cid in problematic_cids:
            print(cid)
            new_id = card_created_now()
            while mw.col.db.scalar('select id from cards where id = ?', new_id):
                new_id += 1
            mw.col.db.execute('update cards set id = ? where id = ?', new_id, cid)
            mw.col.db.execute('update revlog set cid = ? where cid = ?', new_id, cid)
        mw.col.mod_schema(check=False)


gui_hooks.profile_did_open.append(fix_ids)

It’s a slightly modified version of what Abdo posted.

Though I must point out that Anki never crashed for me because of huge card ids.

2 Likes
6

I first encountered the issue when using the Card Info During Review add-on. Initially I thought it’s a bug in the add-on, but then I found that opening the native Card Info screen of an affected card from the browser causes the same issue.

Maybe the crash only happens on Windows (notice the SystemTimeToFileTime function in the error message, which doesn’t accept years larger than 30827). Other OSs could be more tolerant of large dates.

Thanks for the code, and for the great deck!

1 Like
7

This is unlikely to be a bug in Anki. It’s more likely something like users passing a millisecond timestamp to genanki/package.py at fc8148ab5cabeb16e8957ebb3e7d8ec48bed7cf5 · kerrickstaley/genanki · GitHub or similar. Once you have a bad ID in your collection it will spread as abdo has pointed out, so checks on import would be nice to have.

1 Like
8

I had the same problem and found that if there are any cards left in the Custom Study Session deck they have special IDs that get in the way of the normal max(id) +1 counter. Just finish your custom study before adding any new cards.

9

Card ids do not change when moved into a filtered deck. Perhaps you are thinking about due numbers instead?

10

Even if card IDs don’t change in a filtered deck, my experience is that having cards in a filtered deck definitely causes ridiculously high ID numbers to be assigned to added Notes.

11

That would surprise me. Can you provide a set of steps that reproduces the issue in the latest version, without add-ons or an existing invalid id?

12

Sorry but I can’t reproduce the problem now either with or without add-ons loaded. I used to have a problem with high numbers. I say problem because Check Database would complain about them. Then when I emptied the filtered deck the problem went away. I must have had some problematic card I guess.
Sorry for taking your time.

13

I have many cards with large cid numbers. I would like to use your code (tatsumoto san) to correct them. Sorry for taking your time but I don’t understand how to use the code. Could you please describe how to run it in Anki? Is it python? I have no experience using python. If it is complicated, please point me to instructions for using python with Anki.

14

It is Anki add-on code. Let me publish it on AnkiWeb so that everybody could run it. Wait a few minutes.

15

Here. Hope it helps.

4 Likes
16

Thank you!!! Anki took a while to open so I thought something was seriously wrong, but then it opened with all the cid’s corrected!! Great. Maybe your AnkiWeb add-on instructions could including a warning that it takes a bit of time so people don’t get scared like me.

1 Like
17

Alright