When I started developing addons, I was looking for a good example of an existing addon that uses the same programming languages and values unit tests. It was difficult because there are nearly 2,000 addons, and unfortunately, the Addons List provides very limited details.
Recently, I created a dataset containing more comprehensive information scraped from the Addons List, GitHub, and the Anki Forum:
Interesting idea. I only skimmed it, but noticed the first add-on is already misnamed. You might want to double-check how you’re pulling the names- could be an issue with the scraping or parsing logic
The parsing algorithm has the ability to override automatically scraped data with correct manual values.
The name “Anki Monitor” was overridden (for testing purposes) in overrides.yaml.
I’ll remove this value.
Thanks for mentioning it!
This is against AnkiWeb’s ToS, so it would have been nice to ask first. If you plan to do this on an ongoing basis, please make sure you minimize the load you’re placing on AnkiWeb.
Coincidentally, I had made a fresh dump of all add-ons a few days prior to judge psutil usage, and I’ve just uploaded it to GitHub in case others will find it useful.
I’d like to assure you that my script will not place any noticeable load on AnkiWeb:
It caches all raw responses from AnkiWeb and the GitHub API, so it never reads the same page twice (if I need to regenerate the dataset for the same date).
I plan to update the dataset only once per month.
The script reads each add-on page just once and caches it.
Most of the information about the source code is retrieved from the GitHub API.
It doesn’t retrieve anything from the Anki Forum.
Sorry, I didn’t review any terms of use, as I considered this data to be open-source.
Unfortunately, your dump of add-on sources doesn’t contain all the information I would like to present in the dataset, so I can’t use just it.
The tarball I provided is a supplement, not a replacement. It’s mainly useful for grepping through the entire codebase to search for issues, or for gauging how common a particular feature/API is being used.
@dae, my motivation for creating the addons dataset was my own need to find good examples when I tried to develop my own addons (Note Size and Cross-Field Highlighter).
I wanted to find not just the most popular addon, but one that has unit tests, because I really detest manual testing and want to cover as much as possible with unit tests.
Recently, Svelte has started to replace PyQt in addons, and I wanted to have some examples of addons utilizing it. My dataset includes both the language used and the number of unit tests, so I (and other developers) can find what we need.
I looked into Svelte a bit but I haven’t used it yet. The workaround for accessing Svelte from add-ons seems broken in the latest Anki (the latest Svelte compatibility is the cause), also developers familiar with Svelte add-ons haven’t been active on Anki recently and their code no longer works. So it seems simpler and more stable to just use JavaScript and Python for add-on development for now.
