When I started developing addons, I was looking for a good example of an existing addon that uses the same programming languages and values unit tests. It was difficult because there are nearly 2,000 addons, and unfortunately, the Addons List provides very limited details.
Recently, I created a dataset containing more comprehensive information scraped from the Addons List, GitHub, and the Anki Forum:
Interesting idea. I only skimmed it, but noticed the first add-on is already misnamed. You might want to double-check how youâre pulling the names- could be an issue with the scraping or parsing logic
The parsing algorithm has the ability to override automatically scraped data with correct manual values.
The name âAnki Monitorâ was overridden (for testing purposes) in overrides.yaml.
Iâll remove this value.
Thanks for mentioning it!
This is against AnkiWebâs ToS, so it would have been nice to ask first. If you plan to do this on an ongoing basis, please make sure you minimize the load youâre placing on AnkiWeb.
Coincidentally, I had made a fresh dump of all add-ons a few days prior to judge psutil usage, and Iâve just uploaded it to GitHub in case others will find it useful.
Iâd like to assure you that my script will not place any noticeable load on AnkiWeb:
It caches all raw responses from AnkiWeb and the GitHub API, so it never reads the same page twice (if I need to regenerate the dataset for the same date).
I plan to update the dataset only once per month.
The script reads each add-on page just once and caches it.
Most of the information about the source code is retrieved from the GitHub API.
It doesnât retrieve anything from the Anki Forum.
Sorry, I didnât review any terms of use, as I considered this data to be open-source.
Unfortunately, your dump of add-on sources doesnât contain all the information I would like to present in the dataset, so I canât use just it.
The tarball I provided is a supplement, not a replacement. Itâs mainly useful for grepping through the entire codebase to search for issues, or for gauging how common a particular feature/API is being used.
@dae, my motivation for creating the addons dataset was my own need to find good examples when I tried to develop my own addons (Note Size and Cross-Field Highlighter).
I wanted to find not just the most popular addon, but one that has unit tests, because I really detest manual testing and want to cover as much as possible with unit tests.
Recently, Svelte has started to replace PyQt in addons, and I wanted to have some examples of addons utilizing it. My dataset includes both the language used and the number of unit tests, so I (and other developers) can find what we need.
