It’s not that I’m being thorough, it’s just that what my addon is doing is pretty dangerous and I don’t want it to screw things up.
Either way, the reason I want to test via the most human-like route is because I think addons need different tests than Anki itself needs.
See, I don’t want unit tests. Unit tests are of little value to me. Unit tests are useful when you change your units; I made this addon years ago and I haven’t really added any new functionality to it since. (Although I’m going to.) Because it works fine. It does a very simple—dangerous, but rather simple—job of slightly delaying siblings of cards, so that they don’t appear too close to each other. (By the way, my users strongly thing that this should be core Anki functionality :p).
Anyway, you may ask, why do you want tests at all, then? Well, it’s because I can’t pin my dependencies, or rather my dependncy, which is Anki. You can ship Anki with Python 2 if you want to, but if my addon does not support the latest version of Anki, people are going to be a little sad.
So what I want is to make sure my addon doesn’t blow up with a new Anki version. That is, I want integration test, the most integrationey ones there are. And it’d be awesome if I can put them into GitHub actions, and make those fetch the latest prerelease version of Anki and run tests against that. So instead of learning of any issues 6 months after users report them on my addon page (I don’t think I get emails about addon comments?), GitHub emails me about failing CI jobs on Dependabot’s PR.
I have kind of done this already, actually, and it works pretty well. For now. But since Anki is sometimes a bit mean with tests, I had to invent various clutches to make tests work. It’s likely that upcoming versions of Anki are going to be breaking these clutches. Maybe more so than the addon itself. So it would be nice if I didn’t need these clutches.
I just think making Anki a tiny bit testing-friendlier, and making a little official testing framework, would greatly benefit Anki ecosystem, at very little cost.