Pin Python dependencies

Anki Desktop
1

Today, continuous integration tests broke for addons that obtain Anki from pypi, apparently because Google released a new version of protobuf. This also means that if you install older Anki from pypi for regular use, it will not work.

Please pin dependencies so that installing Anki from pypi always installs Anki as it was at the moment of release.

(Is seems that it is possible to pin dependencies for already published packages as well! That would be amazing, otherwise every addon that uses pypi (including Anki-Connect!) will have to pin Anki dependencies for Anki.)

2

The packages on PyPI are mainly intended for consumption as a library, and libraries should not pin exact versions of dependencies, as that causes problems - the pinning should only be done at the root node of the dependency chain.

While you are using the packages so that you can run Anki as an application rather than a library, your CI is still effectively the root node in this case, and you can avoid the breakages by pinning the dependencies there instead.

2 Likes
3

Pinning to the exact versions is rarely done, but what’s done often is pinning to a range of versions. From your link,

For example, if you know you need >= 1.2 , but 1.4 is broken, then you can write something like somepkg >= 1.2, != 1.4 . If one of the things you know is that somepkg follows SemVer, then you can add a < 2 .

Note the last sentence. protobuf was updated from 3.20.1 to 4.21.0. They added a new major version so that libraries and apps that properly restrict their dependencies would not break, that is to prevent the exact thing that happened with Anki.

You can often use the caret notation, e.g. ^3.20.1 would restrict protobuf to versions >= 3.20.1 and < 4.

Edit: if i were to pin requirements myself, how do I get them? I see there is python/requirements.txt, but it contains linters and stuff, so I suppose it is not meant for building the release version?

Edit: and what about versions such as 2.1.53rc1? it seems that there’s no such tags on git…

4

Yes, I’m familiar with how version requirements work. The reality is that not all projects follow SemVer, and not all SemVer bumps are breaking - as evidenced by the fact that 2.1.50+ runs on both protobuf 3.x and 4.x, due to differences in codegen that they made in a minor update. The only way to be sure of which versions break things is after they’ve broken things, which is why you should be pinning exact versions yourself.

You can use requirements.txt as a pip constraints file if you wish, or you could simply freeze your dependencies the first time you add a new version.

5

I don’t think using requirements.txt is a good idea:

  • it can’t be used as constraints without some careful sed’ding to remove hashes and extras;
  • its location changes between versions;
  • you can’t get this file for a pre-release version of Anki since there’s no tags;
  • it’s rarely updated anyway and hence won’t reliably produce the environment that actual Anki uses. (Or so I am assuming.)

But I think I’ve found a rather hilarious solution: pypi timemachine! This allows running pip as if you are in the past. tox.ini becomes a little crazy, but the test do pass. So I guess this issue has been resolved.