I’ve been working on migrating Anki’s current Makefile-based build system to Bazel. It’s still a work in progress, but is now in a state where people can try it out, and early feedback on how it works on other people’s machines would be appreciated.
A few benefits of the new implementation:
Tested on Windows, Mac and Linux
Fewer dependencies need to be installed prior to building
Incremental builds are faster, with less unnecessary rebuilding
Tests from different subcomponents can now run in parallel
A few of the things left to do:
Wheel building is not implemented yet
Python dependencies are not yet pinned to specific versions
Anki will run more slowly, as it is not being compiled with optimization
Windows CI on GitHub is broken, and may require moving to a different CI platform.
The code is now on the master branch. If you already have an Anki source folder, I recommend you start in a new folder. After downloading, please see the platform-specific docs.
It certainly brings a bunch of things to the table, but it proved to be a lot more work to integrate than initially expected - mainly due to rough edges in the various language rules.
It looks like Windows Defender suddenly thinks the file has a virus. If you unquarantine the file or try an older version, it should work around the issue.
So no more 32bit support, huh? @agentydragon you are somewhat familiar with bazel, right? How much work would it be to build Anki on 32bit debian with bazel?
@zjosua Bazel itself is only released for 64-bit, but the binaries Bazel builds can (AFAIK) target any platform, though it’s not a configuration I’ve used in my projects so far.
I think CROSSTOOL might be the right keyword to look around for 32-bit builds?
I’d rather compile Anki on the 32bit device itself rather than cross compiling it on another computer. And according to the crosstool README, it provides toolchains for x86_64, ARMv7-A, and ARMv8-A, but I have i686.
But now it seems that the rules for my architecture are missing (or wrong). Honestly, I didn’t know Bazel before Damien introduced it to build Anki and so far I didn’t read much about it either, so my assumptions might well be wrong.
But due to this output I assume that a 64bit binary of node was installed:
zjosua@debian:~/sw_dev/anki$ ./run
Starting local Bazel server and connecting to it...
... still trying to connect to local Bazel server after 10 seconds ...
INFO: Repository npm instantiated at:
/home/zjosua/sw_dev/anki/WORKSPACE:14:11: in <toplevel>
/home/zjosua/sw_dev/anki/defs.bzl:48:17: in setup_deps
/home/zjosua/.cache/bazel/_bazel_zjosua/1c79c7ac51bf6692c8fc3f9a9a4ef48a/external/build_bazel_rules_nodejs/index.bzl:89:18: in yarn_install
Repository rule yarn_install defined at:
/home/zjosua/.cache/bazel/_bazel_zjosua/1c79c7ac51bf6692c8fc3f9a9a4ef48a/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl:424:31: in <toplevel>
ERROR: An error occurred during the fetch of repository 'npm':
Traceback (most recent call last):
File "/home/zjosua/.cache/bazel/_bazel_zjosua/1c79c7ac51bf6692c8fc3f9a9a4ef48a/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl", line 401, column 13, in _yarn_install_impl
fail("pre_process_package_json.js failed: \nSTDOUT:\n%s\nSTDERR:\n%s" % (result.stdout, result.stderr))
Error in fail: pre_process_package_json.js failed:
STDOUT:
STDERR:
/home/zjosua/.cache/bazel/_bazel_zjosua/1c79c7ac51bf6692c8fc3f9a9a4ef48a/external/nodejs_linux_amd64/bin/node: line 19: /home/zjosua/.cache/bazel/_bazel_zjosua/1c79c7ac51bf6692c8fc3f9a9a4ef48a/external/nodejs_linux_amd64/bin/nodejs/bin/node: cannot execute binary file: Exec format error
/home/zjosua/.cache/bazel/_bazel_zjosua/1c79c7ac51bf6692c8fc3f9a9a4ef48a/external/nodejs_linux_amd64/bin/node: line 19: /home/zjosua/.cache/bazel/_bazel_zjosua/1c79c7ac51bf6692c8fc3f9a9a4ef48a/external/nodejs_linux_amd64/bin/nodejs/bin/node: Success
ERROR: no such package '@npm//@bazel/labs': pre_process_package_json.js failed:
STDOUT:
STDERR:
/home/zjosua/.cache/bazel/_bazel_zjosua/1c79c7ac51bf6692c8fc3f9a9a4ef48a/external/nodejs_linux_amd64/bin/node: line 19: /home/zjosua/.cache/bazel/_bazel_zjosua/1c79c7ac51bf6692c8fc3f9a9a4ef48a/external/nodejs_linux_amd64/bin/nodejs/bin/node: cannot execute binary file: Exec format error
/home/zjosua/.cache/bazel/_bazel_zjosua/1c79c7ac51bf6692c8fc3f9a9a4ef48a/external/nodejs_linux_amd64/bin/node: line 19: /home/zjosua/.cache/bazel/_bazel_zjosua/1c79c7ac51bf6692c8fc3f9a9a4ef48a/external/nodejs_linux_amd64/bin/nodejs/bin/node: Success
INFO: Elapsed time: 34.211s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
FAILED: Build did NOT complete successfully (0 packages loaded)
Node and npm were already installed through apt. Isn’t there a way to let Bazel use the binaries from that installation?
This statement (by a former Google dev no less) makes me wonder if I even have a chance to succeed. But it might not be applicable to my situation.
rules_nodejs may provide some way of specifying a local node installation, but I do not know if you’ll run into further issues. I’m afraid the writing has been on the wall for 32 bit for some time - Qt and orjson already did not provide 32 bit Linux builds, and with Bazel not providing them either, it is hard to justify spending engineering effort trying to keep a dying platform working when the upstream tools have already decided against it.
(of course, you should be able to continue to use 2.1.35 on that machine for the foreseeable future)
Yeah no worries @dae. Thanks for supporting 32bit for as long as you did.
I expected things to break for 32bit for a while now and after the introduction of rust about a year ago, I was surprised when I found that I could still build the latest code.
I’m having big trouble trying to build Anki for Flatpak, for numerous reasons:
inability to access the network during the build process
no clear enumeration of dependencies
The proliferation of languages in Anki (Rust, C++, Java for Bazel(?), NodeJS, Python) only compounds this issue I’m afraid.
I think a good first step would be to get a list of the sources that Bazel uses for the build, so I can a) enumerate them in the manifest and b) download them in advance.
‘bazel sync’ will download all the requirements up front, but will pull in dependencies from other platforms as well, so it’s not as efficient as fetching them during the build. The easy route would be to build (or download from pypi) the wheels separately and just install them into the flatpak. If you don’t want to do that, some work will likely be required to get one opinionated build system to play nicely with the other.
Hi, first time trying to build Anki from source. I’m on a Mac and I followed all the instructions. I installed Python 3.8 and all the dependencies. I installed Rust. However, I haven’t been successful in building Anki using ./run
I’ve had many errors that I’ve been able to work through (like error with rustc using -Z argument which is only available in nightly build… had to use rustup default nightly)
Anyway, I’m currently stuck. It’s an issue with installing raze__reqwest. I get the following error: (snipped a bunch of extraneous lines)
Error in fail: error running 'git init /private/var/tmp/_bazel_michael/4abc2a92eb58c6b690000d6614d796c4/external/raze__reqwest__0_10_8' while working with @raze__reqwest__0_10_8:
fatal: invalid branch name: init.defaultBranch =
ERROR: Error fetching repository: Traceback (most recent call last):
File "/private/var/tmp/_bazel_michael/4abc2a92eb58c6b690000d6614d796c4/external/bazel_tools/tools/build_defs/repo/git.bzl", line 174, column 30, in _new_git_repository_implementation
Has anyone successfully built Anki on a fresh Mac?
Ok, I figured out the git init error. Somehow, my .gitconfig had a section for defaultBranch that was blank. I’m not sure how it got set that way. Anyway, after removing it, I got past those errors.
I still had additional errors with the build. I’m using pyenv to manage Python and made sure I was using v3.8.6. I ended up installing Python 3.8.6 from python.org and now it seems to be building properly.
I’m not sure why it isn’t working properly with pyenv as that seems to be the recommended way to run Python on the Mac.
Anyway, now that I’ve got it to build, I can start playing around with some of the ideas I have.
