How to correctly import a dependency of a third-party package?

[Quinnium]

Hi all,

I’m developing my first Anki add-on and have been struggling to figure out how to properly import dependencies of a third-party package.

Context

My add-on depends on the openai python library, which utilizes python’s typing_extensions module (version 4.8.0).

I’ve managed to partially import the openai library by adding the following code to my add-on’s __init__.py (thanks to /u/abdo for sharing their solution):

import os
import sys
import importlib
from pathlib import Path

module_name = "openai"
dir_name = "lib"
ADDON_ROOT_DIR = Path(__file__).parent
sys.path.insert(0, os.path.join(ADDON_ROOT_DIR, dir_name))
module_path = os.path.join(ADDON_ROOT_DIR, dir_name, module_name)
source = os.path.join(module_path, "__init__.py")
spec = importlib.util.spec_from_file_location(
    module_name, source, submodule_search_locations=[]
)
module = importlib.util.module_from_spec(spec)
sys.modules[module_name] = module
spec.loader.exec_module(module)

I’ve also added a copy of the typing_extensions library to the my add-on’s lib folder (the same directory as the openai module) because whenever it’s not there, I run into a ModuleNotFoundError: No module named 'typing_extensions' error.

While this fixes issues related to my add-on not finding the openai and openai not finding the typing_extensions module, I’m still unable to use the openai package due to the following error:

Anki 23.12.1 (1a1d4d54)  (ao)
Python 3.9.15 Qt 6.6.1 PyQt 6.6.1
Platform: Windows-10-10.0.22621

When loading addon-root-folder:
Traceback (most recent call last):
  File "aqt.addons", line 245, in loadAddons
  File "C:\Users\qpmch\AppData\Roaming\Anki2\addons21\addon-root-folder\__init__.py", line 17, in <module>
    spec.loader.exec_module(module)
  File "C:\Users\qpmch\AppData\Roaming\Anki2\addons21\addon-root-folder\lib\openai\__init__.py", line 6, in <module>
    from typing_extensions import override
ImportError: cannot import name 'override' from 'typing_extensions' (unknown location)

Here’s the line that causes the exception: openai-python/src/openai/__init__.py at main · openai/openai-python

And the overload function I think (?) openai is trying to import: typing_extensions/src/typing_extensions.py at main · python/typing_extensions

Based on this, I’m under the impression the typing_extensions module is not being properly imported and thus referenced by the openai module. Any recommendations for how I might go about fixing this?

Add-on Structure

My add-on folder/file structure is as follows. I’ve marked the relevant files with an arrow (<-----).

├───addons21
│   └───addon-root-folder
│       │   config.json
│       │   __init__.py    <-----
│       │
│       ├───lib
│       │   ├───openai
│       │   │   │   pagination.py
│       │   │   │   py.typed
│       │   │   │   version.py
│       │   │   │   _base_client.py
│       │   │   │   _client.py
│       │   │   │   _compat.py
│       │   │   │   _constants.py
│       │   │   │   _exceptions.py
│       │   │   │   _files.py
│       │   │   │   _models.py
│       │   │   │   _module_client.py
│       │   │   │   _qs.py
│       │   │   │   _resource.py
│       │   │   │   _response.py
│       │   │   │   _streaming.py
│       │   │   │   _types.py
│       │   │   │   _version.py
│       │   │   │   __init__.py    <-----
│       │   │   │   __main__.py
│       │   │
│       │   └───typing_extensions
│       │       │   test_typing_extensions.py
│       │       │   typing_extensions.py    <-----
│       │       │   _typed_dict_test_helper.py

Resources Consulted

I’ve also consulted the following resources, which were helpful in several aspects, but did not solve my issue:

Please forgive the lack of links - I am unable to post more than 2 links as a new user.

• Google API Python Client Import Issue - Development - Anki Forums
• Best Practice for Add-ons using Third-Party Packages - Add-ons - Anki Forums
• Best practice for imports - Development - Anki Forums
• Add-On Structure - Development - Anki Forums
• Is it possible to use external modules with addons? - Development - Anki Forums

Thank you for reading

Please forgive me if this ends up being a simple solution - it’s entirely possible I just don’t have a great understanding of python importing works.

In any case, I’d be extremely grateful if someone could point me in the right direction!

[Quinnium]

Hi all,

I’m happy to report I solved my issue (funny how writing things out and a good night’s sleep helps!)

Solution

I discovered I could use pip to install the openai library (and its dependencies) directly into my add-on’s lib folder using the -t flag:

pip3 install openai -t ./src/add-on-root-folder/lib

Optional: you can also use poetry combined with requirements.txt for effective package and environmental management:

poetry init 
(answer setup questions and generate pyproject.toml)

poetry add openai

poetry export -f requirements.txt --without-hashes > requirements.txt 

pip install -r requirements.txt -t ./src/anki-root-folder/lib

where ./src/anki-root-folder/lib is the path of the folder where your add-on’s dependencies will live.

You can then use one of the following methods to add this dependency folder to sys.path, so your add-on’s dependencies will be properly imported by Anki when Anki is launched:

Method 1

Add the following code to your add-on’s __init__.py:

import sys
import os

dep_dir_nam= "lib"
sys.path.append(os.path.join(os.path.dirname(os.path.abspath(__file__)), dir_name))

import <your module>

Method 2

Add the following code to your add-on’s __init__.py

import sys
import os
import importlib

module_name = "<your module>"
dep_dir_name = "lib"
ADDON_ROOT_DIR = Path(__file__).parent
sys.path.insert(0, os.path.join(ADDON_ROOT_DIR, dep_dir_name))
module_path = os.path.join(ADDON_ROOT_DIR, dep_dir_name, module_name)
source = os.path.join(module_path, "__init__.py")
spec = importlib.util.spec_from_file_location(
    module_name, source, submodule_search_locations=[]
)
module = importlib.util.module_from_spec(spec)
sys.modules[module_name] = module
spec.loader.exec_module(module)

import <your module>

Hope this helps for any new add-on developer’s out there.