Custom progress updates not showing up in CollectionOp run in sync_did_finish

Anki Development
1

I’ve been banging my head on this problem where I’m running two CollectionOps back-to-back during sync and only the first one shows progress updates. I’m using the collection_op.with_backend_progress(on_progress_func) for showing progress updates. Is this some problem where the progress dialog for the first op doesn’t close because the next op begins so soon but the next op no longer has access to the dialog?

First op runs in sync_will_start and the second in sync_did_finish:

I’ve verified that the problem seems to be that the on_progress func I pass to with_backend_progress never gets called during the second op no matter how long the op takes, which would explain why no updates are shown despite the progress_update_def having stuff to show. But why doesn’t the function get called anymore and how do I fix it?

2

From what i understand, there are only a handful of backend ops that actually report progress:

Apart from sync_will_start preceding a sync, i don’t see the addon doing a collection op that would report progress :thinking:

3

Sorry, I wasn’t clear. I’m providing my own custom progress updates in my addon:
progress_updates

The problem is that my updates aren’t being communicated to the progress dialog in this one edge case of the two ops running back-to-back during sync.

4

I made a simple test addon that produces some progress updates. I found that running the test op through the menu shows the progress updates while running the same op in either sync hook shows no progress updates.

Test addon __init__.py 
import time
from aqt.gui_hooks import sync_will_start, sync_did_finish
from aqt.operations import CollectionOp
from aqt.progress import ProgressUpdate
from anki.collection import Progress, OpChanges
from aqt import mw
from aqt.qt import QAction


def run_op(title: str):
    start_time = time.time()
    elapsed_s = 0.0

    def op(_) -> OpChanges:
        nonlocal elapsed_s
        while elapsed_s < 5:
            elapsed_s = time.time() - start_time
            print(f"{title}: elapsed_s: {elapsed_s}")
            time.sleep(0.5)
        return OpChanges()

    def on_progress(_: Progress, update: ProgressUpdate):
        nonlocal elapsed_s
        elapsed_time = time.strftime("%H:%M:%S", time.gmtime(elapsed_s))
        update.label = f"{title}: {elapsed_time}"
        print(f"elapsed_s: {elapsed_s}")
        update.value = int((elapsed_s) / 5 * 2)
        update.max = 100

    return (
        CollectionOp(
            parent=mw,
            op=op,
        )
        .with_backend_progress(on_progress)
        .run_in_background()
    )


def first_op():
    return run_op("First operation")


def second_op():
    return run_op("Second operation")


sync_will_start.append(first_op)
sync_did_finish.append(second_op)


run_op_action = QAction("Run progress test op", mw)
run_op_action.triggered.connect(lambda: run_op("Op through menu"))
menu_for_helper = mw.form.menuTools.addAction(run_op_action)
5

I’m not fluent enough on this topic, although I imagine it’s because you can’t have the same run_op operation in parallel, and the sync_will_start and sync_did_finish sync events finish before your first operation.

Have you tried the following approach?

import threading

first_op_done = threading.Event()

def first_op():
    first_op_done.clear()
    return run_op("First operation").success(lambda _: first_op_done.set())

def second_op():
    first_op_done.wait()
    run_op("Second operation")

sync_will_start.append(first_op)
sync_did_finish.append(second_op)

I feel like something like this could work, since second_op always waits for first_op to finish, regardless of whether sync_did_finish happens first, without blocking the main thread.

6

Thanks but that this seemed to only cause the second op to hang on the first_op_done.wait() call.

I think the problem may be with the sync hooks themselves doing something different compared to running an op through a menu. I tested just running a single op, either in sync_will_start in or sync_did_finish, and the custom progress updates don’t get shown in either case.

7

What about updating the progress dialog within the background operation?

def op(collection: Collection) -> OpChanges:
    nonlocal elapsed_s

    while elapsed_s < 5:
        elapsed_s = time.time() - start_time
        elapsed_time = time.strftime("%H:%M:%S", time.gmtime(elapsed_s))

        aqt.mw.taskman.run_on_main(
            lambda: aqt.mw.progress.update(
                label=f"{title}: {elapsed_time}",
                value=int((elapsed_s) / 5 * 2),
                max=100
            )
        )

        time.sleep(0.5)

    return OpChanges()

With:

CollectionOp(
    parent=mw,
    op=op,
).with_progress().run_in_background()
8

Changing to use mw.taskman did fix this. However, this isn’t how CollectionOp is supposed to be used. I suspect there is a bug in the sync_did_finish hook. I’ll do some testing and see, if the test addon using with_backend_progress(on_progress) worked in a previous Anki version.

Test addon with working custom progress updates using mw.taskman instead

import time
from typing import Optional, Callable
from aqt.gui_hooks import sync_will_start, sync_did_finish
from aqt.operations import CollectionOp
from anki.collection import OpChanges
from aqt import mw
from aqt.qt import QAction

def run_op(title: str, on_success=Optional[Callable[[OpChanges], None]]):
start_time = time.time()
elapsed_s = 0.0

def op(_) -> OpChanges:
    nonlocal elapsed_s

    while elapsed_s < 5:
        elapsed_s = time.time() - start_time
        elapsed_time = time.strftime("%H:%M:%S", time.gmtime(elapsed_s))

        mw.taskman.run_on_main(
            lambda: mw.progress.update(
                label=f"{title}: {elapsed_time}", value=int((elapsed_s) / 5 * 2), max=100
            )
        )

        time.sleep(0.5)
    return OpChanges()

col_op = CollectionOp(
    parent=mw,
    op=op,
)
if on_success:
    col_op = col_op.success(on_success)
return col_op.run_in_background()

def first_op():
def on_success(_):
print(“First operation done”)

run_op("First operation", on_success)

def second_op():
def on_success(_):
print(“Second operation done”)

run_op("Second operation", on_success)

sync_will_start.append(first_op)
sync_did_finish.append(second_op)

run_op_action = QAction(“Run progress test op”, mw)
run_op_action.triggered.connect(lambda: run_op(“Op through menu”))
menu_for_helper = mw.form.menuTools.addAction(run_op_action)

Updated code in my actual addon in case anyone is interested in custom progress update handling: