Audio auto play not work when setPlaybackRequiresGesture to True

[madcc]

Hello everyone, thanks for all your work at first.
I have multiple audios in one card, and I want to automatically replay one specific audio in a loop.
A simple

<audio id="audioreplay" src="{{audiofile}}.mp3" autoplay loop></audio>

(referred to as tag_a latter) should do the work, and it works when I uncheck Don't play audio automatically option of my deck, but the other audio files with [sound:xxx] format also are played once.
I don’t want the other audio to play, thus I check Don't play audio automatically option. However, my tag_a won’t get played.
I searched for several days and tried some js codes like

var replayer = document.getElementById("audioreplay");
replayer.play();

which also not worked.
And the error I got is Uncaught (in promise) DOMException: play() can only be initiated by a user gesture.
According to https://forums.ankiweb.net/t/is-it-possible-to-load-audio-dynamically-with-javascript/13011/6, and the newest code: https://github.com/ankitects/anki/blob/main/qt/aqt/reviewer.py#L372, when audio automatically play is off, the play() will failed since self.web.setPlaybackRequiresGesture(True).

So how to get my play() call to work? Can you give me some suggestions? Maybe self.web.setPlaybackRequiresGesture(True) should be removed in anki code?
I know I can add a button and call play() in it’s click function, but I just want to this audio play automatically when I click show answer.

Thanks for reading!

[dae]

@nwwt I believe you’re the one who originally added this code - thoughts?

[madcc]

Thanks for your reply.
I forgot to mention that I mainly use this deck in AnkiDroid instead of the desktop version, so I mainly want to have this work in Android.

[nwwt]

IDK about the specifics of AnkiDroid code, but some things to consider first:

• The setPlaybackRequiresGesture value does just that, it allows the WebView to autoplay, it does not actually cause this autoplay let alone control it.
• TMK the autoplay (“play ASAP after loading”) attribute concerns the individual media tag — multiple tags don’t coordinate any order/sequence, so they will essentially play at once. Everything else thus necessarily needs Javascript.
• More importantly, the WebView’s (HTML tag-based) audio is TMK (resp. was at the time of the commit) entirely independent of the Reviewer’s ([sound:xyz] based) audio — setPlaybackRequiresGesture does not influence the latter. So if both are present, both will autoplay simultaneously. (I believe we talked about this scenario and ended with ① just don’t mix both and ② if you do, recall the “Javascript is unsupported, you’re on your own” caveat.)

So long story short, if you want some non-trivial control:

1. Use only HTML tags, no [sound:xyz] ones.
2. Since the autoplay attribute’s behavior is too rigid, you will need Javascript.
3. In particular, you will need to write an event listener that gets appended as ended attribute to each relevant media tag at runtime — basic instructions in this answer’s first link.

Note that 3. gets complicated for all further functionality like playback controls (pause/restart, hotkeys, …) and/or the tag’s visual representation.

[madcc]

Thanks for your explanation.
I agree most of your thoughts, but it seems you misunderstand my question…Let me be clearer.

@nwwt

The setPlaybackRequiresGesture value does just that, it allows the WebView to autoplay, it does not actually cause this autoplay let alone control it.

Yes, and I use js code likes replayer.play(); to control the play. But this replayer.play(); will failed since setPlaybackRequiresGesture(True). Here the parameter is True, which means the replayer.play(); will not take effect until a user click somewhere.
I don’t want to manually click to play, and I want to use my html/js to control the autoplay.
So I think maybe setPlaybackRequiresGesture(True) this line should be deleted since the user may want to control the audio automatically play or not.

@nwwt

setPlaybackRequiresGesture does not influence the latter.

[sound:xyz] will be replaced with:

<fayin>
<a class="replay-button soundLink" href="#" onclick="pycmd('play:a:3'); return false;">
    <svg class="playImage" viewBox="0 0 64 64" version="1.1">
        <circle cx="32" cy="32" r="29"></circle>
        <path d="M56.502,32.301l-37.502,20.101l0.329,-40.804l37.173,20.703Z"></path>
    </svg>
</a></fayin>

IDK whether setPlaybackRequiresGesture influences [sound:xyz] or not.

Btw, is there something like pycmd(stop) or js hooks which can stop or clear the sounds list generated from [sound:xyz]? @nwwt @dae , then I can control to stop all the audios.

@nwwt

So if both are present, both will autoplay simultaneously.

If I turn off the deck Don't play audio automatically option, the two audios will autoplay simultaneously. If I turn on the option, the two audios will both not autoplay, and I think this option should only control the audios of [sound:xyz], but this option can influence my html/js autoplay either.

@nwwt

Use only HTML tags, no [sound:xyz] ones.

It is an option but I don’t like it… These [sound:xyz] are in complex html tags already, and I have to substitute [sound:xyz] with <audio ...> tag since I may want to click those audios manually when I review the cards.

@nwwt

you will need to write an event listener that gets appended as ended attribute to each relevant media tag at runtime

I only want to loop one audio (the pronounce of a new word). I have already tried such ended event in js, and it is not a problem.

[madcc]

I use a trick to stop [sound:xyz] audios to play from https://forums.ankiweb.net/t/controlling-display-and-function-of-audio-buttons/10234.
The code likes:

<script>
  var elem = document.querySelector(".soundLink, .replaybutton"); 
  if (elem) { elem.click(); }
</script>

It uses a click() of DOM element generate from [sound:xyz] to stop the successive audios to play.

Just like I have mentioned above, is there a more direct way to stop the audios instead of call a click()?

I tried the trick and on my AnkiDroid, and it works just sometimes.

Let’s say that I have [sound:a.mp3], [sound:b.mp3], [sound:c.mp3] in my cards. When Don't play audio automatically is off, the a.mp3 will play automatically, and a click() call will make a.mp3 play once more and stop the successive b.mp3 and c.mp3 to play. That’s cool but on some of my cards it won’t work and b.mp3, c.mp3 will still be played.

[madcc]

I am so lucky to see this issue.
The code from there makes the trick in my above post works for every cards.
Thanks the original author and I copy it here for reference:

(() => {

async function playMoney() {

  let x = document.querySelector("#audiomoney .soundLink, #audiomoney .replaybutton");
  if (x) {
    console.log("going to click money...");
    x.click();
    console.log("clicked on money");
  }

}
if (onShownHook !== undefined) {
 onShownHook.push(playMoney);
}

})();