[AnkiMobile] Auto-advance vs. html audio element?

[julienvincenot]

With some help here and on Reddit, I got much closer to my little goal which is to emulate the behavior of the Glossika app with my own soundfiles + subtitles in Anki.

1. I discovered the Auto-advance feature the other day, which has an option to “wait for audio”

2. I have the code bellow that allows to play a given mp3 n times (specified in a {{Repeat}} ) with a specific playbackRate (specified in {{Speed}} ). With the little modulo 2 mechanics I’m even able to alternate unmuted and muted audio, to give myself enough time to repeat.
   This way for instance, I can review older cards only a couple times each time with normal or faster speed, and spend more time on newer cards with slower speed and 4-5 repetitions each. So just the way I would do in Glossika, but actually smarter.

This repeating / muting / playback systems works great when I preview the card on both Desktop and AnkiMobile.

But there is another problem I cannot figure out… (see below)

<div style='font-family: Hoefler Text; font-size: 27px;'>{{Hanzi}}</div>
<div style='font-family: Hoefler Text; font-size: 17px;'>{{Pinyin}}</div>

<audio controls id="player">

<script>
    {
        const audios = Array({{Repeat}} * 2).fill(`{{Audio}}`);
        const player = document.getElementById("player");
        let index = 0;

        player.addEventListener("ended", () => {
            if (++index < audios.length) {
                player.src = audios[index];
              player.muted = index % 2;
              player.playbackRate = {{Speed}}
                player.play();
            } else {
                // If there are no more audio files, make it possible to play from
                // the first file again when the play button is clicked.
                index = 0;
                player.src = audios[index];
            }
        });
        // play the first audio file

        
        player.src = audios[index];
        player.playbackRate = {{Speed}}
        player.play();
    }
</script>

For some reason I can’t seem to reconcile these two solutions. Either on Anki Desktop or Mobile (on iOS), turning on the Auto-advance just jumps to the next cards instantly, without waiting for Audio.

@dae is Auto-advance only aware about the default audio tag (drag and drop, with [Audio:foo.mp3] in the card) and not html audio elements way?
Do you see any way around it? I understand from many older posts here that the default one has limitations, but maybe that’s not the case anymore?

Otherwise, would this justify to introduce a little more flexibility on orchestrating time with the Auto-advance, or some way to do this in the card definition (some kind of “wait” and “force next” events) would be very relevant for language practice.
For instance, after my 3 audios (and 3 muted audios interleaved), I would probably like to add 1-2 seconds to breathe before the next card.

Thanks in advance for your help !

[julienvincenot]

is the title clear / category appropriate?

Not sure if that’s a bug or a feature now

[sorata]

It’s in the right place I believe (and dw, dae never really misses anything when he goes through forum posts).

If I’m correct, the issue you are facing is Auto Advance not waiting for the audio to finish, and you suspect that’s because you’re using a JS script+HTML to play it instead of the usual way?

To make sure, have you tried and see what happens without the JS?

[julienvincenot]

Thank you for your reply ! Yes absolutely, sorry if that wasn’t clear :
the Auto-advance worked by default (with “wait for audio” on) as long as I just used audio directly dragged into the card field, i.e. where the filename has a [sound:filename.mp3] around it, which automatically replaces it with the default player (round play button) in the card.
In that case, as soon as I enable Auto-advance with the shortcut, my cards just run automatically and the sound files are heard in sequence automatically (hands-free), as expected.

But here the problem :

• I have no control on how many times each file is played — unless I copy manually the [sound:filename.mp3] object n times in the field (in that case, Auto-advance just waits for all the files to finish)
• there is no way to control of the playback speed.
• there is no way to have every other repeat muted for speaking out loud (I need that exact same length for speech, potentially stretched with individual playback speed for hard sentences)
• and potentially no way to add some extra silent time before the next card for breathing

That’s as far as I know, perhaps I missed a way to do these things with the standard player?

Thanks !

[sorata]

@julienvincenot

I have no control on how many times each file is played — unless I copy manually the [sound:filename.mp3] object n times in the field

I am not a programmer myself, but I think you can get JS to take a sound file and insert it into some empty divs using the default [sound:filename.mp3]. Then you don’t need to manually change things up.

@julienvincenot

there is no way to have every other repeat muted for speaking out loud (I need that exact same length for speech, potentially stretched with individual playback speed for hard sentences)

Maybe you can use a silent audio file to play and again have some divs like <div id="target"></div> and get the JS to insert them there depending on how many times the real sound is going to play.

[jhhr]

is Auto-advance only aware about the default audio tag (drag and drop, with [Audio:foo.mp3] in the card) and not html audio elements way?

That’s right, only [sound:...] tags are automatically played.

[jhhr]

I think you’re going to need to turn Anki’s own Auto Advance off and instead implement auto-advancing functionality with javascript where you include the condition of waiting for your audio to finish playing before automatically proceeding.

Using the basic code for auto-advancing (by Keks here)

Since you already have javascript for playing your audio in exactly the way you need, I think what you’d do is call the answer function buttonAnswerEase*/ pycmd('ease*') in your player’s “ended” event like this

        const audios = Array({{Repeat}} * 2).fill(`{{Audio}}`);
        const player = document.getElementById("player");
        let index = 0;

        player.addEventListener("ended", () => {
            if (++index < audios.length) {
                player.src = audios[index];
              player.muted = index % 2;
              player.playbackRate = {{Speed}}
                player.play();
            } else {
               // All audio played, automatically advance by answering "Good"
               try {
                  // AnkiDroid
                  buttonAnswerEase3();
               } catch {
                  // Desktop
                  pycmd(‘ease3’);
               }
            }
        });
        // play the first audio file

        
        player.src = audios[index];
        player.playbackRate = {{Speed}}
        player.play();

Assuming that the code is on your card Back. If you’re playing the audio on the Front then use

		try {
			// AnkiDroid
			showAnswer();
		} catch {
			// Desktop
			pycmd("ans");
		}

Then you’d just do buttonAnswerEase3();/pycmd(‘ease3’); on the Back side using the timer style or immediately.

[julienvincenot]

I thought about the silent audio option but that’s really not a good option for me. Potentially I would have hundreds or thousands of small sentences. That would mean for each card generating a separate file with silence but identical duration — to repeat out loud exactly with the same rhythm. It’s really overkill, but also would mean the double amount of storage for something that can be done programmatically.

[julienvincenot]

Thank you for the reply, I spent a few hours after reading your reply and tried as much as I could to make it work, but tbh I was tempted to reply immediately…

I realize I forgot to mention this in my first post but I need this to work on AnkiMobile.

There is no way for this method to work on iOS, pycmd is only accessible on Desktop (I can reproduce it here) and Android (I cannot test this).

Also most of the examples to “flipToBack” I could find in the page your mention, believe for iOS a function window.sendMessage2 allows to fake-tap the screen on iOS. But this cannot work because window.sendMessage2 doesn’t exist — it’s shows as undefined on my Anki Mobile.

[julienvincenot]

Is there a way to edit the title / first post to clarify

I’m looking for a solution that works on Anki Mobile (iOS). I need this to practice sentences “alla Glossika”, on the go and hands free. So Desktop is not an option for me

Hoping for some input of @dae on this, since this is really your domain

After the previous attempts (native Auto-advance not trigger audio, JS auto-advance not possible), do you see another way to approach this?

Thanks in advance !

[Danika_Dakika]

@julienvincenot

Is there a way to edit the title / first post to clarify

Sure, you can edit your title or your post. Or if you can’t find the edit , I might be able to do it for you.

You can definitely stop pinging Damien though. If he has something to offer you, I’m sure he’ll respond when he is available. Daily pings (4 from you in as many days …) aren’t going to get to him to respond any faster. It just means he has more pings to sort through!

[julienvincenot]

No I can’t edit it anymore for some reason

Yes sorry about that (and sorry to Damien) I didn’t mean to be too insistent. I got used to quote people’s names whenever I’m addressing them

[dae]

I’m afraid [sound:…] tags are the only officially-supported way to use audio in Anki. When you use HTML tags, you bypass this, and things like the ‘replay audio’ and ‘wait for audio’ features will not work.

[julienvincenot]

I understand, so I can only use those if I want Auto-advance, which I need for this particular purpose in my Anki mobile deck.

But in that case, is it true that the [sound:…] tags are still unequipped to do anything else than just playing sound?

By that I mean :

• repeat a given tag n times (by way of looping them)
• play some of them muted (to have silences with the right timing between each sentence)
• play with different speed

Is there any way to control all those but with the [sound:…] tags ?

As we saw all of those things are technically feasible on the mobile side, as proved by the little JS experiment I did. It’s not just ok, it works really perfectly. But they are completely useless without the Auto-advance

The only other way would be to have some JS “auto-advance” mechanics in Anki Mobile, just like Desktop and Android. Would this not be easier to expose for users? (showAnswer and buttonAnswer would be enough)

[dae]

You’re correct that the only way to accomplish those things is with JavaScript. Taps can be scripted, but please see the warning on the following link.

Trigger "Show Answer"/flipToBack in javascript

UPDATE I bought the template the has this function built-in from the the original author, and it cannot be used separately from the template because of the js being obfuscated. It is permissible to deobfuscate the js and keep it for personal use according to the CC BY-NC-ND 4.0 DEED, but given the fact that the author claimed to have the capacity to de-activate users if the number of the active devices are beyond limit, it might not be encouraged. I am sorry that I cannot distribute the js be…

[julienvincenot]

In that case, that’s a big problem because it already doesn’t work for me on AnkiMobile
(window.sendMessage2 in particular doesn’t seem to exist in AnkiMobile today at all)

Is this something that by nature is not easy to add because of Apple/iOS limitations?
Otherwise I don’t understand why iOS would be by design different from the others.

@dae

I’m unsure about exposing this functionality publicly, as I can imagine scenarios such as a deck author deciding that the answer should automatically show after 5 seconds, and then confused users will end up writing in here.

To be honest I’m a bit skeptical about this argument of confusing users. It is the responsability of people making / distributing decks publicly to explain what those decks are supposed to do to their audience.
Also, those hypothetical decks should already exist, because the functionality is already available on AnkiDroid and Desktop. Did you notice any surge in confused users? I mean, compared with regular Anki stuff…

On my hand this is something that I want for personal use, I do not plan to publish anything. And not having these triggers accessible in Anki Mobile I must say is very frustrating — especially considering they are available on every other platforms. And from here those people just look like very happy people to me

I believe those are really important missing features that limit Anki’s scope for the purpose of language study.

Time, and in particular study time is a very rare ressource especially as you get older. The only times when I can do these intensive language practice things is either : on the go (in transports for instance) or while doing chores with bluetooth headphones. Not in front of a computer.

For those reasons, would you please reconsider exposing these functionalities in AnkiMobile publicly? Many thanks in advance !

[dae]

Please try sendMessage without the 2.

@julienvincenot

Also, those hypothetical decks should already exist, because the functionality is already available on AnkiDroid and Desktop

The desktop API is not public either, so the same warnings apply. If this were an officially-supported API, I’d expect to see more adoption. AnkiDroid is run by a different group of developers, and they make their own decisions.

[julienvincenot]

@dae

Please try sendMessage without the 2.

I did change this and it doesn’t seem to work either.
window.sendMessage seems to be undefined on today’s AnkiMobile

(I’m not entirely sure of when my test function is evaluated so maybe that’s not true internally)

Here is the Front card I used to test :

<div id="my-text">  </div>

<script>

        let myVariable = "window.sendMessage? " +  window.sendMessage;        
        // Assign it to the HTML element
        document.getElementById("my-text").textContent = myVariable;

</script>


<div style='font-family: Hoefler Text; font-size: 27px;'>{{Hanzi}}</div>
<div style='font-family: Hoefler Text; font-size: 17px;'>{{Pinyin}}</div>

<audio controls id="player">

<script>

	// flipToBack reference to https://github.com/git9527/anki-awesome-select
	function flipToBack() {
		if (typeof pycmd !== "undefined") {
			pycmd("ans")
		} else if (typeof study !== "undefined") {
			study.drawAnswer()
		} else if (typeof AnkiDroidJS !== "undefined") {
			showAnswer()
		} else if (window.anki && window.sendMessage) {
			window.sendMessage("ankitap", "midCenter")
		}
	}


</script>




<script>

	{
        const audios = Array({{Repeat}} * 2).fill(`{{Audio}}`);
        const player = document.getElementById("player");
        let index = 0;

        player.addEventListener("ended", () => {
            if (++index < audios.length) {
                player.src = audios[index];
              player.muted = index % 2;
              player.playbackRate = {{Speed}}
              player.play();
            } else {
               // All audio played, automatically advance by answering "Good"
               
               flipToBack()

            }
        });
        // play the first audio file

        player.src = audios[index];
        player.playbackRate = {{Speed}}
        player.play();

    }



</script>

Here is the Back :

<script>

	flipToBack()

</script>

Here’s what I get in Anki Mobile :

I did set up my midCenter taps to show answer and Answer Good but after the playback repeats it just stays there and doesn’t flip to answer

[dae]

Please try webkit.messageHandlers.cb.postMessage(JSON.stringify({scheme:"ankitap", msg:"midCenter"}));

[julienvincenot]

Thank you so much Damien, now it works beautifully on Anki Mobile !

A few details for others who might need to do the same :

On front side I replaced the flipToBack function from my last example to :

<script>
	// flipToBack reference to https://github.com/git9527/anki-awesome-select
	function flipToBack() {
		if (typeof pycmd !== "undefined") {
			pycmd("ans")
		} else if (typeof study !== "undefined") {
			study.drawAnswer()
		} else if (typeof AnkiDroidJS !== "undefined") {
			showAnswer()
		} else {		webkit.messageHandlers.cb.postMessage(JSON.stringify({scheme:"ankitap", msg:"midCenter"}));
		}
	}


</script>

This auto flips to Back card in Anki Mobile !

Now some problem remained for me on the back card. For some reason, I couldn’t immediately re-use that webkit function. But it works if you add some delay.
(Also make sure to have your AnkiMobile tap settings set similarly to my screenshot in previous post, the mid center tap needs to be set to answer good or hard)

First I tried using the following, which works ok :

<script>
		setTimeout(() => {    		webkit.messageHandlers.cb.postMessage(JSON.stringify({scheme:"ankitap", msg:"midCenter"}));
		}, 2000);   /// change delay time in milliseconds here
</script>

It was fine but in my case, I wanted to adjust the exact timing based on the audio length of the previous card. So I just copied the code from my front card, and adapted it to only play a single time the audio, but muted.

So my back templated ended up looking like this :

<audio controls id="player">
<script>
	{
        const audios = Array(1).fill(`{{Audio}}`);
        const player = document.getElementById("player");
        let index = 0;
        player.addEventListener("ended", () => {
            if (++index < audios.length) {
                player.src = audios[index];
              player.muted = index % 2;
              player.playbackRate = {{Speed}};
              player.play();
            } else {
               flipToBack()
            }
        });
        player.src = audios[index];
		   player.muted = 1;
        player.playbackRate = {{Speed}}
        player.play();
    }
</script>

Anyway, that’s a huge lifesaver, with this I can now refine parameters for my pseudo-Glossika system, start importing my own sentences, experiment with playback parameters…

Many thanks again @dae !