As far as I remember, there is no built in way to prevent an audio file from playing again after flipping the card, if the Back template also has the field containing the audio file reference.
Maybe you could try one of these workarounds:
Create a new field and move all the audio files/references to it. Include this new field in the Front template only. E.g.