This happens because the filenames are extracted with a regex. It could be fixed with a more sophisticated parser, though I suspect it doesn’t affect many users.
Yeah, it should be quite a rare occurrence. It still might result in the occasional accidental loss of media, since images pasted e.g. from the web may have the > symbol in their alt attribute (happened to me).
Would using a more sophisticated parser slow things down?