Debug works, Run does not
Got it!
The processor was not refusing to startup, it actually did run!
I didn't "startup" the SPI2 (audio output) by giving it a first value (SPI2->DR = 0x00). So the program got stuck waiting forever for the interrupt caused by completion of the data send operation. And there's no way to detect that when in Run mode, as there's no "pause" to go check what's happening. I had to do some "led debugging": changing a led value, and then moving this step by step further into the code. Until the led change didn't happen anymore, meaning that the program got stuck just before.
No idea why it did startup in Debug mode however... Very strange!
I'm getting more and more the idea that STM32 is absolutely not suited for assembler programming...