I use Prowlarr, but had to configure it in Sonarr. I don't see any option for it in Prowlarr
I was also flabbergasted when I observed this. No idea why this is not default, but you can configure "Fail downloads" = "Executables", "Potentially dangerous" in the Sonarr Indexer options to prevent these downloads. Enable advanced settings first.