There are downsides to consider...

1. Most people aren't really phased by doorbell cameras....but everyone feels a security camera pointed at them
2. The angle is generally worse for this purpose
3. Unlike a doorbell, there is likely no existing data line or even power
4. No way to sugarcoat it, security cameras kind of looks like shit. The best you can hope for is one that doesn't stick out too bad

There's options for decent-looking doorbell cameras without all the fuss. I use Amcrest, I know a lot of others use Reolink