It's being pushed by Google as a format exclusively designed for the web. It's annoying as fuck because it just started showing up randomly and it's annoying to use for the reasons you mentioned.
Interestingly enough, the first thing it did was put a speedbump in front of is downloading an image from a Google images search. Imagine that.
I think browsers are starting to convert it locally as saving images seems to be slowly returning to some kind of normal. Maybe.