At the end of the day, majority of the use cases will be okay with either the series 9 or ultra. It ultimately depends on your need. You should pick the ultra if you want the better battery life, action button or the bigger and rugged design. You should pick series 9 if you prefer the smaller and sleeker design.
I myself would love to have the ultra's battery life, but I don't like the look and feel of the ultra (too big and just prefer the size of the regular series better), so I would pick series 9 if those are my 2 choices. However, I don't really do any long workouts or disappear from civilization for extended period, so an "all-day" battery of the regular series is fine for me.