Yeah, anything above f/8.0 tends to be a bit too soft for me at 600 mm. At the lens' lowest focal length of 150 mm, using f/6.3 tends to yield the sharpest results. Also remember that the larger the aperture, the smaller the number (f/6.3 is a larger aperture than f/8.0).
The effect is greatly exaggerated by the focal length, yes. The background is mostly trees that are like 20 metres away.
And yeah, photographing birds can be a bit tricky. So fast!