Clouds.
Without the power grid to back them up, solar panels are only as reliable as the weather. A week of overcast skies will drain everything you have in your batteries. No output during the night either. There's also the problem of theft, vandalism, natural wear, and the difficulty of finding replacement parts.
If you have solar energy and a way to store it, it should be used for more vital applications, like lights, cooking, and refrigeration.