The problem isn't that IR light, and whatever else, are getting in. Presumably the solar panel captures it. The issue is that any light it collects becomes heat. Solar panels get really hot, because they're absorbing as much light as possible. This should get slightly less hot, because it's only absorbing some light, and letting some through, but any heat coming in will become heat, because none is reflected. Any that's reflected won't become electricity.