It depends on which devices he had access to. If he was using your computer, he could potentially access your full history, but you would've been able to see him do it.
If he was just using your router, it would be highly unlikely he was able to review your history. However, if you're a high-value target (someone trading with bitcoins or something like that), an hostile party could pretend being from the ISP to try to infiltrate your network.