Without being there it's always a guess. But putting "You know" at the front like that is pretty damn explicit. She wasn't just making a random comment. That's like movie trope obvious.

There are always outliers though, she could of had a brain fart, or he could have said something to make her believe he felt she was working late.