you should definitely know what type of authentication you use (my opinion) !! the agent can hold the key forever, so if you are just not asked again when connecting once more, thats what the agent is for. however its only in ram, so stopping the process or rebooting ends that of course. if you didn't reboot meanwhile maybe try unload all keys from it (ssh-add -D, ssh-add -L) and see what the next login is like.
btw: i use ControlMaster /ControlPath (with timeouts) to even reduce the number of passwordless logins and speed things up when running scripts or things like ansible, monitoring via ssh etc. then everything goes through the already open channel and no authentication is needed for the second thing any more, it gets really fast then.