Just tried it again, and it worked but for whatever reason, did not use my actual timer app on the phone, and used Google Assistant. For whatever reason the timer was muted by default, so that's why I thought it didn't work at the last time I used it.