You're being too broad. Gotta specify what you're doing with coffee at that point, and a slight expansion.

Basically, when bringing it up make it sound more interesting and mysterious, with the potential of a follow-up story.

For instance, people often ask me what brought me to Los Angeles. I tell them, "it's lady Gaga's fault". Hooks 'em every time.