Ask, don't assume you can do whatever you want.
Other than that its like talking to someone any other time, treat them how you'd like to be treated.
So turning down is as simple as "No thank you" or "I'd like to keep doing this instead" or whatever depending on what's being asked.