I would hope the 'neutral word' was said at the same volume/rate/tone. Otherwise, yeah, it would definitely be a factor. Just like we can make ourselves angry with our thoughts, there must be a large difference in adrenaline released depending on how you emote.