While there are people who are too trigger happy with the term, and a sizable gray area between cultural exchange and cultural appropriation, I do think there are cases where people cross over into objectionable cultural appropriation.
A really good example was when white American college students wore fake native American headwear with significant cultural and spiritual importance as decoration for drunken parties. I can definitely sympathize with native Americans not wanting their culture treated with disrespect.