“Chinese characters” means a certain writing system that is used by several languages (and not just Japanese and Mandarin, but also Cantonese, Korean, Vietnamese etc.), but doesn’t inherently belong to any one of them

Yeah, upon further consideration it seems less like a loanword being considered part of another language and more like they'd have to be called Chinese characters because that's the origin and nothing else has replaced it, apart from some variants. It's like how a lot of modern languages use the Latin alphabet, even if some add diacritics like ñ in Spanish.