In terms of language you are correct. But in terms of SI usage it seems to me OP is expressing it correctly. The SI unit prefixes have a name, a symbol and a multiplier. The prefix is a concept that encompasses all three of those attributes. So "kilo" is one way of identifying the 10^3 unit prefix, but the name kilo is not the prefix itself. It's just the name we use to refer to it. And the symbol k in km is certainly the unit prefix portion of that unit of measure.