This problem is, if anything, even more prevalent when dealing with societies. Your culture has slavery, say -- mostly debt slavery, and the overwhelming number of slaves are used as household servants, since they have peasants to work the fields. But if you use the word "slave" you know that a good chunk of readers are going to switch onto "antebellum South". Fortunately, there are other terms. You don't even have to invent them: bondsman, thrall -- perhaps even serf, though you might want to watch the baggage on that one, too. Well, maybe you don't have to invent them. You might not want the baggage even with those, and then you have to invent your own. Then you really need a good ear. Otherwise you end up, from distaste for all the baggage that all the terms for "practitioner of magic" have, creating a term like "magic-user."
And magical creatures have baggage too. Wrote a story based on some Eastern European fairy tales and had it critiqued. Some of them refused to believe that the place where a dragon lived was cold. Others -- despite my having indicated its size many times -- declared that the dragon was too big to do one thing I had it do. I don't think I could change its name, but you have to deal with baggage.