I wish some fantasy writers asked it more often. I read one too many book in which there are inexplicable human beings with pointy ears roaming about. And not even humans from a very distinct culture. Very little reading about in primary source material would turn up human beings who would look exotic next to the elf -- or the humans he associates with. Try distinguishing people by some cultural differences.
It applies to other imaginary races, too. I still remember the story I critiqued where the main character was a centaur. I found this out two pages in. Every now and again, the story would say, "She's a centaur" -- and never would it have the least effect on anyone's reactions, or on anything she did.