Not because humans and elves can't interbreed. In fact, the four humanoid species -- humans, elves, dwarves, and nymphs -- can all interbreed freely.
It's just that all their children will be one of those species. For a child not to be one of the parents is very abnormal and occurs only in circumstances that even people of the world can recognize -- for instance, an elf and an undine whose child is conceived deep within the earth, where the elemental power of earth is greatest might have a dwarf child. But usually there is no discernible rhyme or reason in how many children belong to one parent's species, and how many to the other.
I think one character's going to be an elf who is the offspring of seven generations of human and elf marriages. And another is going to have had three human wives and over a score of children -- all human, all now dead.