It has to do with the hierarchical social structure and laws and punishments they have.
Probably they started to pull away as mundanes started to get annoying about leveling and rule of law and trials and lawyers and no blood feuds. An uneven process -- though it would help explain the timing of why the magic went away.
Of course, with an explanation like that -- all the justifications constrain certain plots and encourage others -- but this one screams for a plot where someone tries to bring some sanity to the magical world's justice system.
Or perhaps not. Perhaps they can use magic to bring about justice. Depends on how it's set up.
The social structure is always going to be a problem if they had to deal with the modern world. The Faerie Court as a constitutional monarchy. . . .