But it's as dangerous as any backstory that tempts you into using the past perfect, because it's not the present day.
Flashbacks, sometimes, work, but they can be even more dangerous when it's about events that didn't even happen to the characters. And there's always the question, if you don't use an omniscient narrator, whether the characters would even notice what clues there were to history, and whether anything would remind them, and if they even know anything -- oral history is not reliable after about a century and a half, under the best of situations.
Sometimes the flashbacks are a clue you need to start earlier -- in a prologue if nowhere else -- but that can be problematic if there's a lot of it. And if it's local-color history, it won't work in a prologue, which signals that "This is important!"
Sometimes you can push it back so that the story is well begun before the history comes in, but sometimes that makes it unintelligible.
The joys of writing