Especially primary source. But secondary sources that focus on odd corners of the era -- even those you think have nothing to do with your story -- may shed unexpected light.
For one thing, overviews of the era have to generalize; it's the nature of the beast, especially if you are dealing with The Victorian Era, say, or worse, The Middle Ages. This smooths out the roughness and craggy corners that are the chief source of vivid local color.
Metaphors and analogies, in particular, can be useful even in the oddest works. For one thing you can take them at face value, since they are tangential to the writer's point, and were chosen to illuminate, not to argue about. When St. Augustine compares a particular theological point to a betrothal, which lasts a couple of years so the bridegroom will not devalue his bride by getting her so easily, you may not agree with the theological point, but you are quite safe saying that in the Roman Empire, betrothals lasted a couple of years, and at least some people said that was the reason why.
Other oddments turn up. As when I was reading through a book about the Pre-Raphaelites, and ran across comments on The Order of Release.
Which, when it was released, got some reviews. Such as
The subject is simply that of a wife, with child in her arms, coming with an order of release for her husband, who has been taken in the Civil Wars. The husband, overcome with emotions, and weak from a recent wound (his arm is in a sling), can but fall upon her neck and weep; moan, "firm of purpose," sheds no tear; she has none to shed; but her eye is red and heavy with weeping and waking; and she looks at the stern and unconcerned gaoler with a proud look, expressing that she has won the reward for all her trouble past
Well, actually, not only all the other reviewers, but the general public, agreed with the second. They had had to put up railings at exhibitions to protect popular paintings, but this one had to have a guard.
Of course, a history of Pre-Raphaelites is not about the position of Victorian women, which gives that book an advantage. I have seen a book about Victorian women discuss the painting solely in terms of the first review here. . . . Even for secondary sources, being tangential to the main issue can be useful