Was sending them through a room where prisoners would have been held, and not having them encounter danger. Except that there would have been some kind of guard.
Throwing in that these are the pleasant cells not only because the prisoners would be of noble birth but because they would not have been guilt of grave crimes only let me tone it down some. Having it not attack because they went in did more.
I suspect this will be something to bear in mind the entire crawl. On the other hand, I have already set up that someone mucked about with the set up precisely to keep those in a position of authority from getting prisoners out, so there's some leeway.