Of course, then you get the books where the characters have read such books, and you notice they have a strong tendency to be comic. Even to have the characters appear to know they are not characters, but people. All right, you can get away with someone's saying, "I'd ask what else could go wrong, but I don't want to know" because even in real life, people avoid tempting fate. But warning people not to produce family photos, especially when they've just been introduced -- a princess complaining that the evil fairy actually showed up at the christening but was appeased by her family and didn't curse her -- thinking that the beautiful young woman who shows up has to be the mad scientist's daughter, there to rescue him -- there is no quicker way to announce, "This is Not Real!" and while it may be funny, it's not likely to move people.
The most effective technique I've seen is to have the characters aware of the genre, or at least genres very like it, written either somewhat worse or somewhat differently than the story they are in. An occasional thought that if it had been like in the pulps, there would have been a convenient crowbar about to knock a bad guy over the head with, goes far to note that you can't, in fact, rely on the fiction as a guide.
Or, the characters can observe that they do not know what roles they are cast in. John Barnes's One For The Morning Glory perhaps does it best; they know they are in a story, but when Calliope objects that stories don't end this way, Amatus counters that they do not know how their story will ened; after all Sleeping Beauty features a hundred princes who die on the thorns outside the castle.
Though there is a little problem that behavior that would be natural comes across as cliched. In The Incredibles, what would be more natural than Syndrome telling Mr. Incredible his plan? He wants respect. To get respect, he has to show that his plan is worthy of it.