Still, limiting the spells works.
The strongest rule would be a logical connection to the bloodline. Pathfinder Playtest did some stuff to link them up -- fey bloodline to druid spells, for instance -- but when the sorcerer just draws on wizard spells, it makes no sense to make them all available.
A bard's connection would be harder to justify, but limits similar could be imposed to curb their knowledge. Perhaps it has to be learned from another if it's not an logical extension of what they already know.