Sometimes a writer sits down at a computer (or typewriter or pen and notebook) and churns out a short story. And sometimes he churns out a novel. (And sometimes he churns out something that falls into the Great Unpublishable Void, being too short to sell as a novel and too long to sell as a short. But let us glide over that unhappy circumstance.) By their own nature, the writers tend to write either one or the other spontaneously. Their ideas just flow into the form.
The reason why this happens is that writers get ideas of varying stickiness.
Some story ideas are stickier than others. Indeed, they vary over an enormous range. And on the degree of stickiness depends the length of the story.
Short-short ideas are like steel bearings; nothing else attaches. And multi-volume novel ideas are like sticky burrs; yanking out just one is impossible. If not, indeed, like marshmallows in hot and sticky weather, not only impossible to to yank apart, but impossible to tell apart.
In between, ideas get stickier and stickier, pulling in more things, making the story longer. . . . Novels aren't just longer than short stories. They have more stuff in them, and this stuff has more complex interrelations merely from its numbers.