Well, first of all, you have to write your first work. And preferably in salable form. But if you don't want to end "and they all lived happily ever after" and mean it, you need to set up possibilities in the first novel. The real trick is doing so without interfering with closure in the first work. Even in The Lord of the Rings, with its cliff-hanger endings, The Fellowship of the Ring and Two Towers both ended with drastic changes; the world of each novel had resolved, even to even nastier problems. Still more so when each work is supposed to be a complete story.
And, OTOH, having the story lead to happily ever after except that in the very last scene, we see that the apparently dead villain is alive after all, is a cheap trick that screams that this is not the logic of the story but a sequel hook.
On the whole, the best way to set up a sequel is in the middle, not the end. The hero outfoxes a gatekeeper and leaves him screaming, "I hates him forever!" while he makes his get-away. The heroine foils the spell the Necromancer casts but doesn't kill him. The old hermit in his library tells them that the prophecy goes on but that this is what faces them now. The challenges are laid out to the heroes even though they do only one this book. The defeated rival goes off to sulk. The romantic rival is a perfectly good guy, just not suitable for the heroine -- or rather, this heroine. Even if we end with the villain's brother swearing revenge, it's best to hint that he had a brother who would care that he died, or regard his death as show of weakness that would reflect poorly on him, or finally see the chance to show himself better than his brother.
On the other hand, you want to sufficiently tweak these set-ups so they don't interfere with closure, that this story is done. I read a novel once in which a hero and heroine left an inn, and it blew up. Three friends of theirs, including a cousin of hers, would have died -- if they had been in the inn. They can't find out either way and have to go off on their mission. And at the end, they didn't get mentioned. The author wanted to leave the issue open for a possible sequel, but I would have been happy if they had just mentioned that they still remembered and wondered. (It was not so happy an ending that that would have jarred.) Another novel held an ending where the last scene was the hero of the book being summoned before his superiors to be told of a theft and sent to recover the goods and kill the thief. My reaction was, Good, they noticed. A conclave at this location had been disrupted by the machinations of an evil sorcerer, they had had to fight desperately against his creatures, and the theft had not been detected during all the uproar. To see him sent off made it clear what the next book would be about, but in this book, it made it clear that the thread has not been lost.