Every book in a series should give a sense of conclusion regarding the characters in the plot. The key conflicts should be relatively resolved as they pertain to the characters.
Now, the universe the book is placed within - there can and should be plenty of mystery to it. The first book in the series should be a setting of the stage or a portrayal of the universe where things are going to happen. The story of the first book in a series is usually just a vessel for the structure of the universe. The books that follow should expand upon that universe and introduce deeper plot conflicts (that may take more than one book to fully resolve - but they should still end with a sense of closure even if that closure is not a full resolution).
Take the Harry Potter series, for example. While it isn't necessarily the greatest of literary works - the first book introduced the concept, let you in on what was what, and left you with an interesting story that was neither incomplete nor closed to the telling of 'the rest of the story.' The second book was also a self-contained story. While I didn't follow the entire series - it is my understanding that this pattern continued. Each book resolved the majority of its own plot, with only a few plots carrying through the entire series.
Now, compared to books like Lord of The Rings - there is far less plot resolution between the books of Lord of The Rings - and both were popular works that many people enjoyed within the same genre. Even so, each book 'ended' at certain points in the story. You may not have known what the 'grand resolution' to the plot would be - but it didn't end with many of the key characters in absolutely dire straights.
