Have you noticed how often fantasy and science fiction - speculative fiction - comes in fat trilogies? And how historical fiction is a bit that way inclined as well? That's partly because of the need for what spec fickers (rightly) call "world-building" and hist fickers (less wisely) call "the researched stuff". That's not just about the politics or logistics of two kingdoms being at war, or their technology, food or writing systems; it's also about the manners and mores of the inhabitants, the traditions, the religions, what the radicals are trying to make happen, gender relationships, psychotropic substances, and so on.
Were your novel set within living memory, in Britain, or the US, or somewhere else your potential readers know at first or at least second-hand, then in theory you could write phrases like "Trafalgar Square Tube" or "Bridezilla" or "TV Evangelist", and they'll conjure up a full set of denotations and connotation, which then become part of your readers' experience of the novel. Even you do need to think about it: I've blogged about what's going on when your storytelling relies on readers to get cultural references.
And, of course, for readers to feel the same density and complexity in life in 5th Century Athens or on the planet Zog, you can't rely on that existing knowledge: you're going to have to supply more of the cultural/physical hinterland. What's more, if, say, your story springs from the big political and historical circumstances of your world - which is so often exactly what you want to be exploring - then you may need quite a bit of that hinterland. But how much is too much?
And, of course, readers who enjoy the Otherness of historical and speculative fiction by definition want to buy into a full, substantial world: to sense that for each street the story takes them down, there's a whole village fanning out beyond it, for each moment in the chapel there's a whole history and geography of faith and heresy underpinning and overlying it that single prayer. Readers want to sense that every wardrobe or weapons store that's opened has twenty garments or guns in it; that's true even if the only ones that actually matter are the right one the character fails to pick out because it belonged to their hated, abusive grandmother, and the wrong one they do pick out, which will lead to disaster in thirty pages' time. But the reader doesn't know what those other eighteen will look like so, rather than do as cheap TV dramas do with their obviously empty suitcases, you're going to have to fill those cupboards.
Or are you? I'm reading Fahrenheit 451 at the moment, and it's startling what Ray Bradbury doesn't put in: what doesn't get explained, what doesn't get described, what actually isn't there at all. And yet I'm experiencing a vivid, wholly Other world; it's self-sufficient and convincing in how the characters struggle to operate in the only reality they know, even though that reality is conjured by the most minimal of means. What Bradbury's done, of course, is leave the spaces for my mind to fill in. If you asked me to free-write my experience of that world, it would be full of things Bradbury didn't put in it. (Which is probably why "the film of the book" - any book - always leaves fans of that book disappointed.)
In The Desirable Difficulty of Sleeve and Paint I thought more about how and why the writer might try to get the reader's imagination working harder, but of course, it's not coincidental that it's Bradbury who says, in Zen in the Art of Writing, "The artist learns what to leave out." Just because readers want to sense the presence of all those other streets and lives and faiths and garments and guns, doesn't mean that you should put them all in.
But notice that Bradbury hasn't said, "The artist learns to leave everything out"; what we must learn is to know what the reader can and can't do without, in order to create the world for themselves, hear the voices you want them to hear, and feel the emotions you want them to feel. The art is in picking which glimpses you offer the reader of those other streets, strange words, sayings and folk-tales , religions, garments and guns.
That almost certainly means you imagining or researching an awful lot more than ends up on the page, and only then picking which bits of your material will have the best and strongest effect, while taking up the least space on the page. In Yours To Remember, I thought about more on how to let those bits of data compost down into your work, and for more on how to pick the details that are evocative, without slowing up the story, click here. And finding the right kind of feedback and beta-readers is the way to grow your own judgement of what to leave out, and the confidence to do it.
The other reason that all the imagining/researching too often ends up on the page, is that we too easily feel that any map smaller, and with fewer dimensions, than the world itself, is imperfect. (Of course in strict logic it is "imperfect": some things are left out, as they must be in any map which isn't the same size and number of dimensions as the original.) When you've done all that imagining, there always seems to be another bit that could and "should" go in: another alleyway or heresy to write, another, subtly different garment which these people really did have ... That's perfectionism in the negative sense: the idea that if you don't create the perfect version, you've failed.
So, although it's important to learn what to leave out, it's just as important to learn to forgive your (nearly) finished novel for all the things it's never going to include: for all the things it could have been, for all the roads you could have taken this project down and didn't, because you chose to go another way. Unless you're utterly incompetent and have no capacity at all to change how you write something, there will always be other ways you could have written it. There is no such thing as a perfect novel in the absolute sense: there will always be avenues/heresies/weapons-stores you could write, which might be just as good, in a different way - but you've decided not to. So be it.