When I asked Twitter last night what I should blog about, one suggestion was "How do you know when to give up on a work-in-progress? Or when to stop and come back? Or when to re-conceptualise the project?". It was a good question, so thank you Damon Young, although I'm absolutely sure there isn't a clear-cut answer, because it's always going to depend on you and the night and the music.... Sorry, you and the write(ing) and the novel. So, I think the best I can do is suggest some things to ask yourself and the novel, in the hope that things get a bit clearer.
Are you aware that something's really, definitely not working, but you don't know what? You could try the diagnostic aid which is my post 17 Questions to Ask Your Novel.
Are you in the 30k Doldrums? When you're becalmed, almost any feverish hallucination of inadequacy - yours, or the book's - can get you in its grip.
Is that sexy new story flashing its legs at you? Is it that which is making you feel that the novel you're wedded to is drearily familiar? Are you tempted to forget your vows and go after the Other Novel? You could have a think about why that might be.
Are you bored by the central idea? (It's always possible to be bored, today, with the idea of Sitting Down and Getting On With It, but that's different.). Did you, perhaps, set out on this project because you thought you could sell it, or it seemed like a good idea at the time? Maybe your heart's not in it. A bit of drawer-time - a trial separation - may tell you if that's temporary or permanent. On the other hand, it could just be the middle-third blues, especially if you've been away from it for a while: of course Boy Meets Girl is a dull story. It's also the story of about half of all great literature; it's the flesh on the bones that makes a story compelling.
Is the novel trying to turn into something else - is it signalling that by refusing to do what you want - and you can't decide whether to let it? My instinct would always be to go with the new version, at least for a while. But you should probably do it on a new copy of the file, and keep the old one. You may, even, eventually see how you could integrate the two.
Are you trying to write a kind of thing which you, as a writer, are fundamentally not suited to? Did you think to ask your talent about that, when you were deciding whether to do for it? Or have things changed, and it no longer suits you and you it?
Have you had feedback from several quarters that are saying versions of the same thing? Is it something you know how to cure? Do you want to do this cure? If you do, keep going. If you don't, you may need time away from it before you know if it's a road you want to go down: your resistance could be absolutely right, or could just be... resistance. If it's a matter of murdering your darlings, might there be a Ram In The Thicket, to make it bearable?
Have you realised what's wrong, but don't have the technical capacity to put it right? Essentially, has your project landed you in problems you're not equipped to solve? You could try re-thinking the project, or you could try enlarging your toolkit. Or you can do what I do, which is to go away and write something else, from which you learn a whole lot of other things. Either way, at some point you can hope to realise that either you've changed enough to manage this project, or it has has changed enough to become manageable.
Have you tried it this way and that way and every which way and you've sorted out everything else that's wrong and somehow, you cannot make it work? This is a tricky one, because you've probably spent huge amounts of time and skill on it, and in lots of ways it's now very good. First, can you work out what still isn't working? (Maybe try those Seventeen Questions, if you haven't before?) When you have you need to decide what to do about it. What could you do, however drastic that it would be? Ideally, I'd suggest not going near it for a while, so that you're coming to it as fresh as you can. Then print the darned thing out, retire to the sofa with a pen and a large pot of tea, and read it through like a reader, looking solely for where and how you'd do what needs doing.There, maybe, you'll find your answer.
Do you know what needs doing, seen how, and realised that it's basically (re-)writing the book from scratch? If you're naturally persevering, you may feel incapable of throwing in the towel. But is that really the best thing to do? Are you ready, yet, to come to it absolutely fresh, and cool about the existing material? Are you too attached to certain characters, ideas, scenes? Might you do better to put it away and write something else? (which in itself will teach you things you didn't know, because the book you've put aside happened not to ask them of you so you didn't learn them.)
Only you can decide which way to go now. It might be time for something completely different, deliberately designed not to be goal-oriented. It might be time for giving the new novel a whirl, or a set of short stories... for now. Most of us are familiar with the character or place which won't go away, although it may take three or four incarnations, in different projects, before you find the right match for it and you. But when you do, it'll probably be the best thing you've ever written.
The good thing about novels, though, is that unlike spouses if you put them aside they don't wander off on their own. This novel - this project - this central idea - will still be there, if - when - you come back.