The midrange zoom choice is often hard. You have two stellar, fast prime lenses and no zoom on this planet is gonna optically beat them. If you can bear to read through my long-winding rambling, there's a suggestion in the end.
From where I am standing there're three options:
1. slow zoom with VR I had the
24-120/f4 VR for one trip to Tuscany in Italy because I wanted to go light. It is rather sharp in the wide end (24~35 mm) and doesn't really need much stopping down to be decent. That is a definite plus. And the VR is excellent. The lens has a worrisome look, when you start zooming... one plastic tube after another plops out of the barrel. I don't know about the possible longevity of that structure, but it feels a lot more solid than it looks.
The smaller, lighter and cheaper
24-85 f/3.5~4.5 VR is optically about equal to the above lens. Note, that there's three different 24-85 from Nikon and the VR is the latest. It also shares the similar telescopic plastic tube structure as the 24-120.
(Funnily enough, the absolute best of the 24-85 lenses optically by Nikon is the oldest AF 24-85 D f/2.8~4, but that has been discontinued long ago. It also has the slow and less accurate screw-drive autofocus.)
2. fast zoom f/2.8
The 24-70/2.8 or 24-70/2.8 VR are the best that Nikon offers in the midrange zoom category. Intended for journalism, travel and walk around use, but are pretty big and heavy for that. Even unwieldy and cumbersome, I'd say. I'm not much of a zoom guy myself, I only have 4 zooms at the moment and one of them is the predecessor to these two, namely the AF-S 28-70/2.8 that I never felt necessary to upgrade. I don't use it much, though.
3. even faster prime
You have two stellar primes. How about adding 28mm f/1.8G to the mix? Superb lens. There's really nothing else to say. Wipes the floor with the 24-120 at 28mm and is 2 and 1/3 stops faster so doesn't need VR.
Conclusion
My personal conclusion follows, take it for what it is worth
I'd pick the best of both worlds. I'd trade the 24-120 to a 24-85VR
and 28/1.8G. There you'd have the versatility of a VR-equipped zoom lens in a small package, and a top quality prime lens for those images that matter. Win win.
After the trip to Tuscany I traded the 24-120 up to a Nikkor 58/1.4G, but since you are already covered in the 50ish range, the 28mm might be the best choice.
Good luck.