Is there something you don't like about your D4?
How many shots do you process per session?
How large do you usually print? If you print larger than 24x36", then maybe the D810 would be a great choice, but not all of us do.
How long have you had your D4? Sometimes it takes a while to get used to a newer camera with a different layout. If you've had your D4 for a while and are used to it's button layout, you might find it hard to change just to have more pixels. Pixels are nice, but if you don't get the shot, you don't get the shot.
I don't think there is a single person on this forum that can answer the question you are asking yourself.
If I had a D4 I wouldn't even LOOK at a D800/810!
D4 too tall in bags, and too heavy. Biggest "issue" is at 16Mpx it shoots the same as my Fuji X-T1, so not enough differentiation. Speed and low light. Whereas the D810 would provide a significant step up in resolution.
Shots per event: around 800 to 1500
Had the D4 for almost 18 months.
I do LOVE the D4, but at the end of the day it is a tool not a person, so if the D810 was to be a better fit, especially at no extra cost, than it is worth considering. I agree it is a difficult and personal decision, that's why I am soliciting different people's viewpoint.
Appreciate the feedback.