Just watched the Moose Peterson blog video re the AF S 70-300 VR and he says clearly leave the VR on all the time so who knows.
I use it for panning, otherwise if shooting over 1/500sec, like others have said, it's not really needed--the shutter speed alone will freeze action and shake without the need for stabilization.
Leaving it on, even at high shutter speeds can cause issues, especially older VR lenses where the activation induces a lot of movement. I've grown to prefer Tamron's VC over Nikon's VR.