No. I am sure there is a more in depth sciency answer, but you need a mix of all spectrums during every stage of growth for the best results. The manufacturer recommends running both in Flower as well.
Those lights are ~250 true watts each, if you turn off the Veg switch you probably lose ~30-40% of those watts during an extremely important time to keep the light intensity up.
Also I bet you notice that there are some red spectrum lights on when you only have the Veg switch on, so if you turn those off you will lose that too.
Hope my answer helps! 👊