If it's only a few minutes here and there, it's likely not going to be a big difference. Nature doesn't turn the sun on and off at the same time everyday. But invest in a timer so you can set it and forget it. One less chance of ruining a 3 month crop because you forgot to turn lights on, or wasn't able to turn them on.