Hey Growmie,
Yes, you should go ahead and remove them because if you don't the plant will focus on repairing them instead of focusing on growth. Hopefully, I answered your question and remember ..."Happy Growing" and "Free your mind one puff at a time"