As long as the plants are healthy and growing well. Taking a few leaves off here and there won't cause damage in fact it does the opposite, some stress is good for the plant. And in your situation I would definitely remove some of the lower growth and some of the bigger fan leaves just do it bit by bit, never remove too much on the same day. And good to do it on a feed day where they get the nutes they need to recover quickly 🌱👍