Yes, I agree, by defoliating so late into flowering, this has handicapped your plant.
Fewer big, green, healthy leaves = less energy production = less stored vital growth elements = smaller and slower growing flowers.
Only yellow leaves should be removed, for by the time they are yellow, they have served their purpose.
The big fan leaves are the energy "factories" of your plant - less leaves, less energy, therefore, smaller flowers. Cannabis has not evolved to perfection by getting rid of healthy green leaves. Try leaving all but the yellow leaves on your next grow. You may be surprised by how fast and how large your plants flowers grow by not being handicapped. Those "shaded" bud sites actually need the big fan leaves to grow out of the shade. The tiny "bud leaves" can not provide anywhere near the amount of energy of just one big, green, healthy fan leaf. Hope this helps..... Organoman.