All those leaves you removed were making the energy the plant needed to grow the flowers.
Less leaves = less energy being made = less growth...............too late now, you will just have to ride it out and harvest what you can rather than what could have been.