Unless they are completely yellow and looking almost dead, don't cut them off.
Your plant is taking the goodness from those leaves to help grow the flowers.
Removing the leaves too early means your plant won't have access to these pre made essential elements, meaning it will have to use energy to re make them from new, and that energy can be better utilized in flower growth.
Just because those leaves are starting to look a bit ugly, it does not mean that they are useless!