Once there are 98-99% brown pistils/hairs, you will be entering the harvest zone.
As it is now, it is still too early to harvest..........you probably still got another 2 weeks to go (roughly).
Don't worry about the leaves, it is somewhat normal for them to go yellow at this late stage of flowering and there is nothing you can do this late in the plants life to "fix" anything.
The plant is not dying, it is just taking goodness from the older leaves to use for growing the flowers.