Yellowing - something to worry about?

When should one start to worry about the yellowing leaves? Until which point is part of a plant’s normal aging and when does it become sign of some defficiency?