I'd get rid of the yellow leaves if I were you... they're not doing anything for the plant (i.e., they're not helping with photosynthesis) and it will make the plant LOOK better and give more light... and no, getting rid of just a couple of non-working leaves won't stress the plant at all. Just my opinion but that's what I would do if the plant were mine!