I remove them before the stems outgrow the benders... by that time the stem has become rigid (there's a technical term for the process, but i can't recall). No reason to damage the vascular tissue as the stems grow thicker than the plastic allows.
As long as the stems have become rigid, they won't change much after that... they can lurch upward but will never correct the bend in a significant way.
The bend doesn't restrict anything, but plastic cutting into the stem can -- xylem is just as important as phloem and unnecessary damage is wasted resources healing it.
Based on the picture shown, those stems will definitely outgrow the plastic benders at some point.. when they start to feel tight and don't jiggle, take them off. A little scratch is nothing to worry about.