can't see anything for certain in either of those two images.
if just a "banana / nanner", there's a chance it doesn't do much damage. If it was a fully grown pollen sac, that's a different matter entirely. you would see a bunch of 'nanners' pop out of a larger pod and that could seed your entire room.
they turn yellow when 'potent' / mature. if you pluck when green it probably doesn't release any pollen. A sac before it opens is obviously the time to pluck those.
depending on severity of hermaphriditism, you may want to chop the plant... if not too bad and only nanners could keep plucking. When i had plants nanner out, i saw maybe 1seed per 20 grams or so? give or take. i plucked when i saw but i definitely missed some too. lone nanners don't have the full compliment of capabilities and produce lower amounts of pollen that can in some cases be impotent.