Should I harvest the seed or let it stay?

Yes as organoman said this is a terrible way to go about making seeds.. You've made Hermie seeds and you'd be better to just use Regular seeds then using seeds of a hermied plant.. Not sure what your trying to do really.. Waste your time? Learn about hermaphroditism in Cannabis? either way a good little example for others to see what happens when you push the harvest window too far.

You can probably take the seed out. It looks ready. Toss the banana. Seed should be dormant, looks developed. Good luck and enjoy growing it again. 🌲🌲🌲👊

when they are mature they will fall on their own or with a gentle shake. Happy growing.