Stop rummaging through soil where you plant things -- that's counterproductive.
Next time, don't do that. The plant will take care of it with no problems. If the seed shell is on what sprouts from ground, let it prove it cannot pop it off on its own.. moment you see the cotyledons gree inside, gently pry it off. Some water can make it softer. Use a pipette -always good to have some of those around for measuring things out anyway.
I know they say to plant it this orientation or that... but if you plant it so the tap goes straight down unimpeded, you'll more often have that shell on the terminal growth. plant crown horizontal -- provides a good lever and the soil will tip the shell off slowly. the plant senses gravity and uses it to determine where to send the root and shoot. you don't have to worry.
keep it simple.. plant it.. forget it.. make sure it has light within 24 hours in case it's a fast sprouter to avoid stretch.
Feel like digging around? Don't do that! You'll cause 10x more problems then you solve, if you solve any at all.