The mushrooms/toadstools usually grow on a piece of rotting wood or bark, which are typically found in potting soil mixes.
Just pull them out as they appear, they won't harm your plant, but I would strongly advise against eating them yourself!!
Hope this helps,...........
Organoman.