mushrooms grow from spores... spores got into your soil.
Unless you can identify what they are, you likely want to remove them...
if you can say with certainty they provide some sort of symbotic benefit, leave them. more likley they are jsut competition for similar resources, though.