You sure can!
I suggest re-amending it, though. You may need to add more aeration, as well, if you used perlite or rice hulls instead of pumice or lava rock. The best stuff i've found for re-amending soil is called Craft Blend, made by BuildASoil. It has everything the plants could possibly need and it's totally organic, too. You don't need much, just 2 cups per cubic foot of soil.
If you're afraid of pests being brought in with the soil, get your hands on some organic pesticides that can be used as a soil drench. The Amazing Doctor Zymes, Mammoth CannControl, and JADAM Wetting Agent(which is really easy to make yourself) can all be used. It really is better to be safe than sorry.
I hope this helps.