What could be causing this? Should I be concerned?

Hard to tell but it look like a inch worm or cabbage looper , Although not as effective against caterpillars as a BT product, Spinosad can be a good choice, especially if your plants are also suffering from other insects like aphids, spider mites, thrips or white flies since this will attack all of them at once.