I’d say it’s certainly a risk. Pollen is dispersed by the wind, so if you are close enough and down wind. It seems you didn’t have this issue with the current grow - which looks awesome btw!
I would suggest growing your plants on the other side of some kind of solid obstruction — like you house or other building. This would at least decrease decrease the risk. Or as others have suggested you could grow under cover (plastic/row cover)... mind you there’s a cost to this of course.