from stong winds, sure.. they probably get quite robus outside.. i see most outside growers with some sort of windshield apparatus around them etc.. enough space not to impede light from sides etc.. the wind will make the stems stronger too, so some stress is good.
pot size will definitely limit size of plant once roots can't grow further. If fairly linear, i'd guess you have plenty of room even for a 2kg yield from a veritable tree, but could be wrong.