Sitting with your plants is fine, but smoking in a confined area won't be doing them any favours.
Beware though, any over exposure to light, especially UV light, can in some cases cause skin cancers.
I would be wearing sunnies too.
Cheers, Organoman.