Buds will become fluffy & underdeveloped because of the bugs, had this every time with mine…
You can mix water with dishwashing liquid & spray whole plant, will kill all the bugs but won‘t prevent new ones from occuring.
Also your Fan leaves will die off so photosyntheses is decreased.
I don‘t think you will get a good quality product out of this plant.