I had tiny white bugs that did the same, dip and diving through the soil. I treated with neem oil, hydrogen peroxide water, tried flushing them out, drying them out and scooping them out. Eventually they left. Mine almost appeared fuzzy or many legged. I'd suggest try to scoop out what you can, possible try the sticky papers, and possibly the hydrogen peroxide water, dont do neem as you're so close to flower. You could try putting another plant inside for them to piggy back onto and get out. If you're only seeing them in the soil I wouldn't worry too much. I think mine came from using old soil/dry fertilizer in the beginning.
Best of luck.