Have you been able to see these white flies clustered on the bottoms of your leaves? They like to hang out there and can often be photographed. If you're mainly seeing them around the soil level and then flying out once watered that may be fungus gnats.
Either way, I have had the best success at 1) buy "mosquito bites " and water into soil to kill any possible larvae(wont harm roots or plants), 2) Dr. Bonners soap(mixed with water) basically makes an absolutely harmless and effective insecticidal soap...but you have to apply all over and regularly until gone but it's very successful. Any bug dr bonners hits is one dead flying bug. 3) you can also do a hydrogen peroxide water which helps the roots and works moderately well.
For the leaves, maybe back off the neem and then what's your ph like? The yellowing may not be completely connected to the spray. Also its very common for the first 3 sets of leaves to yellow and die off.👍