Yes, there are disadvantages to starting bloom fertilizers too early, as cannabis still needs adequate nitrogen during the early flowering phase.
What I have been doing for 25+ years and which works really well for me, is to continue with grow nutrients until there are small "budlets or tufts of pistils, then change to half grow and half bloom nutrients for the next 2-3 weeks, depending on flowering speed, then and only then finally changing to straight bloom nutrients through until maturation.
This seems to produce the biggest buds on plants that don't completely exhaust themselves by maturity and end up a sickly yellow colour.
Any extra P and K are really only needed during the middle third of flowering, so giving extra P/K too early is not that beneficial.
Without adequate nitrogen in early flowering, your plants will start cannabilizing itself way before it is fully mature, leading to metabolic upset that I believe impedes flower development and overall size and also degrades trichome health.
Hope this helps, Organoman.