Some rotting stuff around the buds is always bad.
I had to throw away around 50% of my outdoor plant yield this season :(
In this case it's always better to be safe than sorry.
I would remove all the infected parts and keep a close eye on the plant.
Nothing is more frustrating than a little bit of bud rot spreading and ruining your whole plant, or even worse, your entire yield when stored together with healthy buds.
Good luck!