No, just like how apples do not only grow on the sunny side of trees, more light directly on buds does not necessarily mean bigger buds. Buds are basically fruits that grow idependantly of light exposure.
Genetics and thriving in suitable conditions is the main determination of bud size...basically, healthy plants, healthy fruits.
Defoliating in the hope of more light hitting buds so that it will result in bigger buds is misguided. More leaves means more energy being made which means healthier (and bigger) fruits. Energy is finite and plants can not determine where the energy gets used, as in "focussing" it in particular areas...however, apical dominance will determine where the plant "senses" that the energy is required the most, this is why having a flat, even canopy is seen as a desirable goal in indoor cultivation.
Leaf removal reduces energy production, so there can be no re-direction or focussing of spare/ extra energy elsewhere.
So, in order of importance...
Genetics
Maximum leaf coverage
Light quality/quantity.
Nutrition.
Also, as an aside, big buds does not always mean quality....quality is entirely related to trichomes, whether those trichomes are on football sized buds or golf ball sized buds, does not matter.
Otherwise, 0011 has also provided good reasoning.